FAccT 2026 Tutorial on Every Eval Ever
Building Community-Governed AI Evaluation Infrastructure
🪧 About
Existing model evaluation results are scattered across leaderboards, papers, and technical reports in incompatible formats. This fragmentation obscures transparency, hinders progress, and disadvantages researchers, civil society, policymakers, and industry alike, especially those who can’t afford to run evaluations from scratch. Built once, shared eval infrastructure serves us all.
In this tutorial, we walk through Every Eval Ever, a community-governed open source infrastructure that unifies all evaluation results under a shared metadata schema. We then present Evaluation Cards, an interface and interpretive layer for evaluation reporting designed around practitioner needs from stakeholder interviews, and show how participants can find, compare, and contribute evaluations themselves.
All technical experience levels are welcome. If you can, please bring a laptop or tablet! 💻
📅 In-Person FAccT Tutorial
- When: Friday, June 26, 2026 · 5:00 – 6:00 PM (Canada)
- Where: ACM FAccT 2026, Montreal (in person)
- Room: Jarry Room at Le Centre Sheraton Hotel
🏛️ Tutorial Program Committee
- Jan Batzner, Weizenbaum Institute, Technical University Munich
- Sree Harsha Nelaturu, Zuse Institute
- Anastassia Kornilova, Trustible
- Avijit Ghosh, Hugging Face
- Angelie Kraft, Weizenbaum Institute
- Usman Gohar, Iowa State University
- Michelle Lin, Mila, Quebec AI Institute
- Yanan Long, StickFluxLabs
- Jennifer Mickel, EleutherAI
- Wm. Matthew Kennedy, University of Oxford
- Leon Staufer, University of Cambridge
- David Hartmann, Weizenbaum Institute
- Leshem Choshen*, MIT, IBM Research, MIT-IBM Watson AI Lab
- Irene Solaiman*, Hugging Face
📬 Contact
We are looking forward to meeting you! For any questions, please reach the EvalEval Organizing Team here.
