A live interpretive layer over AI evaluation reporting. Evaluation Cards composes run data, benchmark metadata, and model metadata into one canonical record — and surfaces reproducibility, completeness, provenance, and comparability across the whole ecosystem.
The largest and most comprehensive corpus of reported evaluation results yet assembled (as of 9 June 2026).
Results are reported everywhere and in every way — but rarely with enough information to interpret them. Evaluation Cards fixes the reporting layer.
Scores live in leaderboards, model cards, blog posts, papers, and posters. With no shared standard, there is no way to compare results across sources or identify what a report leaves out.
Three organizations report GPT-5's MATH-500 score anywhere from 84.7% to 98.9% — depending on configuration that is rarely fully reported. Evaluation Cards surfaces the discrepancies and why they exist.
Evals are more than numbers — they are interpretive claims. By unflattening the model–benchmark–score triple, we recover the structure needed to answer deeper questions about what results actually mean.
Across 100,000+ records, every card computes four signals — grounded in our user interviews with evaluation stakeholders. Explore them beside a live view of the underlying data.
Evaluation Cards composes the raw data the ecosystem already produces — at scale — into a single, interpretable evaluation record.
Who ran the eval, on what model, with what generation settings — sourced from Every Eval Ever (EEE), the Coalition's standardized schema and community repository for eval run data.
The benchmark's measurement target, scoring, provenance, and known risks — drawn from IBM AutoBenchmarkCards, which extract documentation from papers and system cards.
Release date, parameter count, and weight accessibility — pulled automatically from community catalogs (hub-stats, models.dev) covering both open and API-hosted models.
Every score is one cell in a model × benchmark matrix. Read down a column — one benchmark, every model — or across a row — one model, every benchmark it has reported.
Browse the whole corpus — by evaluation or by model.
Both reader modes read identical data — they differ only in how it's displayed. Toggle between them without losing your place.
For evals researchers, model developers, and technical AI governance. Foregrounds methodology — missing fields, setup-driven divergence, and full metric configuration.
For policy researchers, qualitative researchers, journalists, and anyone who prefers narrative. Plain-language elaborations of the measurement target, central caveats, and intended uses.
Our roadmap is public and ideas are welcome — see what's next.
Aggregating the four signals across the reporting ecosystem reveals startling early findings (5 June 2026 snapshot).
Of 50,000+ records, 96.5% are missing at least one field needed to re-run them. Max-tokens is absent 95.6% of the time, temperature 93.9%. Even 95% of GPT-5's documented results lack the parameters to reproduce them.
Across all 635 benchmarks, median documentation completeness is a paltry 10.7%. Scores are present, but construct definitions, validation, intended use, and limitations are missing far more often than not.
98.2% of model–benchmark pairs are reported by only one party. Of the rest, scores from different parties diverge substantially more than half the time (51.9%).
An open, community-built initiative
contributors span institutions, sectors, continents, and specializations
Fixing evaluation reporting is a community effort that benefits everyone — and the Coalition can't do it alone. Contribute your evaluation data, earn a verification badge, and help us compare results apples-to-apples.
Report your evaluations, review third-party results for discrepancies, and earn a verification badge.
Upload your evals, setup, and config so others can find, run, and build on them.
Use Evaluation Cards in your analysis and cite us in papers, reports, and policy briefs.
If Evaluation Cards informs your research, please cite the paper:
Avijit Ghosh, Anka Reuel, Jenny Chim, et al. 2026. Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting. arXiv:2606.09809. Read it here
@misc{ghosh2026evaluationcardsinterpretivelayer,
title={Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting},
author={Avijit Ghosh and Anka Reuel and Jenny Chim and Wm. Matthew Kennedy and Srishti Yadav and Jennifer Mickel and Yanan Long and Andrew Tran and Anastassia Kornilova and Damian Stachura and Kevin Klyman and Felix Friedrich and Jeba Sania and Jan Batzner and Anoop Mishra and Eliya Habba and Yixiong Hao and Nathan Heath and Shalaleh Rismani and Usman Gohar and Andrea Loehr and David Manheim and Ruchira Dhar and Sree Harsha Nelaturu and Aarush Sinha and Leshem Choshen and Drishti Sharma and Ishan Khire and Amit Saha and Subramanyam Sahoo and Michael Hardy and Michael Alexander Riegler and Kabir Manghnani and Michelle Lin and Yanan Jiang and Yilin Huang and Asaf Yehudai and Jessica Ji and Aris Hofmann and Mubashara Akhtar and Max Lamparth and Nuno Moniz and Yacine Jernite and Stella Biderman and Zeerak Talat and Sanmi Koyejo and Mykel Kochenderfer and Irene Solaiman},
year={2026},
eprint={2606.09809},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2606.09809},
}