|
2026 |
Agents Are Systems, Not Models: Rethinking Agentic Evaluation |
Wiedmann et al. |
arXiv |
|
2026 |
Artificial Analysis Intelligence Index v5 |
Analysis |
|
|
2026 |
Demystifying evals for AI agents |
Anthropic |
|
|
2026 |
Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking |
Messing |
arXiv |
|
2026 |
Survey on Evaluation of LLM-based Agents |
Yehudai et al. |
arXiv |
|
2025 |
Epoch AI Benchmarking Hub |
AI |
|
|
2025 |
Establishing Best Practices for Building Rigorous Agentic Benchmarks |
Zhu et al. |
arXiv |
|
2025 |
Evaluation and Benchmarking of LLM Agents: A Survey |
Mohammadi et al. |
Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 |
|
2025 |
Holistic Agent Leaderboard |
SAgE |
|
|
2025 |
Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation |
Kapoor et al. |
arXiv |
|
2024 |
Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations |
Miller |
arXiv |
|
2024 |
AI Agents That Matter |
Kapoor et al. |
arXiv |
|
2024 |
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference |
Chiang et al. |
arXiv |
|
2024 |
Inspect |
Institute |
|
|
2024 |
Inspect Evals |
contributors |
|
|
2024 |
LMArena |
LMArena |
|
|
2024 |
Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks |
Fourney et al. |
arXiv |
|
2024 |
METR Task Standard |
METR |
|
|
2024 |
SEAL Leaderboards |
AI |
|
|
2023 |
Holistic Evaluation of Language Models |
Liang et al. |
arXiv |
|
2023 |
OpenAI Evals |
OpenAI |
|
|
2021 |
lm-evaluation-harness |
EleutherAI |
|