Conference paper · 2025
PaperBench: Evaluating AI's Ability to Replicate AI Research
| Venue | Proceedings of the 42nd International Conference on Machine Learning |
|---|---|
| URL | https://proceedings.mlr.press/v267/starace25a.html |
| Topics | ReplicHAI / Agent & LLM run-to-run variability ReplicHAI / Agentic reproduction: systems & benchmarks ReplicHAI / Cited in CHI 2027 draft |
| Tags | replichai |
| citationKey | starace2025paperbench |
| conferenceName | Proceedings of Machine Learning Research |
| extra | arXiv: 2504.01848 |
| libraryCatalog | background/agentic-systems/refs.bib, background/bib/bib-q1q2.bib, chi2027/paper/refs.bib |
| pages | 56843-56873 |
| publisher | PMLR |
| volume | 267 |
BibTeX
@inproceedings{starace2025paperbench,
title = {{PaperBench}: {Evaluating} {AI}'s {Ability} to {Replicate} {AI} {Research}},
volume = {267},
url = {https://proceedings.mlr.press/v267/starace25a.html},
booktitle = {Proceedings of the 42nd {International} {Conference} on {Machine} {Learning}},
publisher = {PMLR},
author = {Starace, Giulio and Jaffe, Oliver and Sherburn, Dane and Aung, James and Chan, Jun Shern and Maksin, Leon and Dias, Rachel and Mays, Evan and Kinsella, Benjamin and Thompson, Wyatt and Heidecke, Johannes and Glaese, Amelia and Patwardhan, Tejal},
year = {2025},
note = {arXiv: 2504.01848},
keywords = {replichai},
pages = {56843--56873},
}