Blog post · 2026
Why we no longer evaluate SWE-bench Verified
| URL | https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/ |
|---|---|
| Topics | ReplicHAI / Agent Evals / 02 Software engineering & coding |
| Tags | agent-eval |
Abstract
Measures: Retirement of SWE-bench Verified (saturation, flawed tests). Metric: -.
BibTeX
@misc{openai_why_2026,
title = {Why we no longer evaluate {SWE}-bench {Verified}},
url = {https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/},
abstract = {Measures: Retirement of SWE-bench Verified (saturation, flawed tests). Metric: -.},
author = {{OpenAI}},
month = feb,
year = {2026},
keywords = {agent-eval},
}