Repository Context Files May Reduce Coding Agent Performance仓库上下文文件可能会降低代码代理的性能

The following content is generated by LLMs and may contain inaccuracies.

I’ll help you create a structured deep dive on this research paper. Let me first fetch the full paper to examine its contents and then search for related work.Let me search for more information about related work on context in coding agents and the authors of this paper.Now let me search for more information about the benchmark dataset they created (AGENTbench).Now I have comprehensive information to create a structured deep dive. Let me synthesize everything into the requested format.

Context

Repository context files like AGENTS.md have become widespread in software development to tailor coding agents to repositories, with over 60,000 open-source repositories adopting them by early 2026. These files function as “READMEs for agents,” providing repository overviews and information on developer tooling to help coding agents navigate repositories, run build and test commands, adhere to style guides, and solve tasks. The AGENTS.md format emerged from collaborative efforts across the AI software development ecosystem, including OpenAI Codex, Amp, Jules from Google, Cursor, and Factory.

Despite strong industry encouragement from model providers like OpenAI and Anthropic, this paper from ETH Zürich’s Secure, Reliable, and Intelligent Systems Lab addresses a critical gap: there is currently no rigorous investigation into whether such context files are actually effective for real-world tasks. The work challenges prevailing assumptions at a moment when coding agents are rapidly advancing on benchmarks like SWE-bench, where top agents score 20% on the full benchmark and 43% on SWE-bench Lite.

Key Insights

Counterintuitive core finding: Across multiple coding agents and LLMs, context files tend to reduce task success rates compared to providing no repository context, while also increasing inference cost by over 20%. This directly contradicts agent developer recommendations.

Benchmark innovation: The authors constructed AGENTbench, a novel benchmark comprising Python software engineering tasks from 12 recent and niche repositories, which all feature developer-written context files. This complements existing evaluations: SWE-bench tasks from popular repositories are evaluated with LLM-generated context files following agent-developer recommendations, while AGENTbench provides a novel collection of issues from repositories containing developer-committed context files. The distinction matters because context files have only been formalized in August 2025, and adoption is not uniform across the industry.

Differential impact by provenance: Developer-provided files only marginally improve performance compared to omitting them entirely (an increase of 4% on average), while LLM-generated context files have a small negative effect on agent performance (a decrease of 3% on average). This pattern held across different LLMs and prompts used to generate the context files.

Behavioral mechanism: Both LLM-generated and developer-provided context files encourage broader exploration (e.g., more thorough testing and file traversal), and coding agents tend to respect their instructions. The problem is not agent non-compliance but rather that unnecessary requirements from context files make tasks harder. Context files lead to increased exploration, testing, and reasoning by coding agents, and, as a result, increase costs by over 20%.

Content analysis of existing files: One recommendation for context files is to include a codebase overview. Across the 12 developer-provided context files in AGENTbench, 8 include a dedicated codebase overview, with 4 explicitly enumerating and describing the directories and subdirectories in the repository. Functional directives (build, test, implementation detail, architecture) dominate, while guidance on non-functional requirements (security, performance, usability) is relatively uncommon. These files exhibit a median update interval of 22 hours, with most changes involving the addition or minor modification of 50 words or fewer.

Implications for practice: The authors recommend omitting LLM-generated context files for the time being, contrary to agent developers' recommendations, and including only minimal requirements (e.g., specific tooling to use with this repository). This aligns with emerging practitioner wisdom: Factory advises aiming for ≤ 150 lines, warning that long files slow the agent and bury signal, while some developers argue for ruthless minimalism—just a one-sentence project description and package manager specification.

Broader context tensions: This finding sits within ongoing debates about retrieval-augmented approaches for coding. Industry practitioners like Nick Pash, Head of AI at Cline, argue that RAG can be a ‘seductive trap’ for coding tasks because code is inherently logical and structured and does not always benefit from being broken down into semantically similar but contextually isolated chunks. Anthropic ultimately abandoned RAG approaches when agentic search consistently outperformed RAG across both internal benchmarks and subjective quality evaluations.

Related work on repository-level code generation: The paper builds on the SWE-bench ecosystem, where given a codebase and an issue, a language model is tasked with generating a patch that resolves the described problem. Recent work shows SelectSolve demonstrates that in fully observable environments such as SWE-bench, simply providing the entire codebase to a long-context LLM with proper prompting can achieve, and sometimes surpass, the performance of carefully designed multi-tool approaches, suggesting that when sufficient context capacity exists, explicit context management may become less critical.

Direct link to the paper: Gloaguen et al., “Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?"

Open Questions

1. What is the optimal context budget allocation? If context files increase costs by 20%+ with negative performance impact, how should those tokens be reallocated—toward deeper code retrieval, longer conversation history, or expanded test coverage feedback?

2. Can selective, task-adaptive context files outperform static ones? Rather than a single AGENTS.md file consumed at initialization, could agents dynamically query minimal, task-specific guidance (e.g., “build commands only” for dependency issues, “test patterns only” for bug fixes) to capture the marginal benefit of developer-written context while avoiding the breadth penalty?