Changkun Ou/Library

ReplicHAI › Agent Evals › 07 General, long-horizon & economic tasks

Shared by Changkun Ou with everyone

39 entries Export .bib
Year ↓ Title added Authors Venue
Preprint 2026 AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications Zhao et al. arXiv
Blog post 2026 ARC-AGI-3 Foundation
Preprint 2026 BizAgentBench: Unveiling Hidden Challenges in Deploying Long-Horizon Agents for Real-World Business Zhao et al. arXiv
Blog post 2026 Company Knowledge Bench Kapa.ai
Preprint 2026 DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints Zhang et al. arXiv
Webpage 2026 Digital Work Index AI
Webpage 2026 MBABench 2.0 leaderboard AI
Webpage 2026 SnorkelFinance 2.0 leaderboard AI
Webpage 2026 SnorkelLegal leaderboard AI
Webpage 2026 SnorkelManufacturing leaderboard AI
Webpage 2026 SnorkelRevOps leaderboard AI
Webpage 2026 SnorkelUnderwrite 2.0 leaderboard AI
Webpage 2026 WorkplaceAgents leaderboard AI
Preprint 2025 AgentBench: Evaluating LLMs as Agents Liu et al. arXiv
Preprint 2025 AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments Schmidgall et al. arXiv
Preprint 2025 ARE: Scaling Up Agent Environments and Evaluations Froger et al. arXiv
Preprint 2025 BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games Paglieri et al. arXiv
Preprint 2025 CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments Huang et al. arXiv
Preprint 2025 Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making Li et al. arXiv
Preprint 2025 EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents Yang et al. arXiv
Preprint 2025 Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks Bigeard et al. arXiv
Preprint 2025 GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks Patwardhan et al. arXiv
Blog post 2025 GDPval: measuring the performance of our models on real-world tasks OpenAI
Preprint 2025 MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents Jiang et al. arXiv
Webpage 2025 Remote Labor Index CAIS
Preprint 2025 Remote Labor Index: Measuring AI Automation of Remote Work Mazeika et al. arXiv
Preprint 2025 TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks Xu et al. arXiv
Webpage 2025 Vending-Bench 2 Labs
Preprint 2025 Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agents Backlund, Petersson arXiv
Preprint 2024 AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents Ma et al. arXiv
Preprint 2024 BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation Li et al. arXiv
Preprint 2024 OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation Wang et al. arXiv
Preprint 2024 TravelPlanner: A Benchmark for Real-World Planning with Language Agents Xie et al. arXiv
Preprint 2023 LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning Liu et al. arXiv
Preprint 2022 Benchmarking the Spectrum of Agent Capabilities Hafner arXiv
Preprint 2022 MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge Fan et al. arXiv
Preprint 2021 ALFWorld: Aligning Text and Embodied Environments for Interactive Learning Shridhar et al. arXiv
Preprint 2020 The NetHack Learning Environment Küttler et al. arXiv
Preprint 2019 TextWorld: A Learning Environment for Text-based Games Côté et al. arXiv

/ filter · j k or ↑ ↓ select · Enter open page · c copy BibTeX