Changkun Ou/Library

ReplicHAI › Agent Evals

Shared by Changkun Ou with everyone

256 entries Export .bib
Year ↓ Title added Authors Venue
Webpage 2024 LMArena LMArena
Preprint 2024 Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks Fourney et al. arXiv
Computer program 2024 METR Task Standard METR
Preprint 2024 ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code Tang et al. arXiv
Preprint 2024 MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation Huang et al. arXiv
Preprint 2024 OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation Wang et al. arXiv
Preprint 2024 OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web Kapoor et al. arXiv
Preprint 2024 OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments Xie et al. arXiv
Preprint 2024 Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study Li et al. arXiv
Blog post 2024 Raising the bar on SWE-bench Verified with Claude 3.5 Sonnet Anthropic
Preprint 2024 RedCode: Risky Code Execution and Generation Benchmark for Code Agents Guo et al. arXiv
Preprint 2024 SciCode: A Research Coding Benchmark Curated by Scientists Tian et al. arXiv
Webpage 2024 SEAL Leaderboards AI
Preprint 2024 SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents Cheng et al. arXiv
Preprint 2024 Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows? Cao et al. arXiv
Preprint 2024 SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation Ma et al. arXiv
Preprint 2024 StreamBench: Towards Benchmarking Continuous Improvement of Language Agents Wu et al. arXiv
Preprint 2024 SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories Bogin et al. arXiv
Preprint 2024 SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains? Yang et al. arXiv
Preprint 2024 SWE-Bench+: Enhanced Coding Benchmark for LLMs Aleithan et al. arXiv
Preprint 2024 SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Jimenez et al. arXiv
Preprint 2024 TravelPlanner: A Benchmark for Real-World Planning with Language Agents Xie et al. arXiv
Preprint 2024 VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents Liu et al. arXiv
Preprint 2024 VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks Koh et al. arXiv
Preprint 2024 WebArena: A Realistic Web Environment for Building Autonomous Agents Zhou et al. arXiv
Preprint 2024 WebLINX: Real-World Website Navigation with Multi-Turn Dialogue Lù et al. arXiv
Preprint 2024 WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models He et al. arXiv
Preprint 2024 Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale Bonatti et al. arXiv
Preprint 2024 WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks? Drouin et al. arXiv
Preprint 2023 API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs Li et al. arXiv
Preprint 2023 CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion Ding et al. arXiv
Preprint 2023 GAIA: a benchmark for General AI Assistants Mialon et al. arXiv
Preprint 2023 Gorilla: Large Language Model Connected with Massive APIs Patil et al. arXiv
Preprint 2023 Holistic Evaluation of Language Models Liang et al. arXiv
Preprint 2023 InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback Yang et al. arXiv
Preprint 2023 LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning Liu et al. arXiv
Preprint 2023 Mind2Web: Towards a Generalist Agent for the Web Deng et al. arXiv
Computer program 2023 OpenAI Evals OpenAI
Preprint 2023 RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems Liu et al. arXiv
Preprint 2023 ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs Qin et al. arXiv
Preprint 2023 ToolQA: A Dataset for LLM Question Answering with External Tools Zhuang et al. arXiv
Preprint 2023 WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents Yao et al. arXiv
Preprint 2022 Benchmarking the Spectrum of Agent Capabilities Hafner arXiv
Journal article 2022 Competition-Level Code Generation with AlphaCode Li et al. Science
Preprint 2022 DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation Lai et al. arXiv
Preprint 2022 MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge Fan et al. arXiv
Preprint 2022 ScienceWorld: Is your Agent Smarter than a 5th Grader? Wang et al. arXiv
Preprint 2021 ALFWorld: Aligning Text and Embodied Environments for Interactive Learning Shridhar et al. arXiv
Preprint 2021 Evaluating Large Language Models Trained on Code Chen et al. arXiv
Computer program 2021 lm-evaluation-harness EleutherAI
Preprint 2021 Measuring Coding Challenge Competence With APPS Hendrycks et al. arXiv
Preprint 2021 Program Synthesis with Large Language Models Austin et al. arXiv
Preprint 2020 The NetHack Learning Environment Küttler et al. arXiv
Preprint 2019 TextWorld: A Learning Environment for Text-based Games Côté et al. arXiv
Preprint 2018 Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration Liu et al. arXiv
Preprint 2016 OpenAI Gym Brockman et al. arXiv

‹ Prev 201–256 of 256

/ filter · j k or ↑ ↓ select · Enter open page · c copy BibTeX