Preprint · 2026

DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints

Yinger Zhang, Shutong Jiang, Renhao Li, Jianhong Tu, Yang Su, Lianghao Deng, Xudong Guo, Chenxu Lv, Junyang Lin

Abstract

While agent evaluation has shifted toward long-horizon tasks, most benchmarks still emphasize local, step-level reasoning rather than the global constrained optimization (e.g., time and financial budgets) that demands genuine planning ability. Meanwhile, existing LLM planning benchmarks underrepresent the active information gathering and fine-grained local constraints typical of real-world settings. To address this, we introduce DeepPlanning, a challenging benchmark for practical long-horizon agent planning. It features multi-day travel planning and multi-product shopping tasks that require proactive information acquisition, local constrained reasoning, and global constrained optimization. Evaluations on DeepPlanning show that even frontier agentic LLMs struggle with these problems, highlighting the importance of reliable explicit reasoning patterns and parallel tool use for achieving better effectiveness-efficiency trade-offs. Error analysis further points to promising directions for improving agentic LLMs over long planning horizons. We open-source the code and data to support future research.

BibTeX Copy

@misc{zhang_deepplanning_2026,
	title = {{DeepPlanning}: {Benchmarking} {Long}-{Horizon} {Agentic} {Planning} with {Verifiable} {Constraints}},
	shorttitle = {{DeepPlanning}},
	url = {http://arxiv.org/abs/2601.18137},
	doi = {10.48550/arXiv.2601.18137},
	abstract = {While agent evaluation has shifted toward long-horizon tasks, most benchmarks still emphasize local, step-level reasoning rather than the global constrained optimization (e.g., time and financial budgets) that demands genuine planning ability. Meanwhile, existing LLM planning benchmarks underrepresent the active information gathering and fine-grained local constraints typical of real-world settings. To address this, we introduce DeepPlanning, a challenging benchmark for practical long-horizon agent planning. It features multi-day travel planning and multi-product shopping tasks that require proactive information acquisition, local constrained reasoning, and global constrained optimization. Evaluations on DeepPlanning show that even frontier agentic LLMs struggle with these problems, highlighting the importance of reliable explicit reasoning patterns and parallel tool use for achieving better effectiveness-efficiency trade-offs. Error analysis further points to promising directions for improving agentic LLMs over long planning horizons. We open-source the code and data to support future research.},
	urldate = {2026-11-06},
	publisher = {arXiv},
	author = {Zhang, Yinger and Jiang, Shutong and Li, Renhao and Tu, Jianhong and Su, Yang and Deng, Lianghao and Guo, Xudong and Lv, Chenxu and Lin, Junyang},
	month = jan,
	year = {2026},
	note = {arXiv:2601.18137 [cs.AI]},
	keywords = {Computer Science - Artificial Intelligence, Computer Science - Computation and Language},
}