'ProjDevBench' arXiv 2602.01655 — end-to-end project-development benchmark for coding agents goes beyond function-level evaluation to full-project completion
The ProjDevBench arXiv paper (2602.01655) introduces an end-to-end project-development benchmark for coding agents — evaluating not just function-level code completion (SWE-Bench scope) but full-project completion including setup, dependency management, multi-file integration, and end-to-end testing. The benchmark scope addresses a gap in the H1 2026 coding-agent evaluation landscape.
The substantive piece is the full-project-scope evaluation methodology. SWE-Bench Verified and Terminal-Bench evaluate agents on relatively scoped tasks — bug-fixes, function implementation, narrow refactors. ProjDevBench extends to full project-development scope including the integration-and-orchestration work that distinguishes production-deployment-grade coding agents from research-tier ones. The capability gap (function-level competence vs project-level competence) was identified through 2025 but not systematically benchmarked until ProjDevBench.
The competitive read against ClawBench's live-site browser evaluation and other H2 2026 benchmark-stratification work is that agent-evaluation methodology is broadly maturing into workload-specific suites. ProjDevBench for coding-agent project-development, ClawBench for browser-agent live-site execution, Mem2ActBench for memory-integration tool-action workloads. Procurement evaluation should match benchmark choice to deployment workload-shape.
arXiv — ProjDevBench: End-to-end project-development benchmark for coding agents (2602.01655) → · MorphLLM — Best AI Coding Agents (June 2026): Scored Leaderboard →