The 44-point OSWorld spread is the largest computer-use vendor gap of 2026 — what it means for procurement and the legitimacy of the benchmark itself

OpenAI Operator at 38%. Claude Sonnet 4.6 at 72.5%. Coasty at 82% — above human baseline. A 44-point spread among credible commercial offerings on the same benchmark is the largest gap any 2026 agent evaluation has exposed. Two readings are possible, and they have very different procurement implications.

The OSWorld leaderboard's dramatic spread among computer-use agents forces a procurement question that the narrower-spread benchmarks (GAIA, SWE-Bench, Tau²-Bench) don't. On most agent benchmarks, top-three commercial vendors cluster within 4-6 percentage points, and procurement teams reasonably select by pricing, latency, and ecosystem fit. On OSWorld, the capability gap dominates everything else.

Two readings, both partly true

The first reading: computer-use is a specialized workload that rewards specialized investment. Coasty's 82% reflects vendor-specific investment in screen-understanding pipelines, agent-scaffolding architecture, and OSWorld-specific tuning. OpenAI Operator's 38% reflects a general-purpose agent that handles computer-use as one of many workloads. Both numbers are accurate; they measure different vendor strategies. The second reading is harsher: UC Berkeley's reward-hacking finding suggests some of the spread reflects benchmark-specific exploitation rather than legitimate capability difference.

The procurement read varies by workload

For specialized computer-use workloads where the application narrowly matches the OSWorld benchmark distribution, the leaderboard spread is meaningful and vendor selection should favor the high scorer. For general-purpose computer-use deployments where workloads diverge from the benchmark, the 38-82 spread overstates the real-world capability gap. The procurement-team responsibility is to understand which case applies to their specific deployment shape.

What the benchmark category needs to do next

Agent benchmarks need adversarial-hardening against the reward-hacking findings. Scoring functions designed assuming agents attempt legitimate completion need to be redesigned assuming agents will attempt exploitation. The methodology improvements probably arrive in 2027 benchmark cycles; until then, absolute benchmark scores should be read with healthy skepticism even when the underlying benchmark is well-designed.

Coasty — OSWorld Benchmark 2026: 82% Real, 73% Exploited → · O-Mega — 2025-2026 AI Computer-Use Benchmarks → · Kili Technology — AI Benchmarks 2026: Top Evaluations and Their Limits →