A ceiling on thinking longer

Two papers land on the same conclusion from different directions: inference-time scaling has bounds, and how much it buys you depends on training. The knob has a stop.

BAPO bounds put a theoretical ceiling on chain-of-thought token complexity, while a solvable linear model shows the returns shrink as training improves.

The assumption being tested

Inference-time scaling has been the industry's answer to hard problems for two years: let it think longer, sample more candidates, pick the best. It works, it is empirical, and nobody had an account of where it stops.

A complexity bound converts that from a hope into a parameter. Beyond the bound, extra tokens buy variance rather than capability — and reasoning tokens are billed.

The trade runs the wrong way

The second result is the more uncomfortable one. Best-of-n pays in proportion to the spread between a typical attempt and the best available; training compresses that spread on everything the model genuinely learned. So the hard tail where extra samples help gets thinner as the model gets better.

You cannot economise on training and make it up at inference. The better your model, the less inference scaling has left to recover.

Why the toy model earns its keep

Deriving this in a solvable linear setting is a deliberate choice. A closed-form result establishes a mechanism rather than a correlation, and whether the constants transfer to a frontier transformer is a narrower question than whether the effect is real.

It also connects to a complaint running through the whole evaluation layer: measuring systems where the conditions are convenient produces numbers that do not survive contact with deployment. Theory that tells you which experiments not to run is worth more than another leaderboard.

Thinking longer is starting to look less like a scaling axis and more like a fallback for what training left behind.

arXiv — Reasoning about Reasoning: BAPO Bounds on Chain-of-Thought Token Complexity in LLMs → · arXiv — Learning Shrinks the Hard Tail: Training-Dependent Inference Scaling in a Solvable Linear Model → · OpenReview — Inference Scaling, Learning to Reason, and Agentic Systems →