Logical phase transitions: reasoning collapses rather than degrades
Work on logical phase transitions finds that LLM reasoning does not decay smoothly as problems get harder — it holds and then collapses. Related results on reasoning skills report fewer tokens and higher accuracy, complicating the assumption that more visible deliberation means better thinking.
Phase-transition behaviour is bad news for evaluation practice specifically. Smooth degradation means a benchmark score at one difficulty predicts performance a little past it. A sharp transition means it does not — a model at 95% just below the boundary can be near-useless just above it, and no amount of testing on the easy side reveals where the edge is.
That has a direct deployment consequence. Anyone sizing a model to a task by benchmarking representative examples is implicitly assuming continuity. If the transition is sharp and the task distribution has a tail, the failures will cluster in exactly the hard cases where the cost of being wrong is highest.
The fewer-tokens-more-accuracy result cuts against a common intuition in the same direction as the latent-reasoning argument: length of visible deliberation is not a proxy for quality of reasoning, and treating it as one has been quietly shaping both product design and evaluation.
arXiv — Logical phase transitions: understanding collapse in LLM logical reasoning → · arXiv — Thinking with reasoning skills: fewer tokens, more accuracy → · arXiv — DeonticBench: a benchmark for reasoning over rules →