Smaller rigs, better questions

Two papers point the same way: post-training is under-optimised, not under-resourced. If that holds, one of the frontier's assumed moats is thinner than advertised.

A method matches full reinforcement learning on clean comparisons, and the authors conclude heavy RL post-training infrastructure may be disproportionate to the problem. Costs are reported in dollars, against public on-demand GPU pricing.

Reporting cost in currency is the quiet contribution

Most post-training papers report GPU-hours or nothing, which makes results incomparable across labs with different hardware deals. A dollar figure at a published rate is a number any reader can check and any reviewer can dispute. More of this, please.

"Clean comparisons" is doing a lot of work in that claim.

RL results are notoriously sensitive to tuning budget. Matching full RL under matched-and-limited tuning is a narrower result than matching it when both approaches are at their best. The paper is careful about this. Summaries of it will not be, and the claim will be repeated without the qualifier by the end of the month.

The second paper points the same direction

Supervised fine-tuning on A* execution traces, plus RL against A*-informed process rewards, takes 1B–3B models from near-zero to beating baselines.

Handing a model worked examples of systematic search, from a classical algorithm whose procedure is fully inspectable, beats asking it to induce the strategy from outcomes. The reward model scores the process rather than only the answer.

The scope limit is real and the authors state it: A* needs a state space, a successor function and an admissible heuristic. Most tasks people want reasoning for have none. This is strong evidence that small models lack procedure rather than capacity — not a general method.

Why it matters commercially

Elaborate post-training pipelines are one of the moats large labs are presumed to hold. If cheaper methods reach the same place, capability shifts toward whoever has good data and good judgement. It also means the three-week release cadence gets shorter, not longer.

What this asks of you

If you have been assuming you cannot compete on post-training without a cluster, re-check that assumption this quarter. And if you are running small models in production, the A* result says the ceiling you hit may have been a missing procedure rather than a parameter count.

arXiv — Rethinking RL for LLM Reasoning → · arXiv — Learning to Reason Efficiently with A* Post-Training →