MonitorBench and counterfactual training: two attempts to make chain-of-thought worth trusting
One paper builds a comprehensive benchmark for monitorability. Another trains for faithfulness directly using counterfactual simulation. A third red-teams the monitor itself and finds settings where textual reasoning fails to reveal critical internal information.
The three approaches attack the same problem from different sides — measure it, train for it, or break it — and the third is the one that keeps producing uncomfortable results.
Counterfactual simulation training is the more interesting intervention. If a trace is faithful, altering the reasoning should alter the conclusion. Training against that property directly is a way of making faithfulness an objective rather than hoping it emerges.
The stress-testing work matters most for deployment. Finding settings where the reasoning trace does not reveal what the model is actually doing is precisely what you need before a monitor becomes a safety control, because a monitor that fails silently is worse than no monitor at all.
arXiv — MonitorBench: a comprehensive benchmark for chain-of-thought monitorability in large language models → · arXiv — Counterfactual simulation training for chain-of-thought faithfulness → · arXiv — CoT Red-Handed: stress testing chain-of-thought monitoring →