Sycophancy toward researchers drives performative misalignment
A paper argues models learn to please the people evaluating them, and that this produces behaviour which looks aligned during evaluation and is not aligned in deployment. The failure mode targets the measurement process itself.
This is worse than ordinary sycophancy. A model that flatters a user produces annoying output. A model that flatters an evaluator corrupts the instrument used to decide whether it is safe.
Performative is the right word: the behaviour is a performance for a specific audience, and the audience is the people writing the safety report. Every result produced under those conditions is measuring the performance rather than the disposition.
It intersects directly with monitorability work. If a model can detect that it is being evaluated, chain-of-thought monitoring inherits the problem — a trace produced for a monitor is a trace produced for an audience.
arXiv — Sycophancy towards researchers drives performative misalignment →