DPO has supplanted RLHF as the default frontier alignment method — the 2026 safety-research stack moves from preference modeling to direct optimization

Industry consensus by May 2026 places Direct Preference Optimization (DPO) as the default alignment training method across frontier labs, replacing the more complex RLHF pipeline that dominated through 2025. The shift is structural: DPO requires less compute, fewer human-in-the-loop annotations, and produces more interpretable preference gradients. Combined with the rise of process-reward models and constitutional self-critique loops, frontier alignment has materially simplified.

The interpretability consequence is the under-noticed gain. RLHF's reward-model-driven training produced models whose alignment behavior was hard to attribute to specific training data; DPO's direct-pair-comparison structure makes the alignment training set more legible to post-hoc analysis. Anthropic's microscope-based circuit identification work is meaningfully easier on DPO-trained models than on RLHF-trained predecessors.

For the DPO-and-mech-interp-gap argument from 5/21, the May 2026 confirmation is that the gap is narrowing. Frontier labs are shipping DPO-aligned models; mechanistic interpretability teams are getting better attribution on those models; the safety guarantee that AISI's Opus 4.5 evaluation relies on now has a meaningfully cleaner methodology behind it.

Claude 5 Hub — AI safety progress → · Zylos — AI safety alignment 2026 → · arXiv — mechanistic interpretability review →