Linear-DPO: Linear Direct Preference Optimization for Diffusion and Flow-Matching Generative Models

The deeper issue Linear-DPO surfaces is that most image alignment research has been quietly borrowing NLP-derived loss formulations without auditing whether the underlying assumptions hold for continuous, noise-dependent regression targets. The sigmoid utility function isn't just suboptimal here, it's structurally mismatched to how diffusion timesteps distribute gradients.

This connects to a pattern visible across several recent papers in the archive. The 'Reasoning-Trace Collapse' work from the same day showed that fine-tuning borrowed from one regime (standard instruction tuning) silently degrades properties that only exist in another (reasoning scaffolding). Linear-DPO is the image generation version of that same diagnostic instinct: the transfer of an optimization technique across modalities introduces failure modes that don't announce themselves in headline metrics. The 'Advantage Collapse in GRPO' paper adds another data point, identifying how gradient starvation during preference-based training can stall improvement entirely. Taken together, these papers suggest practitioners are now doing the harder work of stress-testing alignment methods rather than just applying them.

The real test is whether Linear-DPO's EMA reference update holds up under production-scale diversity, specifically whether win-rate gains on standard preference benchmarks like Pick-a-Pic or HPSv2 persist when the prompt distribution shifts significantly from training. If a major text-to-image lab publishes an ablation using this formulation within the next two quarters, that would confirm the approach is being taken seriously beyond academic settings.