Supervised finetuning outperforms expectations with strategic data sampling
Researchers challenge the conventional hierarchy between supervised finetuning and reinforcement learning in posttraining, demonstrating that SFT can achieve stronger generalization than previously assumed when data distribution is optimized appropriately. Rather than redesigning loss functions, the work focuses on curating training data to capture on-policy learning benefits while retaining access to offline expert trajectories. This finding reshapes assumptions about capability transfer and catastrophic forgetting, suggesting practitioners may be underutilizing SFT's potential and that the frontier between SFT and RL methods is less rigid than industry consensus implies.62
