PRISM: Pre-alignment via Black-box On-policy Distillation for Multimodal Reinforcement Learning
Researchers propose PRISM, a three-stage training pipeline that addresses a critical bottleneck in multimodal model alignment. The core insight targets distributional drift, where supervised fine-tuning diverges from both the model's original capabilities and the actual training signal, creating compounding errors in vision-language reasoning. By inserting an explicit alignment stage using on-policy distillation before reinforcement learning, PRISM decouples perception failures from reasoning failures, allowing targeted correction of each. This work matters because it challenges the standard post-training recipe that has dominated LLM scaling, suggesting that naive sequential training stages leave performance on the table for multimodal systems.62
