Self-distillation gains don't generalize beyond training distribution
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
A new empirical study challenges the claimed advantages of self-distillation methods in reasoning tasks, finding that performance gains reported by SDPO and OPSD do not reliably transfer to out-of-distribution problems. The research suggests these improvements stem from overfitting or repackaging existing reasoning capabilities rather than genuine advances in generalization. This matters because self-distillation has been positioned as a more sample-efficient alternative to reinforcement learning for training reasoning models, and this negative result forces a recalibration of expectations around the technique's real-world utility and scalability.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
Skeptical readOur AI-generated reading of the wider context and the next developments to watch.
The paper doesn't just report lower-than-expected gains; it suggests the reported improvements may never have been real improvements at all, rather than a case of techniques working in-distribution but failing out-of-distribution. This reframes the question from 'does self-distillation generalize?' to 'were we measuring the right thing in the first place?'
This finding lands in the middle of a three-week reckoning with on-policy distillation's actual capabilities. The October 9 work on critique-conditioned distillation found that teacher scoring fails to localize errors reliably. The October 7 paper isolated an asymmetry where distillation transfers skills but not knowledge. Now this study suggests the performance gains themselves don't hold up outside narrow test sets. Together, these papers paint a picture of a technique that looked promising in controlled settings but lacks the robustness the field assumed it had.
If the authors release ablations showing which specific out-of-distribution splits broke SDPO and OPSD (e.g., reasoning on novel problem types vs. longer chains vs. different domains), watch whether the October 7 'On-Policy Distillation Teaches New Skills' team attempts to reconcile their skill-transfer findings with this generalization failure. A mismatch there would suggest the 'skills' being transferred are brittle or eval-specific.
This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error
Coverage behind this analysis
These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.
·arXiv cs.CL
Self-distillation critique signals fail to localize reasoning errors
Researchers challenge a core assumption in on-policy self-distillation for reasoning tasks: that a teacher model scoring student rollouts provides reliable token-level credit signals. Testing against gold error annotations, the work finds that standard distillation objectives fail to localize errors or suppress incorrect tokens, and cannot consistently distinguish successful from failed solutions on identical problems. This…
MentionsSelf-Distillation Policy Optimization · On-Policy Self-Distillation · SDPO · OPSD
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Does self-distillation improve reasoning?”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.