Forward KL divergence inflates student entropy in knowledge distillation
Source published ·Modelwire updated
Original coverage: arXiv cs.LG ↗·How Modelwire adds context
The development
Researchers have mapped how different divergence measures shape student model entropy during knowledge distillation, a foundational technique in LLM training. Forward KL divergence provably increases student entropy above the teacher, while reverse KL reduces it until convergence pressure mounts. This theoretical framework explains why on-policy distillation produces lower-entropy outputs and offers practitioners a principled way to predict and control model behavior during training. The findings bridge a gap between distillation's widespread use and its poorly understood mechanics, with direct implications for pretraining and fine-tuning workflows.
Modelwire’s AI-generated summary of coverage from arXiv cs.LG.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The paper doesn't claim distillation is new, but rather explains the mechanism by which different divergence measures produce opposite entropy outcomes. The key insight is that forward KL and reverse KL don't just differ in performance; they provably steer student behavior in opposite directions, giving practitioners a principled way to predict which divergence to use for a given output constraint.
This theoretical work directly illuminates the practical failures documented in recent coverage. The on-policy distillation collapse paper from October 2nd showed that implicit reward signals can cause students to exploit loopholes and generate repetitive output; this entropy analysis explains why reverse KL convergence pressure creates exactly that low-entropy regime. Similarly, the masked diffusion work from the same day highlights credit assignment problems during denoising. Understanding how divergence shapes entropy gives practitioners a diagnostic tool: if a distilled model is collapsing into repetition, you're likely in a reverse KL regime where entropy has been over-constrained. The self-distillation reasoning paper from September 29th also becomes more interpretable through this lens, since verification gaps may interact with divergence-driven entropy shifts.
If practitioners report that switching from reverse KL to forward KL in on-policy distillation workflows reduces the repetition collapse described in the October 2nd paper, that confirms this theory has moved from explanation to actionable guidance. Watch whether the next generation of distillation recipes published on arXiv explicitly specify divergence choice as a control variable for entropy targets, rather than treating it as a secondary hyperparameter.
This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error
Coverage behind this analysis
These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.
·arXiv cs.CL
On-policy distillation collapse traced to misaligned implicit rewards
Researchers have identified why on-policy distillation, a widely adopted post-training technique for language models, produces inconsistent results ranging from performance gains to catastrophic collapse into repetitive output. By framing the problem through reinforcement learning, they show that teachers implicitly reward student behaviors regardless of their own frequency, creating misaligned incentives. When this implicit reward signal…
MentionsKnowledge distillation · Forward KL divergence · Reverse KL divergence · Cross-entropy training · On-policy distillation
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. arXiv cs.LG originally reported this story as “Divergence controls entropy in distillation”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.