Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents

The core contribution is not a new training algorithm but a reframing of what information is already present in the policy itself. The log-probability ratio between a trained policy and its reference checkpoint turns out to encode step-level credit assignment implicitly, meaning the expensive scaffolding of process reward models was always somewhat redundant for this signal.

This connects most directly to the self-distillation diversity tradeoff covered the same day ('On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversity'). Both papers are probing the hidden costs and hidden benefits baked into standard post-training pipelines, and together they suggest practitioners are operating with an incomplete accounting of what their training choices actually do. The progress advantage finding is the optimistic counterpart: where self-distillation quietly narrows exploration, the implicit advantage signal quietly provides step-level supervision for free. The RevengeBench work on policy reconstruction is largely disconnected here, touching agent evaluation from an interpretability angle rather than a training efficiency one.

Watch whether any of the major agent training frameworks (OpenAI's RLVR pipeline or DeepMind's agent scaffolds) cite or absorb this formulation within the next two quarters. Adoption in a production codebase would confirm the practical portability of the approach beyond the paper's own benchmarks.