Skip to content
Modelwire
Subscribe

SFT and RL generalize equally when data and budget align

Source published ·Modelwire updated

Original coverage: arXiv cs.LG ↗·How Modelwire adds context

The development

A new empirical study challenges the conventional wisdom that reinforcement learning inherently generalizes better than supervised fine-tuning. Testing on Qwen2.5 and Qwen3 models, researchers found that SFT's perceived weakness stems not from the training method itself but from data composition and scale. When SFT datasets are carefully curated and scaled to match RL's effective training budget, SFT achieves comparable generalization. This finding reshapes how practitioners should think about training methodology trade-offs, suggesting that data engineering and budget allocation matter more than the algorithmic choice between RL and SFT.

Modelwire’s AI-generated summary of coverage from arXiv cs.LG.

Modelwire analysis

Explainer

Our AI-generated reading of the wider context and the next developments to watch.

The study doesn't just say SFT can work; it isolates that prior conclusions about RL's superiority were confounded by unequal data budgets and composition. The actual finding is that generalization gaps collapse when you control for training investment, not methodology.

This connects directly to the pattern surfaced in the compositional generalization work from November 10th: researchers keep discovering that apparent capability differences vanish once you control for what's actually being measured. Here, the confound is budget and data curation; there, it was train-test structural similarity. Both papers suggest the field has been drawing conclusions from experiments that don't isolate the variable they claim to test. The persona generalization work from November 9th adds another layer: if behavioral spillover follows predictable math, then training methodology effects may also be more constrained and forecastable than the RL-versus-SFT framing suggests.

If Alibaba or other teams reproduce this finding on held-out benchmarks (GPQA, MATH, coding evals) where SFT and RL have historically diverged most, and the gap remains closed with matched budgets, that confirms the confound was real. If the gap re-emerges on any major benchmark even with matched data, the methodology choice still matters for something.

This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error

Coverage behind this analysis

These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.

  1. ·arXiv cs.LG

    Transformer compositional generalization benchmarks conflate equivalence with true generalization

    Researchers have identified why compositional generalization benchmarks for transformers produce inconsistent results: most prior evaluations failed to control for composition equivalence and module coverage in their train-test splits. The work reveals that apparent success in direct models often stems from exploiting hidden structural similarities between training and test combinations rather than genuine compositional reasoning. This…

    Read Modelwire coverage →Original source ↗

MentionsQwen2.5 · Qwen3 · Alibaba

MW

How this coverage is produced

Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.

Modelwire summarizes, we don’t republish. arXiv cs.LG originally reported this story as “Rethinking "RL Generalizes, SFT Memorizes": The Role of SFT Data”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.

More to explore

LoRA behaves differently in RL versus supervised fine-tuning, study finds

Transformer compositional generalization benchmarks conflate equivalence with true generalization

arXiv cs.LG·

Regularization techniques amplify hidden bias transfer in models

SFT and RL generalize equally when data and budget align · Modelwire