Reasoning models inherit pause tokens from data, not RL
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
A new arXiv study challenges the narrative that reinforcement learning teaches reasoning models to use pause tokens like {wait}. By tracing these markers through open model training histories, researchers found that RL doesn't invent the behavior; instead, it amplifies patterns already present in base training data. Models trained on reasoning traces naturally acquire pause tokens during fine-tuning, while those lacking such data rarely develop them even with RL optimization. This work matters because it reframes what RL actually accomplishes in reasoning systems, separating genuine capability gains from inherited linguistic artifacts. The finding has implications for how labs interpret RL's role in scaling reasoning and for understanding what training data truly shapes model behavior.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The study doesn't just show RL amplifies existing patterns; it demonstrates that pause tokens are functionally inherited from pretraining data, not learned behaviors. Models without reasoning traces in their training data fail to develop pause tokens even under RL pressure, suggesting RL's role in reasoning is narrower than commonly claimed.
This directly extends the October 17th analysis showing RL makes high-leverage token choices in only 4% of positions. That work suggested RL encodes behavioral patterns rather than algorithmic shifts; this new finding makes that concrete by tracing one specific behavioral pattern (pause tokens) back to its source. Together, they reframe RL as a filter and amplifier of existing training signal rather than a capability generator. The October 19th sycophancy study also tracked vulnerabilities across training stages, finding that post-training alone doesn't explain model behavior; this reasoning work applies the same lineage-tracing logic to show inherited linguistic artifacts shape what RL can optimize.
If labs report that models trained on non-reasoning base data develop pause tokens after RL on reasoning tasks, that contradicts this finding and suggests RL can induce the behavior de novo. Otherwise, watch whether future reasoning model releases disclose their base training data's reasoning trace composition as a predictor of pause token adoption.
This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error
Coverage behind this analysis
These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.
·arXiv cs.CL
RL's math gains come from 4% of token choices, study finds
A new analysis of reinforcement learning's impact on reasoning reveals that RL-trained models make fundamentally different token choices than base models in only 4% of positions, yet these divergences account for 90% of performance gains on mathematical problems. The finding suggests RL doesn't broadly reshape reasoning but rather teaches models to recognize and exploit specific…
MentionsOLMo-3 · MATH-500
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.