Open-weight models encode output length in hidden states before generation
Researchers have discovered that open-weight language models encode predictions about their own output length directly in hidden states, detectable before generation begins. By training lightweight linear probes on frozen representations from 7-8B parameter models across multiple datasets, the team found response length is linearly decodable from the prompt's final hidden state alone. Critically, probe directions learned on natural datasets generalize to unseen synthetic tasks, suggesting models develop a robust internal mechanism for length planning. This finding illuminates how LLMs structure generation and opens new angles for mechanistic interpretability work, with implications for understanding model internals and potentially improving length control during inference.
Modelwire context
ExplainerThe more provocative implication buried in the finding is that length planning appears to be a pre-generation commitment, not an emergent property of token-by-token decoding. That reframes a long-standing assumption: that LLMs have no meaningful 'plan' before they start writing.
This sits squarely in the mechanistic interpretability thread Modelwire has been tracking. The gradient-based inversion paper from July 1st showed that hidden states carry recoverable input information; this new work extends the logic in the opposite direction, showing hidden states also carry recoverable output intentions. Similarly, the Logit-Contribution Scoring paper (also July 1st) demonstrated that specific attention heads perform computations invisible to surface-level analysis. Together these papers are building a picture of transformer internals as far more structured and pre-loaded than the 'next token prediction' framing suggests. The broader survey 'Understanding Large Language Models' from July 1st provides the conceptual scaffolding for why this matters: distinguishing architectural artifacts from genuine planning-like behavior is exactly the kind of question that survey flagged as unresolved.
The critical test is whether these linear probe directions hold in instruction-tuned or RLHF-trained models at larger scales (30B+), since those models face explicit length-shaping pressure during training. If the linearity breaks down there, the mechanism may be a base-model artifact rather than a general property of transformer generation.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsarXiv
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.LG originally reported this story as “How Much is Left? LLMs Linearly Encode Their Remaining Output Length”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.