Predictable Confabulations: Factual Recall by LLMs Scales with Model Size and Topic Frequency
Source published ·Modelwire updated
Original coverage: arXiv cs.LG ↗·How Modelwire adds context
The development
Researchers have identified a predictable scaling relationship governing how well language models recall factual information, linking performance to both model size and training-data topic frequency through a sigmoid function. The finding, validated across 38 models and 8,900 scholarly references, explains 60-94% of variance in recall quality and suggests factual accuracy is fundamentally gated by a signal-to-noise ratio where concept prevalence acts as signal and model capacity as noise floor. This quantification of the factual-recall scaling law provides practitioners with a framework for predicting hallucination risk and informs decisions about model selection and training-data curation for knowledge-intensive applications.
Modelwire’s AI-generated summary of coverage from arXiv cs.LG.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The more actionable buried finding is directional: topic frequency in training data matters as much as raw model scale, meaning throwing more parameters at a knowledge-sparse domain won't reliably fix hallucination. That reframes data curation as a first-class engineering decision rather than a preprocessing afterthought.
This connects meaningfully to the attention efficiency work we covered in 'DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention' from the same day. DashAttention is trying to reduce compute at inference without sacrificing quality, but this factual-recall paper implies that quality floors are set earlier, during training data composition, not at inference time. The two findings together suggest a division of labor: architectural efficiency work optimizes what a model does with what it knows, while data curation determines the ceiling on what it can know in the first place. That distinction matters for teams deciding where to invest engineering resources.
Watch whether any of the 38 models tested show the sigmoid relationship breaking down at very high topic frequency, which would indicate a saturation regime where additional data stops helping. If that threshold gets quantified in follow-up work, it becomes a concrete budget signal for training data acquisition.
This interpretation is generated from the summary above and available source metadata. Our methodology · Report an error
MentionsLLMs · scaling laws · factual recall · training data composition · signal-to-noise ratio
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.