Surprisal theory lacks falsifiability without constraining language models
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
A new paper challenges the empirical foundation of surprisal theory, a dominant framework linking language model predictions to human reading difficulty. The author demonstrates that any observed difficulty pattern can be fit by some language model, rendering the theory unfalsifiable without additional constraints. This exposes a two-decade implicit assumption in psycholinguistics: that the relevant model is the corpus distribution. The finding threatens a core justification for using LLM loss as a cognitive proxy and forces the field to either specify what makes a language model theoretically valid for human cognition or abandon surprisal as a predictive framework.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The paper's real contribution isn't just that surprisal theory can be fit by any model. It's the identification of an unstated premise: that psycholinguists have been implicitly assuming the 'right' language model is one trained on natural text corpora, without ever justifying that choice theoretically.
This is largely disconnected from recent activity in the space of LLM capability launches and benchmarking. Instead it belongs to a slower-moving conversation about whether language model internals are valid proxies for human cognition at all. The finding matters because it pulls back the curtain on a two-decade empirical program in psycholinguistics that has borrowed credibility from LLMs without establishing the theoretical link. If the field can't specify what makes a model 'valid' for cognition, then LLM loss becomes just another curve-fitting tool, not a window into the mind.
Watch whether major psycholinguistics labs (MIT, UC San Diego, Edinburgh) publish follow-ups within the next 12 months that either propose explicit constraints on which language models count as cognitively plausible, or pivot to testing surprisal against non-LLM baselines. If silence prevails, the field has tacitly accepted the critique.
This interpretation is generated from the summary above and available source metadata. Our methodology · Report an error
MentionsSurprisal theory · Language models · Psycholinguistics
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Surprisal Theory is Tautological (without Rational Grounding)”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.