Researchers separate grammaticality from probability in language model internals
Researchers challenge the dominant paradigm for measuring grammatical knowledge in language models by moving beyond probability-based metrics. The work investigates whether grammaticality is encoded as a distinct feature in model internals, rather than conflated with likelihood, lexical frequency, and world knowledge. This distinction matters for interpretability: if models encode grammar as a separable representation, it reshapes how we evaluate their linguistic competence and debug failure modes. The finding could influence how practitioners design probes and evals for downstream tasks requiring robust syntactic reasoning.
Modelwire context
ExplainerThe paper's core claim is not that models know grammar, but that existing evals cannot tell whether they do. By separating grammaticality from likelihood and frequency effects, the work exposes a measurement problem that has been hiding in plain sight across the field's benchmarks.
This connects directly to the tokenization and multilingual work from earlier today (Ge'ez-script study). Both papers identify foundational assumptions in how we measure model competence that turn out to be wrong. Where that work showed tokenizers obscure linguistic capability in non-Latin scripts, this one shows that probability-based metrics obscure grammatical knowledge across all models. The implication is similar: we've been evaluating models through a lens that conflates multiple independent phenomena, making it hard to debug what they actually understand versus what they're pattern-matching on frequency.
If researchers release probes trained on the proposed linear representations and show they predict grammatical acceptability judgments better than perplexity-based baselines on held-out treebank data, the measurement critique holds. If those probes fail to generalize to out-of-distribution syntax (garden-path sentences, rare constructions), the linear encoding may be an artifact of training data rather than genuine grammatical abstraction.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsNeural language models
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Linear representations of grammaticality in neural language models”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.