Language models learn human-like grammar biases without direct training data
Language models spontaneously develop human-like biases in grammatical structure without explicit training signals, suggesting that linguistic preferences emerge from general learning principles rather than innate architectural constraints. Researchers trained models on corpora stripped of multi-modifier noun phrases, then tested whether the models would infer scope-homomorphic ordering preferences when encountering such constructions. Across multiple model scales, the systems consistently replicated this human cognitive bias, indicating that LLMs may acquire linguistic universals through the same underdetermined learning process that shapes human language acquisition. This finding bridges cognitive science and AI, revealing convergence between neural networks and human cognition on fundamental language patterns.
Modelwire context
ExplainerThe paper doesn't just show models learn human biases, but that they do so without any architectural pressure to do so, suggesting linguistic universals emerge from optimization dynamics rather than built-in constraints. This reframes what we should attribute to inductive bias versus what falls out of general learning.
This connects directly to the August 1st finding on exemplar versus abstraction learning. That paper showed we've been misinterpreting what models actually learn by conflating memorization with abstraction. Here we see a similar pattern: what looks like an innate linguistic preference might simply be what emerges when you optimize a general learner on natural data. The two papers together suggest we're systematically overestimating how much model architecture determines behavior, and underestimating how much comes from the learning process itself.
If researchers can show that models trained on corpora with artificially reversed modifier orderings develop the opposite preference (rather than defaulting to human-like ordering), that confirms the finding is about learning from data rather than a hidden architectural bias. If the preference persists regardless of training distribution, the claim collapses.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsLanguage models · Artificial Language Learning
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Language Models Generalize to Human-like Word Order Preferences”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.