Skip to content
Modelwire
Subscribe

Study reveals how LLMs export English grammar errors across languages

Source published ·Modelwire updated

Original coverage: arXiv cs.CL ↗·How Modelwire adds context

The development

Researchers have created a systematic benchmark to measure how multilingual LLMs incorrectly transfer grammatical patterns from English into other languages, a phenomenon called constructional bleeding. Using 5,583 sentence pairs across seven languages, the study quantifies the degree to which models overgeneralize English morphosyntactic strategies despite training predominantly in English. This work exposes a fundamental cross-lingual generalization failure in current LLMs, revealing that models don't learn language-specific constraints as robustly as assumed. The findings matter for practitioners deploying multilingual systems in production, where such errors degrade output quality and user trust in non-English markets.

Modelwire’s AI-generated summary of coverage from arXiv cs.CL.

Modelwire analysis

Explainer

Our AI-generated reading of the wider context and the next developments to watch.

CALQUE doesn't just measure multilingual performance; it isolates a specific directional failure: English patterns leaking into other languages rather than the reverse. This asymmetry matters because it reveals which language the model treats as the default reasoning substrate, not just that multilingual transfer is imperfect.

This connects directly to the typological feature induction work from early November, which found that LLMs rely heavily on pretraining knowledge rather than genuine cross-lingual reasoning. CALQUE operationalizes that finding: if models can't infer language-specific morphosyntactic constraints from context, they'll default to the dominant language in their training distribution. The romanized Indic evaluation gap from the same week also points to a related problem: our benchmarks miss language-specific failure modes because they're built around English-centric metrics. Together, these three papers suggest the issue isn't just multilingual coverage but the architecture's inability to learn and apply language-specific rules robustly.

If CALQUE's 5,583 sentence pairs show constructional bleeding correlates with model size or training data composition (e.g., larger models bleed more, or models trained on more English-heavy corpora bleed more), that would confirm the mechanism is learnable rather than architectural. If the effect persists unchanged across the next generation of multilingual models released in Q1 2027, it signals the industry isn't treating this as a priority.

This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error

Coverage behind this analysis

These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.

  1. ·arXiv cs.CL

    LLMs rely on memorized language knowledge, not inductive reasoning

    Researchers have built a diagnostic benchmark that isolates a critical failure mode in LLM reasoning: the ability to infer linguistic rules from minimal evidence. Rather than conflating rule discovery with rule application, typological feature induction tests whether models can deduce language structure from paired sentences and translations across 195 languages. The finding that LLMs achieve…

    Read Modelwire coverage →Original source ↗

MentionsCALQUE

MW

How this coverage is produced

Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.

Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “I play of the cello: Evaluating Constructional Bleeding in Multilingual LLMs with CALQUE”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.

More to explore

Study reveals how LLMs fake economic forecasting through training-data leakage

arXiv cs.CL·

LLMs fail robustness tests that fine-tuned models pass

arXiv cs.CL·

Geometric constraints limit cross-lingual knowledge editing in transformers

Study reveals how LLMs export English grammar errors across languages · Modelwire