
Heterogeneity in Formal Linguistic Competence of Language Models: Is Data the Real Bottleneck?
Researchers found that GPT-2 Small models trained on web data struggle with specific grammatical constructions, but injecting just 1% synthetic data targeting those phenomena recovered performance across 8 of 9 failing linguistic benchmarks, suggesting data scarcity rather than architectural limits drive formal linguistic gaps.62




























