Chinese neologisms expose LLM reasoning gaps across 18 models
Researchers have exposed a significant blind spot in how leading language models handle Chinese linguistic innovation. CNeo-Bench, a curated dataset of 4,759 neologisms spanning phonetic substitution and character-based wordplay, reveals that 18 tested models struggle fundamentally with both recognizing and manipulating these expressions, with most scoring below 40% on definition tasks. The work surfaces a deeper architectural vulnerability: models can sometimes describe novel terms without grasping the generative mechanisms that produce them. This gap matters because it signals that current LLMs lack robust handling of language evolution in non-English contexts, a capability increasingly important as multilingual deployment scales.62
