Preferences of a Voice-First Nation: Large-Scale Pairwise Evaluation and Preference Analysis for TTS in Indian Languages

The paper's most underreported contribution isn't the rankings of the seven systems tested, but the diagnostic framework itself: by decomposing listener preference into separate perceptual dimensions, the researchers expose that aggregate MOS scores mask systematic failures specific to certain language families, particularly those with tonal or morphologically complex structures.

This is largely disconnected from recent activity covered on Modelwire, which has focused on LLM benchmarking (the OptiVerse paper from April 23 tested 22 models on optimization reasoning) and consumer AI product moves. The TTS evaluation space sits in a quieter corner of speech research, where the core problem is that most evaluation infrastructure was built around English and a handful of high-resource European languages. The 1,900-speaker scale here is notable precisely because recruiting native speakers across 10 Indian languages is a logistical constraint that most academic labs simply don't clear, which is part of why multilingual speech quality has lagged multilingual text model quality by a visible margin.

Watch whether the multidimensional evaluation framework gets adopted by Indic language model initiatives like AI4Bharat or similar government-backed programs within the next 12 months. If it does, that signals the methodology is portable; if those groups continue using aggregate MOS, the framework stays a one-off research artifact.