
Overcoming Decoder Inconsistencies in Whisper for Dravidian and Low-Resource Languages
Whisper's multilingual ASR capabilities reveal a structural weakness in how decoder attention balances linguistic and acoustic signals, particularly for morphologically complex, low-resource languages like Tamil and Telugu. Researchers identified that Dravidian languages trigger sparse token distributions and character-level errors due to longer words and higher vocabulary diversity, exposing a fundamental mismatch between self-attention and cross-attention mechanisms. The proposed Weighted-Attention decoder enhancement addresses this imbalance, signaling that scaling multilingual models without architectural fixes leaves entire language families underserved. This work matters for anyone building production ASR systems targeting non-English markets and highlights how model design assumptions baked into high-resource language training can systematically disadvantage linguistic minorities.62


























