Sparse circuit optimization cuts multilingual translation tuning to 5% of parameters
Researchers have developed TITAN, a mechanistic framework that identifies and optimizes specific neural pathways responsible for translation in multilingual language models. By combining causal intervention with sparse feature analysis, the approach isolates critical translation circuits and fine-tunes only those components, achieving performance gains while modifying fewer than 5% of model parameters. This work matters because it demonstrates how targeted circuit optimization can enable efficient adaptation for low-resource language pairs like Hindi-to-Sindhi, offering a scalable alternative to full-model retraining and advancing our understanding of how multilingual models internally route language tasks.62
