COMPASS: COntinual Multilingual PEFT with Adaptive Semantic Sampling

The core insight COMPASS bets on is that not all multilingual data hurts equally: semantic clustering lets the framework identify which training samples from existing languages are likely to interfere with a new target language, then down-sample them rather than treating all prior data as equally risky. That selectivity is what separates it from standard replay-based continual learning approaches.

The efficiency angle connects directly to recent coverage on this site. The K-Token Merging paper from April 16 attacked inference overhead through compression in latent space; COMPASS attacks a different bottleneck, the data-selection cost of multilingual adaptation, using semantic structure rather than architectural changes. Both reflect the same underlying pressure: practitioners need LLMs to extend to new domains or languages without full retraining, and the field is generating a range of PEFT-adjacent techniques to meet that need. The RespondeoQA benchmark (also April 22) is a useful counterpoint: it shows that even after adaptation, models struggle with skill-oriented tasks in low-resource language settings, which is precisely the failure mode COMPASS is trying to reduce upstream.

The meaningful test is whether COMPASS's interference-reduction holds on genuinely low-resource languages rather than the higher-resource ones that dominate multilingual benchmarks. If follow-up evaluations include languages outside the top 20 by training-data volume and the gains persist, the semantic clustering approach is doing real work; if results degrade there, the method may be exploiting data density rather than solving the interference problem.