Research Models & Releases·arXiv cs.LG·2d ago

Single-cell foundation models hide failures on rare disease-relevant cells

Single-cell foundation models achieve headline accuracy above 97% yet systematically fail on rare cell populations critical for disease research. A new systematic benchmark across three architectures and six loss functions reveals that standard cross-entropy training masks severe performance gaps on minority classes, with macro-F1 and rare-class recall diverging sharply from overall accuracy. The 162-run study across multiple datasets exposes a fundamental limitation in how foundation models handle long-tail distributions in genomics, forcing practitioners to reconsider whether aggregate metrics adequately reflect real-world utility in clinical and research settings.

Modelwire context

Explainer

The paper's core finding isn't that rare cells are hard to classify (expected), but that standard accuracy metrics actively hide this failure. The divergence between macro-F1 and overall accuracy reveals the benchmark itself is the problem, not just the model.

This echoes a pattern across recent work: aggregate metrics systematically conceal category-level collapse. The SWE agents paper from this week documents the same phenomenon in multi-task RL, where pooled training masks uneven progress across task types. Both papers argue that practitioners deploying these systems in high-stakes domains (clinical genomics, production code generation) are flying blind if they rely on headline numbers. The difference here is domain-specific: genomics has a natural long tail (rare diseases, rare cell states), whereas SWE agents had to engineer one through task diversity.

If the three architectures (scGPT, scBERT, Geneformer) show consistent ranking across the six loss functions tested, that confirms the imbalance problem is architectural, not loss-function-dependent, and would suggest practitioners need to retrain existing models rather than just swap optimizers. If one loss function (focal loss, LDAM, or logit-adjusted softmax) consistently recovers rare-class recall without sacrificing common-class performance across all three models, that becomes a concrete migration path; otherwise the trade-off remains unsolved.

Coverage we drew on

One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents · arXiv cs.LG

This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.

MentionsscGPT · scBERT · Geneformer · focal loss · LDAM · logit-adjusted softmax

Read full story at arXiv cs.LG →(arxiv.org)

Modelwire Editorial

This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.

Our mission How we write

Modelwire summarizes, we don’t republish. arXiv cs.LG originally reported this story as “Rethinking Class Imbalance for Single-Cell Foundation Models: A Systematic Benchmark Across Architectures and Long-Tail Loss Functions”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.