Vision-language models for chest radiography do not always need the image
Source published ·Modelwire updated
Original coverage: arXiv cs.LG ↗·How Modelwire adds context
The development
A new causal audit methodology reveals that leading vision-language models trained on chest radiographs may rely heavily on text priors rather than visual analysis. Researchers tested nine systems by occluding image regions and swapping scans, finding that text-only baselines matched multimodal performance within 5.7 percentage points, and a 119B parameter model showed no statistical advantage over a 7B text-only variant. This challenges the field's assumption that strong benchmark scores on medical imaging tasks reflect genuine visual reasoning, raising questions about model transparency in high-stakes clinical deployment and the validity of current evaluation standards.
Modelwire’s AI-generated summary of coverage from arXiv cs.LG.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The deeper issue isn't that these models are bad at vision; it's that current benchmarks weren't designed to distinguish genuine visual reasoning from sophisticated text-pattern matching. A model can score well by learning that certain clinical phrases predict certain diagnoses, regardless of what the scan actually shows.
This connects to a recurring theme in recent coverage: the gap between what models appear to do and what they actually do internally. The 'Blind Recovery of Latent Domains via Unsupervised Symmetry Discovery' paper from the same day addresses a related problem in a different register, asking how we recover the true underlying signal when the measurement process itself is corrupted or opaque. Here, the corruption is conceptual rather than mathematical: the evaluation signal looks clean but is measuring the wrong thing. Neither story is directly about medical AI, but together they point toward a broader audit problem across applied ML, where surface performance masks structural ambiguity about what a model has actually learned.
Watch whether any of the nine audited systems publish updated evaluation protocols that include text-only ablations as a required baseline. If major clinical AI vendors adopt that standard within the next 12 months, the field is self-correcting; if they don't, regulatory bodies will likely impose it.
This interpretation is generated from the summary above and available source metadata. Our methodology · Report an error
MentionsVision-language models · Chest radiography · Multimodal models · Text-only baseline
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.