LLMs reach right diagnoses from wrong evidence, new medical benchmark reveals
Researchers have identified a critical failure mode in LLM medical reasoning: models often reach correct diagnoses despite weak or misleading evidence, masking poor evidential grounding beneath outcome-based accuracy metrics. The new MedEVM benchmark exposes this Evidence-Value Misalignment across nine LLMs by simulating dynamic clinical workflows where observations arrive sequentially and models must decide when sufficient evidence exists to diagnose. Early findings reveal that even capable models fail to properly track evidence sufficiency, a gap that matters deeply for clinical deployment where lucky guesses carry real harm. This work reframes how the field should evaluate medical AI beyond raw accuracy.
Modelwire context
ExplainerThe paper's core finding isn't just that LLMs sometimes guess right for wrong reasons (known), but that standard accuracy metrics actively hide this failure mode, making it invisible to practitioners building clinical systems. This means current deployment decisions rest on incomplete safety signals.
This connects directly to the QuanReview work from late September, which tackled auditability in LLM-assisted annotation workflows. Both papers share a diagnosis: silent corruption occurs when we accept model outputs without explicit logging of the reasoning chain. MedEVM extends that concern from data labeling into clinical decision-making, where the stakes are higher and the audit trail more opaque. The Transformer read-blindness paper from the same period also hints at a related structural problem: models can fail to self-correct anomalies they generate, suggesting that evidence-tracking failures may have architectural roots, not just training-data roots.
If MedEVM's nine-model benchmark is adopted by at least two major medical AI vendors (or referenced in FDA guidance) within 12 months, the field has accepted that evidence-value alignment is a required safety gate. If it remains confined to academic citations without downstream adoption, the gap between what researchers measure and what practitioners deploy remains unresolved.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsMedEVM · LLMs
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Can LLMs Value the Right Evidence? Evidence-Value Misalignment in Dynamic Medical Diagnosis”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.