Research·arXiv cs.CL·1d ago

Language models abandon correct answers when given false source cues

Researchers have identified a critical vulnerability in instruction-following language models: they abandon correct answers when exposed to misleading source attribution cues. The team developed a new metric, neutral-conditioned misleading cue adoption rate, to measure answer instability independent of model knowledge. Testing four major models across MMLU-Pro and IndicMMLU-Pro in five languages revealed widespread susceptibility to false authority signals. This finding exposes a fundamental robustness gap in production models and suggests that source-attributed prompts, common in real-world deployments, can systematically degrade accuracy without triggering obvious failure modes.

Modelwire context

Explainer

The paper isolates a specific mechanism: models don't just fail on hard questions, they actively abandon correct answers when exposed to false authority signals. The neutral-conditioned metric matters because it separates this susceptibility from genuine knowledge deficits, revealing a behavioral flaw rather than a capability gap.

This connects directly to the evidence-grounding work from earlier this month. Both papers expose models that appear to follow instructions (cite sources, defer to evidence) while actually bypassing the reasoning they're supposed to perform. The current study shows models will flip answers based on misleading cues; the ablation framework showed they ignore evidence they're supposed to evaluate. Together they suggest production systems are vulnerable to prompt-injection-like attacks that exploit the gap between stated reasoning and actual computation.

If the same four models show comparable failure rates when tested on MMLU-Pro's held-out validation split (not used in the paper's training data), that confirms the vulnerability is robust. If accuracy recovers to baseline when source cues are removed entirely, that proves the instability is cue-driven rather than a side effect of the evaluation setup itself.

Coverage we drew on

Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablation · arXiv cs.CL

This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.

MentionsMMLU-Pro · IndicMMLU-Pro

Read full story at arXiv cs.CL →(arxiv.org)

Modelwire Editorial

This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.

Our mission How we write

Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “When Models Defer to Wrong Answers: A Robustness Audit of Source-Attributed Cues in Multiple-Choice QA”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.