Gemma-3 fails to detect lies when poisoned sources outnumber truth
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
A new study reveals that language models deployed in retrieval-augmented generation pipelines exhibit a conformity bias that undermines their ability to reject false claims. When poisoned documents outnumber credible sources in retrieved context, Gemma-3-12b's misinformation detection rate plummets from 25% to 10%, suggesting that RAG systems remain vulnerable to coordinated disinformation attacks. This finding exposes a critical gap in production LLM robustness: models trained to follow evidence can be systematically fooled when that evidence is adversarially aligned, raising urgent questions about RAG deployment in high-stakes domains like fact-checking and scientific research.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The study isolates a specific mechanism: models don't just fail on poisoned claims, they fail *proportionally* to how many poisoned sources appear in context. This isn't about hallucination or training data corruption; it's about inference-time susceptibility to adversarial document ratios.
This finding sits directly alongside work on how models weight competing signals during inference. A paper from late October showed that prompt position and source ordering dramatically influence whether models abandon correct answers, and this new result suggests RAG systems inherit that same vulnerability at scale. The conformity effect also echoes the continual learning work from the same week, which found that surprise-driven selection mechanisms can't distinguish planted falsehoods from novel truths. Together, these papers sketch a pattern: LLMs trained to follow evidence can be systematically exploited when that evidence is adversarially structured, whether through poisoned documents in retrieval, misleading prompt ordering, or streaming data selection.
If researchers test whether the conformity effect persists when retrieved documents are explicitly labeled with confidence scores or source reliability ratings, that determines whether the vulnerability is inherent to RAG or fixable through metadata. Watch for follow-up work on whether the decision model classification approach from late October (identifying conflict types before generation) can mitigate this specific bias.
This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error
Coverage behind this analysis
These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.
·arXiv cs.CL
Prompt position drives model susceptibility to false answers more than source
Researchers isolate three independent factors that influence whether language models abandon correct answers when presented with conflicting information. Using Qwen3.5-27B on MMLU-Pro benchmarks, they find that prompt position exerts far stronger influence than the source attribution of advice, with bare claims causing maximum interference immediately before generation while false rationales prove most disruptive when placed…
MentionsGemma-3-12b · SciFact · RAG
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Language Models Show Conformity Bias in RAG: When the Majority Endorses a Poisoned Claim”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.