AI models often give the right answers but point to the wrong sources
A systematic gap has emerged in how leading language models justify their outputs. Researchers at Peking University documented that GPT and Gemini frequently cite document passages that don't actually support their conclusions, even when final answers prove correct. This 'attribution hallucination' poses material risk in regulated domains like law and medicine where reasoning transparency is non-negotiable. The new CiteVQA benchmark provides the first standardized test for this failure mode, shifting evaluation focus from answer accuracy alone to the integrity of supporting evidence chains.
Modelwire context
ExplainerThe more unsettling implication buried in this finding is that attribution hallucination is nearly invisible in standard evaluations: models score well on answer accuracy while silently fabricating their evidentiary basis, meaning current deployment safeguards in high-stakes domains may be measuring the wrong thing entirely.
This is largely disconnected from recent activity in our archive, as Modelwire has no prior coverage to anchor it to. It belongs, however, to a broader and well-documented conversation in the research community about the gap between benchmark performance and real-world reliability. The specific failure mode here sits adjacent to hallucination research but is meaningfully different: the model is not wrong about the world, it is wrong about its own reasoning trail. That distinction matters most in legal and medical contexts, where a correct conclusion built on a fabricated citation chain can expose practitioners to liability even when the underlying answer holds up.
Watch whether legal-tech or clinical AI vendors that currently cite GPT or Gemini in their compliance materials respond to CiteVQA with their own attribution audits within the next two quarters. Silence from that segment would itself be informative.
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsGPT · Gemini · Peking University · CiteVQA
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. The full content lives on the-decoder.com. If you’re a publisher and want a different summarization policy for your work, see our takedown page.