Who Checks the Citations? Benchmarking Legal Hallucination Detection
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
Legal AI hallucination remains a persistent, worsening problem despite model improvements and professional consequences. Researchers benchmarked five models' ability to automatically detect fabricated citations in court filings, finding that GPT-5 achieves 82.8% recall but only 60.5% F1 score. The work introduces a taxonomy and dataset grounded in over 1,000 real cases of citation fabrication filed year-over-year, establishing that neither scaling nor sanctions have solved the core reliability gap. This signals that downstream verification tools, not just better base models, are becoming essential infrastructure for high-stakes AI deployment.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
Analyst takeOur AI-generated reading of the wider context and the next developments to watch.
The recall-precision gap buried in the numbers is the actual finding: GPT-5 catches most fabricated citations but generates enough false positives that a human reviewer still can't trust the output and walk away. That asymmetry matters enormously for any vendor selling legal AI as a time-saving tool, because noisy alerts erode the productivity case faster than missed ones.
This connects directly to the coherence illusions paper covered the same day ('When Context Misleads'), which showed that surface fluency masks semantic failure in ways models themselves cannot reliably detect. Both papers are pointing at the same structural problem from different angles: models that produce confident, well-formed output are not the same as models that produce accurate output. Together they reinforce that verification infrastructure is not a nice-to-have layer on top of capable base models, it is load-bearing. The legal domain just makes the consequences legible because courts leave paper trails.
Watch whether any of the major legal AI vendors (Thomson Reuters, LexisNexis, Harvey) ship a dedicated citation-verification module citing this taxonomy within the next two quarters. If they do, this benchmark becomes the de facto evaluation standard; if they build proprietary alternatives instead, it signals the research community and the product layer are still operating on separate tracks.
This interpretation is generated from the summary above and available source metadata. Our methodology · Report an error
MentionsGPT-5 · arXiv
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.