Finetuning with Scientific Data Increases Hallucinations: A Multi-domain Factuality Evaluation of LLMs
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
A new benchmark reveals a counterintuitive risk in the push to specialize LLMs for scientific work: domain-specific fine-tuning actually amplifies hallucination rates compared to general-purpose base models. SciFactCheck evaluates 18 models across five scientific domains using a nuanced framework that distinguishes unverifiability, overclaiming, and attribution errors. This finding challenges the assumption that narrow training improves factuality in high-stakes domains like biomedicine and chemistry, forcing practitioners to reconsider whether specialized models require additional safeguards or architectural changes to maintain reliability.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The more pointed finding isn't just that fine-tuned models hallucinate more, it's that SciFactCheck distinguishes between three failure modes (unverifiability, overclaiming, and attribution errors), which means the problem isn't monolithic and different domains likely fail in different ways. That granularity is what makes this actionable rather than merely alarming.
This lands directly alongside our coverage of MedHal-Loc (also published June 19), which exposed a parallel problem: medical hallucination detectors that claim to localize errors often can't actually do so faithfully. Together, these two papers sketch a troubling picture for clinical AI deployment, where both the models generating outputs and the detectors meant to catch errors are less reliable than their architectures imply. The radiology report generation work we covered the same day, which introduced precision-recall tradeoffs as a deployment control, looks more prescient in this light: if fine-tuning reliably worsens factuality, then inference-time behavioral controls may be a more tractable path than training-time fixes.
Watch whether any of the 18 evaluated models release updated fine-tuning approaches that specifically target the attribution error category SciFactCheck identifies. If overclaiming rates drop without unverifiability rising, that would suggest the failure modes are separable and addressable independently.
This interpretation is generated from the summary above and available source metadata. Our methodology · Report an error
MentionsSciFactCheck · LLMs
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.