LLM judges overrate wrong answers without reference data
A new study reveals a critical flaw in using LLMs as evaluation judges for open-ended tasks: without reference answers, these models systematically overrate incorrect responses. The research combines calibration testing with sensitivity analysis across three languages to show that judge performance degrades significantly when ground truth is absent, and improves only when reference answers are explicitly provided in prompts. This finding directly challenges the growing practice of using LLM judges as a scalable alternative to human evaluation, suggesting that practitioners relying on reference-free assessment may be getting inflated quality signals that mask actual model performance gaps.
Modelwire context
ExplainerThe study's multilingual scope across three languages is the detail worth flagging: this isn't a quirk of English-language benchmarks but a consistent pattern, which makes it harder to dismiss as an artifact of a particular dataset or prompt style.
This connects directly to the MemOps benchmark paper covered the same day, which made a parallel argument in a different domain: that evaluation frameworks crediting final outputs can mask unreliable internal processes. Both papers are pushing against the same tendency to treat correctness signals as sufficient. The LLM-judge finding adds a sharper edge to that critique, because LLM judges are themselves often used to score the kinds of agent outputs MemOps is trying to diagnose. If the judge is inflating scores when no reference answer exists, then any benchmark relying on reference-free LLM evaluation is compounding the problem MemOps identified.
Watch whether major eval frameworks like HELM or LM-Eval Harness issue guidance on mandatory reference inclusion for open-ended tasks within the next two quarters. Adoption there would signal the finding has moved from paper to practice.
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsLLM judges
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “LLM Judges Can Be Too Generous When There Is No Reference Answer”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.