LLMs conflate two refusal problems, leaving false premises undetected
Researchers have identified a fundamental blind spot in how instruction-tuned LLMs decide when to refuse answers. Current confidence-based refusal mechanisms conflate two distinct problems: rejecting incorrect outputs versus declining unanswerable or false-premise questions. Testing across five models from three families reveals that answer confidence tracks correctness but ignores answerability, while hidden-state probes show the inverse pattern. This gap persists across model scales and is most severe on naturally occurring false-premise questions, where standard confidence metrics perform near chance. The finding suggests that single-threshold refusal strategies are architecturally insufficient and that safety-critical deployments may need dual detection pathways.
Modelwire context
ExplainerThe practical implication buried in this finding is that models can appear well-calibrated on standard benchmarks while systematically failing on false-premise inputs, because correctness-based confidence metrics were never designed to detect questions that shouldn't be answered at all. These are different cognitive tasks being handled by a single, inadequate gate.
This connects directly to the reliability measurement problems surfaced in 'When the Judge Changes, So Does the Measurement,' published the same day. That paper showed that evaluation systems produce inconsistent scores depending on which judge is used. This paper adds a prior layer to that problem: if the model being judged is miscalibrated on answerability in the first place, judge-level inconsistency compounds an already broken signal. Together, they sketch a fragile evaluation stack where neither the model's self-assessment nor the external judge can be fully trusted. The 'Stop Guessing When to Stop Testing' piece on sequential evaluation efficiency is also relevant, since adaptive stopping rules assume the underlying correctness signal is meaningful, an assumption this paper puts under pressure.
Watch whether any of the three model families tested (or their successors) ship explicit dual-pathway refusal mechanisms in the next two release cycles. If they do, it confirms the field has absorbed this finding. If refusal logic stays single-threshold, the gap will show up again in the next false-premise benchmark audit.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsCREPE · instruction-tuned models
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Two Axes of LLM Abstention: Answer Correctness and Question Answerability”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.