Open models fail to calibrate confidence thresholds despite abstention prompts
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
Researchers testing threshold-based abstention prompts on open 3-8B language models found that explicit formatting dramatically shifts behavior, but not in the expected direction. When models were given a structured abstention option, the abstention rate jumped from 20% to 91% at a 0.75 confidence threshold. However, varying the threshold itself had minimal effect within that format, suggesting models may be gaming the format rather than genuinely calibrating to confidence levels. Without the structured format, higher thresholds paradoxically lowered abstention rates, with up to 40% of outputs violating threshold-independent compliance. The finding challenges assumptions about prompt-based control mechanisms and raises questions about whether current open models can reliably implement stated scoring rules.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
Skeptical readOur AI-generated reading of the wider context and the next developments to watch.
The real finding isn't that abstention fails, but that models are exquisitely sensitive to formatting cues. When given a structured option, they take it at near-ceiling rates regardless of threshold variation. This suggests the models are complying with an implicit instruction (use the format when available) rather than gaming a scoring rule.
This connects directly to the November 8th work on system prompts and response bias, which found that prompt framing drives behavior independent of genuine discrimination ability. Here we see the same pattern: models aren't failing to calibrate confidence, they're responding to structural affordances in the prompt itself. The earlier BMMS framework (November 6th) proposed policy-conditioned compliance measurement, but this paper reveals a prior problem: even when policies are stated explicitly, models may be following meta-instructions about format rather than the rule's actual logic. The gap between stated rules and executed behavior is narrower than it appears.
If researchers test the same models with identical thresholds but randomized formatting (e.g., structured vs. natural language vs. JSON), and abstention rates remain stable across formats while threshold effects persist, that confirms models are genuinely calibrating. If abstention rates track format instead, the control problem is about prompt design, not model reasoning. Results should land within 4 weeks given the paper's recent publication.
This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error
Coverage behind this analysis
These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.
·arXiv cs.CL
System prompts drive LLM safety more than actual safety reasoning
A new evaluation framework reveals that LLM safety hinges less on models' ability to distinguish safe from unsafe requests and more on their willingness to refuse based on system prompt cues. Researchers tested 13 models across three major benchmarks using signal detection theory, finding that prompt framing drives response patterns independent of genuine safety discrimination.…
MentionsKalai et al. · arXiv
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Abstention Is Not Threshold Compliance: Stated Scoring Rules in Open Language Models”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.