New benchmark tests whether LLMs distinguish expert claims from their own beliefs
Researchers have introduced ESFP, a behavioral benchmark that measures whether large language models can coherently shift between different epistemic registers when prompted. The test distinguishes between requests asking what experts believe versus what the model itself believes, evaluating whether systems respond with appropriate neutrality or stance expression. This addresses a gap in existing safety and instruction-following benchmarks by directly probing a model's ability to maintain trustworthy conversational boundaries. The work matters for deployment because it isolates a specific failure mode: systems that conflate external attribution with self-assertion could mislead users about the model's actual knowledge or confidence.
Modelwire context
ExplainerESFP doesn't measure whether models know facts or follow instructions well. It specifically probes whether models can maintain a coherent epistemic boundary: treating 'expert consensus' as reportage versus treating 'model confidence' as stance. That distinction is orthogonal to accuracy.
This connects directly to the LLM judge study from the same day, which found that models systematically overrate responses when ground truth is absent. Both papers expose how models conflate different types of claims. ESFP isolates the register-shift problem (attribution vs. assertion), while the judge paper shows models fail to calibrate confidence appropriately. Together they suggest a pattern: models struggle to maintain epistemic humility across different conversational framings, which matters for deployment because users can't tell whether a model is reporting external knowledge or asserting its own belief.
If ESFP scores correlate inversely with hallucination rates on knowledge-grounded tasks (like medical misconception handling in ThReadMed-QA), then register flexibility is a genuine safety signal worth optimizing during training. If the correlation is weak or absent, ESFP measures something orthogonal to real-world trustworthiness.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsESFP
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Epistemic Stance Flexibility Probing: Measuring Prompt-Conditioned Register Shift in Large Language Models”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.