Researchers train models to detect deception using game-based reasoning dataset
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
Researchers have created AVCORP, a 5,000-utterance dataset from the deception game Avalon, to train language models in detecting coordinated deception through Theory of Mind reasoning. The work maps 37 tactical behaviors and captures nested belief structures, enabling models like Qwen3-8B to outperform untrained baselines and frontier models on held-out and human-played games. This addresses a critical vulnerability in LLM-based agents: their susceptibility to social manipulation. The dataset and training approach establish a foundation for building adversarially robust agents that reason about intent and belief rather than surface-level patterns, directly relevant to multi-agent systems and real-world deployment scenarios.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The key innovation isn't just detecting lies, but training models to reason about nested beliefs and tactical intent. AVCORP maps 37 distinct deception behaviors within a structured game, forcing models to track what each player believes others believe, rather than pattern-matching surface-level falsehoods.
This work directly addresses a gap exposed in recent coverage. The HaluCheck benchmark (Nov 5) showed that LLMs fail at fine-grained error localization in multi-step reasoning, particularly in agent workflows. AVCORP tackles the complementary problem: agents that can localize errors but remain vulnerable to coordinated social manipulation. Where reverse game theory (MIT, Nov 3) embeds incentives to keep agents aligned, this work builds the perceptual layer agents need to recognize when they're being deliberately misled. The two approaches are adjacent: incentive design fails if the agent can't detect the attack in the first place.
If Qwen3-8B and GPT-5.6 Sol maintain their performance advantage on human-played Avalon games (not just held-out test sets) over the next two quarters, that confirms the Theory of Mind training transfers to real adversarial interaction. If performance collapses on new deception tactics not in the 37-behavior taxonomy, that signals the model memorized tactics rather than learning reasoning.
This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error
Coverage behind this analysis
These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.
·arXiv cs.CL
LLMs struggle to localize hallucinations despite basic detection ability
Researchers have released HaluCheck, a 42k-instance benchmark that exposes a critical gap in LLM reliability: while models can detect hallucinations at a surface level, they fail consistently at pinpointing false claims within text or tracing errors through agent workflows. Testing across fourteen LLMs reveals that fine-grained localization and correction remain unsolved, particularly for agent-based systems…
MentionsAVCORP · Avalon · Qwen3-8B · GPT-5.6 Sol · Theory of Mind
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “AVCORP: Identifying and Resisting Deception via Communication Theory and Theory of Mind”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.