Research·arXiv cs.CL·22h ago

Reasoning models outperform on theory of mind through robustness, not reasoning

Reasoning-optimized language models trained via reinforcement learning show markedly stronger robustness to prompt and task variations on Theory of Mind benchmarks compared to standard LLMs. This work challenges assumptions about whether apparent ToM capabilities reflect genuine understanding or emerge from improved stability under perturbation. The finding matters for model evaluation: it suggests reasoning models may succeed not through deeper cognitive modeling but through architectural resilience, reshaping how practitioners should interpret performance gains on psychological reasoning tasks and informing design choices for systems requiring reliable behavior across input distributions.

Modelwire context

Skeptical read

The paper's real contribution is negative: it suggests reasoning models may not possess deeper Theory of Mind understanding at all, only better stability under perturbation. This reframes what looked like a capability gain into a measurement artifact.

This connects directly to the robustness evaluation crisis documented across recent work. The Vision-Language Models study from August 5th exposed how diagnostic accuracy masks fragility under input reordering, and the Subtype Robustness paper from August 2nd revealed that models can maintain high confidence precisely where they fail on unseen variants. This Theory of Mind work follows the same pattern: benchmark performance on ToM tasks may reflect architectural resilience rather than genuine cognitive modeling, meaning practitioners need to distinguish between 'robust to noise' and 'understands minds.' The latent chain-of-thought paper also matters here because if reasoning is compressed into unreadable vectors, we lose the ability to audit whether that reasoning actually reflects psychological insight or just statistical stability.

If the same reasoning models show comparable ToM robustness gains on out-of-distribution psychology benchmarks (e.g., Sally-Anne variants with novel agent counts or temporal orderings not seen during training), that would confirm the finding generalizes. If performance collapses on truly novel ToM scenarios, it signals the robustness is specific to benchmark perturbations, not genuine understanding.

Coverage we drew on

Evaluating the Diagnostic Robustness of Vision-Language Models Under Visual and Textual Perturbations · arXiv cs.CL

This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.

MentionsLarge language models · Theory of Mind · Reinforcement learning

Read full story at arXiv cs.CL →(arxiv.org)

Modelwire Editorial

This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.

Our mission How we write

Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.