Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
Researchers tested whether collective intelligence emerges in large agent societies by probing a 2M-agent platform called MoltBook with hierarchical reasoning tasks. The study found no evidence that scale alone produces emergent group intelligence, with agent collectives underperforming individual frontier models on complex reasoning.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The more pointed finding is directional: agent collectives on MoltBook didn't just fail to exceed frontier models, they fell short of them on complex reasoning, which puts a concrete ceiling on the 'more agents equals more intelligence' assumption that underlies several commercial multi-agent architectures currently being built out.
This connects directly to the token consumption research covered the same day ('How Do AI Agents Spend Your Money'), which showed agentic workflows already burn 1000x more tokens than standard inference. Taken together, the two papers sketch a cost-capability picture that should concern anyone scaling agent societies: you pay exponentially more while collective performance may actually regress on hard tasks. The MoltBook result also adds empirical weight to the implicit tension in the aggregate-vs-personalized judges paper, which asked whether pooling LLM evaluators produces better judgments. Apparently pooling agents at scale doesn't reliably produce better reasoning either.
Watch whether the MoltBook team or independent researchers can identify a task class where collective agent performance does exceed frontier solo models. If no such class surfaces within the next two or three benchmark cycles, the architectural case for massive agent societies weakens considerably.
This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error
Coverage behind this analysis
These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.
·arXiv cs.CL
How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks
Researchers analyzed token consumption across eight frontier LLMs running agentic coding tasks on SWE-bench Verified, finding agentic workflows burn 1000x more tokens than traditional code reasoning. The study also evaluates whether models can predict their own token costs before execution, offering practical insights for teams deploying cost-sensitive AI agents.
MentionsMoltBook · Superminds Test · LLM agents
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.