DiscoBench shows search agents fail by searching, not asking
Source published ·Modelwire updated
Original coverage: The Decoder ↗·How Modelwire adds context
The development
A new benchmark called DiscoBench reveals a critical failure mode in AI search agents: they perform worse when they attempt repeated searches on ambiguous queries instead of requesting clarification from users. The best-performing models achieve only 43 percent accuracy overall, while those that search iteratively without asking follow-up questions drop to 51.9 percent. Removing query ambiguity improves accuracy by up to 40 points, suggesting that agent design must prioritize interactive disambiguation over autonomous search loops. This finding reshapes expectations around agentic AI systems, indicating that production search agents need explicit uncertainty handling and user interaction protocols to function reliably.
Modelwire’s AI-generated summary of coverage from The Decoder.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The benchmark's deeper implication isn't that these agents are bad at searching, it's that they're architecturally overconfident: they default to autonomous looping rather than surfacing uncertainty to users, which means the failure mode is baked into how most agentic pipelines are designed, not just how individual models perform.
This connects directly to the multi-agent architecture work covered in 'Conversable Complexity' from early July, which framed agentic LLM collectives as systems where linguistic interaction is the primary mechanism for transparency and coordination. DiscoBench suggests that same interaction layer is being bypassed in production search agents, with agents substituting iteration for communication. It also rhymes with the RAG diagnostic work covered around the same time ('What Survives Into Context'), where the problem wasn't retrieval quality per se but what gets prioritized under constraint. Both papers point at the same design gap: agents optimizing for autonomy at the cost of reliability.
Watch whether any of the major agentic search products (Perplexity, SearchGPT, or similar) ship explicit disambiguation prompts as a configurable behavior within the next two quarters. If they do, DiscoBench will have had measurable influence on production design; if not, it stays a benchmark paper.
This interpretation is generated from the summary above and available source metadata. Our methodology · Report an error
MentionsDiscoBench
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. The Decoder originally reported this story as “AI search agents don't fail at searching, they fail at asking the right questions when queries get ambiguous”. The full content lives on the-decoder.com. If you’re a publisher and want a different summarization policy for your work, see our takedown page.