Prompt-injection benchmarks mask true agent vulnerabilities, audit finds
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
A systematic audit of 33,000 agent trajectories reveals that current prompt-injection benchmarks fundamentally misrepresent security posture. Most failed attacks are silently ignored rather than actively refused, and changing only the system prompt can swing measured vulnerability from 0.2% to 43% on identical model weights. The research exposes a critical gap: low attack success rates mask a deeper problem where agents either ignore user tasks entirely or remain vulnerable to subtle hijacking. This finding reshapes how the field should evaluate tool-using agent robustness and suggests that published security metrics may overstate real-world safety.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The critical finding isn't just that benchmarks undercount vulnerabilities, but that they measure the wrong failure mode entirely. Agents that silently drop malicious instructions look equally safe as those that actively reject them, yet pose fundamentally different deployment risks.
This connects directly to the pattern exposed in 'Shortcut Sensitivity in Agent-Safety Evaluations' (Oct 15), which showed that safety benchmarks collapse when evaluation splits shift to unseen domains. Here, the researchers demonstrate an even more direct manipulation: identical model weights swing from 0.2% to 43% vulnerability depending only on system prompt framing. Both papers expose how current metrics conflate surface-level performance with genuine robustness. The earlier work on 'Prompt-Injection Probes' (Oct 13) also surfaces this theme, showing that detection systems often measure statistical artifacts rather than actual attack understanding. Together, these three papers form a coherent critique: the field's evaluation infrastructure systematically overstates safety by measuring the wrong signals.
If InjecAgent or AgentDojo benchmarks are adopted by major model providers (OpenAI, Anthropic, Google) in their safety reports within the next six months, watch whether they explicitly report both attack success rates AND the distribution of failure modes (ignored vs. refused). If vendors publish only aggregate success numbers without this breakdown, the research's core warning remains unheeded.
This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error
Coverage behind this analysis
These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.
·arXiv cs.LG
Agent safety guards fail when tested on unseen domains
A new audit framework exposes how safety evaluations for AI agents can be gamed by spurious correlations in training data, rather than measuring genuine safety reasoning. Researchers tested this on LinuxArena and TS-Bench datasets, finding that guards trained to block harmful tool use often rely on shallow statistical patterns like command length instead of understanding…
MentionsInjecAgent · AgentDojo · Attack-Task Outcome Matrix
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.