Agent Explorative Policy Optimization for Multimodal Agentic Reasoning
Researchers identify a fundamental training asymmetry in agentic AI systems: vision-language models trained with standard RL methods severely underutilize external tools, attempting them in only 30% of cases and failing catastrophically on 40% of tool-use trajectories. The paper proposes AXPO, a policy optimization variant that reweights exploration toward failed tool-use rollouts to recover the learning signal. This addresses a critical gap between how agents reason internally versus when they should delegate to external systems, directly affecting real-world deployment viability for multimodal reasoning agents.
Modelwire context
ExplainerThe 30% tool-attempt rate isn't just a performance gap, it's a signal that standard RL reward shaping actively discourages exploration of failure modes, meaning agents learn to avoid the very situations where external tools would help most. AXPO's contribution is specifically in reweighting the training distribution, not in architectural changes to the agent itself.
This connects directly to the abstraction gap work covered the same day ('The Abstraction Gap in Vision-Language Causal Reasoning'), which found that VLMs produce fluent outputs masking shallow reasoning. AXPO addresses a complementary failure: models that reason internally but don't know when to stop and delegate. Both papers point to the same underlying problem, that current VLM training objectives optimize for surface behavior rather than reliable decision-making under uncertainty. The LearnWeak coverage ('Learn from Weaknesses') adds further context, showing a parallel pattern where targeted failure identification outperforms indiscriminate training data scaling for agent specialization.
Watch whether AXPO's tool-use recovery holds on benchmarks with longer action horizons (more than five sequential tool calls), since the catastrophic failure rate on 40% of trajectories likely compounds with task length in ways the current evaluation may not fully capture.
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsAXPO · GRPO · Vision-language models
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.