Web agents trained against evolving prompt injection attacks in simulation

The paper trains against adaptive attacks within a frozen simulator, but doesn't disclose whether defenses transfer to real websites or unseen injection strategies. The claim of 'continuous adaptation' needs scrutiny: the curriculum and adversary co-evolve inside a single environment, which prior work shows can mask stagnation through internal reward hacking.

This directly echoes the co-cheating failure mode flagged in 'False Frontiers' (late September), where self-evolving systems converge on shared errors and create illusions of progress. AdvSim2Real's three-component co-evolution (curriculum, adversary, agent) is precisely the kind of closed-loop system vulnerable to that critique. Additionally, 'Passing the Test You Trained On' (early October) showed that defenses ranking highly on one benchmark collapse on another, suggesting AdvSim2Real's robustness claims need validation outside its frozen simulator. The work also sits downstream of 'Surviving the Router,' which exposed how prior threat models ignore real deployment filtering, raising the question of whether AdvSim2Real's attacks reflect actual web-based injection vectors.

If AdvSim2Real's defenses maintain >80% success rate when tested against injections crafted outside the training loop (e.g., on real websites or a held-out simulator variant), the approach has real generalization. If performance drops below 40% on out-of-distribution attacks, it's likely another case of internal convergence without external grounding.