Vision-language-action models fail on phrasing shifts that language models handle

The critical insight is that VLA brittleness isn't random noise but follows learnable, predictable patterns. This means the problem is tractable at inference time without retraining, which is a practical advantage over the robustness gaps exposed in related VLM work.

This finding extends a pattern visible across recent VLM research: systems that appear capable in controlled settings fracture under minor input variations. The MIST benchmark from late September showed VLMs shift outputs based on irrelevant image context despite explicit instructions. Here, the brittleness is linguistic rather than visual, but the underlying fragility is the same. What distinguishes this work is the discovery that rephrasing alone can recover performance without policy modification, whereas the diversity-driven RL approach from today proposes learning multiple solution strategies to handle distribution shifts. Both target VLA robustness but from opposite angles: one intervenes at inference, the other at training.

If researchers apply the identified rephrasing rules to the LIBERO benchmark tasks used in the diversity-RL paper and show comparable gains to DRIVE's RL approach, that confirms inference-time intervention can match learned robustness. If the gains don't transfer across different task families, the rules may be overfitted to specific language patterns rather than capturing fundamental brittleness.