Research Models & Releases·arXiv cs.LG·20h ago

Latent action models cut annotation costs for language-steered autonomous driving

A new training pipeline addresses a critical bottleneck in vision-language-action models for autonomous driving: the scarcity of natural-language annotations paired with visual data. LADA uses vector-quantized latent action models to extract high-level driving intents from unlabeled camera-trajectory pairs, then trains a vision-language translator on a smaller annotated subset to enable language-conditioned control. This approach dramatically reduces annotation overhead while maintaining steering capability, shifting the data efficiency frontier for embodied AI systems that require human-interpretable instruction following at scale.

Modelwire context

Explainer

The key insight is inverting the annotation bottleneck: instead of labeling trajectories with language first, LADA extracts action structure from unlabeled video, then uses language only to condition that learned structure. This sidesteps the expensive step of hiring annotators to describe driving behavior.

This connects directly to the failure-recovery work from earlier this month on small agent systems (FRESH). Both papers identify that embodied systems deployed at scale hit a data efficiency wall, but LADA tackles it upstream by reducing annotation demand rather than improving error recovery downstream. The latent action extraction mirrors the task-specific geometry insight from the Riemannian metrics paper: both recognize that raw feature spaces need structure imposed by the actual task before they become useful for downstream components.

If LADA's steering performance on real-world driving benchmarks (like nuScenes or Waymo) stays within 5% of fully-annotated baselines while using less than 30% of the annotation budget, the approach has genuine deployment value. If performance degrades beyond that threshold, the latent extraction step is losing critical nuance that language annotation captures.

Coverage we drew on

Learning from Failures: Heterogeneous Graph Memory for Small Language Model Tool-Using Agents · arXiv cs.LG

This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.

MentionsVision-language-action models · LADA · Latent Action Driving Annotations · Vector-quantized bottleneck

Read full story at arXiv cs.LG →(arxiv.org)

Modelwire Editorial

This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.

Our mission How we write

Modelwire summarizes, we don’t republish. arXiv cs.LG originally reported this story as “Less Language, More Latents: Annotation-Efficient VLAs for Driving”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.