VideoFDB: Evaluating Full-Duplex Vision-Speech Capabilities in Conversational Agents

The harder problem VideoFDB surfaces is not perception but generation: producing contextually appropriate nonverbal responses in real time, synchronized with speech, is a constraint that no existing benchmark has formally decomposed. The 11-category annotation rubric is doing significant load-bearing work here, and its validity as a proxy for human social presence remains untested against actual user studies.

This connects most directly to the LoMo paper covered the same day, which identified that current vision-language architectures process visual and textual signals asymmetrically at a structural level. VideoFDB essentially stress-tests that same asymmetry in a dynamic, temporal setting where the modality gap compounds across a live exchange rather than a static query. Both papers are pointing at the same underlying problem from different angles: multimodal models are not actually fusing modalities, they are tolerating them. The Qwen-VLA coverage is also relevant context, since embodied action generation via diffusion decoders is one plausible technical path toward the real-time nonverbal generation VideoFDB is trying to evaluate.

Watch whether any of the major full-duplex voice labs (Google, Hume, or similar) adopt VideoFDB as an external evaluation target within the next six months. Adoption by a third party would validate the rubric; continued silence would suggest the benchmark's annotation categories do not map cleanly onto how production systems are actually built.