From Text to Voice: A Reproducible and Verifiable Framework for Evaluating Tool Calling LLM Agents

The headline finding isn't that voice agents underperform text agents (that's expected) but that no single model leads across both benchmarks, which suggests the audio modality exposes task-specific weaknesses rather than a uniform capability deficit. The framework's value is reproducibility: by preserving original annotations through the TTS conversion pipeline, other teams can audit their own models without building new datasets from scratch.

This connects directly to the MemEye coverage from the same day, which flagged a parallel problem in multimodal evaluation: benchmarks that let systems route around the modality being tested, answering visual questions through text shortcuts. Both papers are essentially arguing that evaluation infrastructure has lagged behind deployment reality. The 'Talk is Not Cheap' taxonomy audit reinforces the same structural concern from a security angle, showing that benchmark coverage gaps create false confidence. Taken together, these three papers from a single day suggest the field is in an active reckoning with whether its evaluation stack actually measures what production systems need to do.

Watch whether the Confetti and When2Call benchmarks get adopted by model providers in their own technical reports over the next two release cycles. If Gemini or OpenAI cite these specific numbers in product announcements, that signals the framework achieved the standardization the authors are aiming for.