SonicCaps dataset brings 15M captions to audio-language model training

The dataset's real novelty isn't scale alone (15M captions) but the structured prompting approach that generates 24 semantically distinct descriptions per clip. This signals a deliberate move away from single-caption datasets toward capturing the ambiguity and richness that audio naturally contains, similar to how vision-language work evolved beyond one-image-one-label.

This connects directly to the annotation budget allocation work from early September, which showed that practitioners can use small-model experiments to guide resource decisions at scale. SonicCaps demonstrates the inverse: using an LLM (Qwen3-Omni) to systematically generate high-signal training data upfront, rather than collecting it passively. Both papers address the same operational reality: annotation is the bottleneck, and structured approaches (whether budget allocation or synthetic generation) beat ad-hoc methods. The radiology summarization paper from the same day also relies on quality training signal, though it focuses on domain-specific accuracy rather than diversity.

If models fine-tuned on SonicCaps show measurable gains on audio retrieval benchmarks that don't appear when trained on single-caption audio datasets of equivalent scale, that confirms the diversity hypothesis. If gains plateau or vanish on out-of-distribution audio tasks, it suggests the synthetic captions overfit to Qwen3-Omni's generation patterns rather than capturing genuine acoustic semantics.