NVIDIA Audex unifies audio and text without sacrificing language ability
NVIDIA's Nemotron Labs has released Audex, a 30B-parameter multimodal LLM that unifies audio and text processing without sacrificing language performance. The model treats audio and text tokens uniformly within a single Transformer decoder, projecting audio into the text embedding space for seamless cross-modal generation. Built on a strong MoE foundation and trained on 157.4B audio tokens plus 320.5B text tokens, Audex demonstrates that audio capabilities can be added to text-dominant models through careful architecture and dataset curation. This approach matters because it sidesteps the typical capability tradeoff in multimodal scaling, potentially influencing how labs design future unified models.
Modelwire context
ExplainerThe buried detail is the training data composition: 157.4B audio tokens alongside 320.5B text tokens is a roughly 1:2 ratio, which is unusually audio-heavy for a model that still claims text parity. Most multimodal models that preserve text performance do so by keeping audio a minority signal, so the MoE architecture here is doing real work to prevent cross-modal interference, not just benefiting from a conservative data mix.
This sits in a cluster of work this site has tracked around making audio a first-class modality rather than a bolt-on. The Hugging Face and Cerebras piece from July 1 showed open-weight models reaching production-grade voice inference, but that integration kept audio and text as separate pipeline stages. Audex's single-decoder approach is architecturally more aggressive. Separately, the TiCodec paper from July 6 on streaming neural speech codecs addresses the inference-cost side of the same problem: once you have a unified audio-text model, efficient codec design determines whether it runs at acceptable latency. These two papers are effectively solving adjacent halves of the same deployment challenge.
Watch whether independent evaluators can reproduce the text benchmark parity on standard held-out suites like MMLU or HellaSwag without access to NVIDIA's training data, since the MoE routing strategy is the key variable and its behavior on purely text inputs is not yet externally verified.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsNVIDIA · Nemotron Labs · Audex · Nemotron-Cascade-2-30B-A3B
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Unified Audio Intelligence Without Regressing on Text Intelligence”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.