TRADE: Transducer-Augmented Decoder for Speech LLM
Streaming speech LLMs have struggled with real-time inference because text generation doesn't naturally align with acoustic frames. TRADE solves this by grafting a transducer branch onto multimodal LLMs, letting the model emit tokens synchronized to audio input while preserving the underlying language model's reasoning. The approach uses a dual-vocabulary scheme and chunk-synchronized training to enable low-latency decoding and robust end-of-utterance detection. This bridges a fundamental gap between how LLMs process language and how speech systems must operate, making conversational AI systems both faster and more reliable for production deployment.62
