Reasoning in embedding space cuts LLM inference cost

The paper's core claim is that intermediate reasoning steps don't need to be converted to discrete tokens at all. This sidesteps the vocabulary projection layer entirely, which is computationally expensive but has been treated as non-negotiable in autoregressive decoding.

This connects directly to the efficiency bottleneck identified in 'Token-Efficient Data Reasoning Agents' from late August, which found that token consumption during reasoning inflates inference costs by orders of magnitude. Where that work tackled the data structuring problem upstream, Soft Latent Thinking attacks the problem downstream by eliminating the need to tokenize internal thoughts. Both papers share the same diagnosis: token-level operations are a cost center in reasoning workloads. The mechanistic insight from 'The First Token Is a Clue' (same period) also becomes relevant here, since if multi-token concepts can be recovered from model internals without explicit tokenization, it suggests the model already encodes reasoning in continuous space. Soft Latent Thinking operationalizes that insight for inference.

If DeepSeek or Meta release production inference implementations using latent reasoning within the next six months, that signals real adoption beyond the benchmark. Otherwise, watch whether the pass@k gains hold on reasoning tasks outside chain-of-thought (e.g., code generation, math) where tokenization overhead is lower and the efficiency win may not justify architectural changes.