On-device time-series language model brings health AI inference to smartphones

The paper doesn't just compress an existing model; it retrains using soft prompts (learnable token prefixes) rather than full fine-tuning, which trades some accuracy for dramatically lower memory footprint during both training and inference. This architectural choice is what makes deployment feasible on phones, not just quantization.

This extends the on-device inference frontier established by Oído (speech recognition on a $5 microcontroller) and Timestral (specialized models that outperform scale) into temporal reasoning. Where Oído proved open-vocabulary tasks work at extreme hardware constraints, PocketTSLM tackles the harder problem of preserving sequential structure in sensor data without cloud offload. The retrieval-augmented clinical model from early October showed that heterogeneous medical data benefits from local reasoning; this work operationalizes that principle for wearables and health platforms where latency and privacy regulation make cloud dependency untenable.

If PocketTSLM's soft-prompt variant maintains within 2-3% accuracy of full OpenTSLM on standard time-series benchmarks (like ECG classification or activity recognition) while running inference under 100ms on a Snapdragon 8 Gen 3, the approach becomes a template for other temporal tasks on mobile. If accuracy drops more than 5%, soft-prompting may be too lossy for health-critical applications.