Distillation bottlenecks revealed in sub-1B extraction models
Researchers systematically measured what's lost when compressing an 8B reasoning model into a 0.6B on-device student for structured text extraction. Using a three-judge evaluation panel, they compared distillation outcomes against few-shot prompting and constrained decoding baselines, isolating performance per subtask (summaries and categorical labels). The student achieved 0.8 second latency per article. This work matters because production extraction pipelines face hard tradeoffs between model scale and inference cost, and the paper quantifies exactly where smaller models fail, informing real deployment decisions in high-volume document processing.
Modelwire context
ExplainerThe paper's core contribution isn't that compression works (it does), but that it fails unevenly: categorical labeling holds up better than summary generation under compression, and the researchers isolate which subtasks degrade first. This granularity matters because it tells practitioners which extraction tasks can safely run on-device and which still need server-side models.
This work sits in the same production-hardening conversation as the SEC 8-K extraction system from earlier this week. Both papers treat structured text extraction as a grounded problem where hallucination and accuracy drift carry measurable cost. The current paper adds a layer: it quantifies what you lose when you compress the teacher model, whereas the SEC work focused on constraining outputs to a taxonomy. Together they suggest the field is moving past 'does the model work?' toward 'what's the minimum viable model for this specific task, and what do we sacrifice?'
If DeepSeek or Qwen release a production deployment guide in the next two quarters that cites this paper's per-subtask failure modes to justify task-specific model selection (e.g., 'use the 0.6B for categorical labeling, fall back to 8B for summaries'), that confirms the work is shaping real infrastructure decisions. If the paper is cited only in academic follow-ups, it remains a useful reference but hasn't yet influenced production pipelines.
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsDeepSeek · deepseek-r1:8b · Qwen3-0.6B · QLoRA
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Different Teachers, Different Capabilities: Sub-1B On-Device Distillation for Structured Text Enrichment”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.