Quantization beats pruning for language, loses to it for vision on edge devices
Researchers systematized the gap between compression benchmarks and real-world edge deployment, revealing that no single technique reliably transfers across tasks and hardware. Testing Qwen3.5 0.8B and other compact models on GPUs, CPUs, and Raspberry Pi, they found quantization dominates for language tasks (93.85 SQuAD F1 with Q5_K_M GGUF) while pruning falters, but the ranking inverts for vision workloads. This empirical analysis matters because practitioners often assume lab compression results predict field performance, when hardware-specific constraints and task structure actually determine which optimization strategy succeeds. The findings establish practical deployment guidelines for teams shipping models to resource-constrained environments.62
















