Making AI models small enough to fit on a phone or a Raspberry Pi doesn't guarantee they'll run well once they get there.
Researchers surveyed dozens of published studies on compressing language and image models for edge hardware, then tested the techniques themselves on GPU, CPU, and Raspberry Pi devices. They compared quantization (lowering numeric precision) against pruning (removing parameters) across question answering and image segmentation tasks. For question answering, a quantized 0.8B-parameter Qwen3.5 model scored 93.85 F1 on the SQuAD benchmark, while pruning the same model by just 1% cost it 16 points of F1. For image segmentation, the results flipped: pruning cut model size by nearly 80% with almost no accuracy loss, while quantization left size and compute largely untouched.
The sharper finding is what compression can hide. Pruned models sometimes got bigger, not smaller, because trimming broke internal formatting the hardware relies on, pushing Raspberry Pi latency up to 3.4x. One compressed model looked fine on the surface - readable output, 71% accuracy on a yes-or-no benchmark - while it had actually collapsed to guessing the same answer 97% of the time, a failure that only shows up if you check balanced accuracy rather than raw accuracy.
The paper's takeaway is blunt: test compressed models on the actual target device and task, because a technique that works for one team's chatbot can quietly wreck another team's vision model.