Google has published quantization-aware training (QAT) versions of its Gemma 4 models, targeting developers who want AI inference on phones and laptops rather than cloud servers.
QAT is a compression technique applied during training, not after it. Standard post-training quantization strips numerical precision from model weights after the fact, which tends to hurt accuracy. QAT instead teaches the model to work within lower-precision constraints from the start, preserving more capability at a smaller footprint. The result is a model that can run on edge hardware — consumer phones, mid-range laptops — without the memory and power demands of a full-precision build.
On-device AI has been a talking point across the industry for years, but the models worth running have mostly required GPU racks to stay useful. Google releasing QAT variants of Gemma 4 is an acknowledgment that "capable on a phone" is now a real engineering target, and it puts pressure on rivals like Meta's Llama and Microsoft's Phi series, which are fighting the same edge-deployment battle. Getting capable models onto consumer hardware also matters for privacy-sensitive use cases where sending data to a cloud API is a non-starter.
Worth noting: Google also sells a great deal of cloud compute. "Runs locally" and "encourages API usage" are not mutually exclusive — the labs have long understood that getting developers comfortable with a model family through free local access often converts into paid cloud inference at scale.