General Instinct, a YC-backed startup, open-sourced a compression toolkit that shrinks a 122-billion-parameter AI model to run on less than 8 GB of GPU memory.
The company released InstinctRazor, which takes Qwen3.5-122B-A10B, a 245 GB model in its original form, and compresses it to a 48 GiB file that fits on consumer-grade hardware. The approach preserves the always-active parts of a mixture-of-experts architecture (routing layers, normalization, vision pathways) while aggressively quantizing the routed expert blocks that only fire occasionally. A step called on-policy distillation then recovers capability lost in compression. The resulting model outperforms Gemma-4-26B-A4B on MMLU-Pro and GPQA-D benchmarks while coming in at a smaller file size.
The co-founders came from robotics, where the gap between frontier model quality and available edge hardware is a daily constraint. Squeezing 122 billion parameters into 8 GB of VRAM matters most for physical systems — laptops, industrial controllers, robot arms — that have no reliable path to a cloud inference endpoint.
Those benchmark comparisons come from the company itself, so treat them as a starting point rather than a verdict. The 5x compression ratio is the more durable claim: in a space where "edge AI" typically means a stripped-down 7B model, landing a 122B-parameter model in VRAM a gamer might own is a different category of result.