Samsung Labs has a new training method that makes heavily compressed language models nearly five times faster than full-precision — by fixing a failure mode nobody had cleanly addressed before.
RaBiT (Residual-Aware Binarization Training) compresses model weights down to just two values, +1 and -1, which eliminates the floating-point matrix multiplications that dominate transformer inference costs. The existing approach to this — stacking binary layers to approximate a full-precision model — was undermined by what the researchers call inter-path adaptation: parallel layers were learning redundant features instead of correcting each other's errors, gutting the entire point of the residual structure. RaBiT enforces a strict sequential hierarchy where each binary layer is derived from the same shared full-precision weight and trained explicitly to fix the residual error left by the previous layer. On an RTX 4090, the result is a 4.49x speedup over full-precision inference, with accuracy the paper says rivals hardware-intensive Vector Quantization methods.
Binarization has long been an interesting corner of the quantization space precisely because matmul-free inference can run efficiently on hardware that isn't purpose-built for AI — not just Nvidia's latest silicon. Closing the accuracy gap with Vector Quantization without a hardware dependency is the kind of result that shifts the math on deployment costs at scale.
Samsung Labs is the source here, and the company has its own chip ambitions, so the result arrives with a strategic subtext. The benchmark is a single GPU under the authors' own evaluation — independent reproduction across diverse model families would be the meaningful next test.