Kuaishou's AI team thinks there is a smarter middle path between brute-force memory and forgetting everything.
The company's researchers published a technical report introducing Kwai Summary Attention (KSA), an attention mechanism designed to make large language models cheaper to run on long inputs. Standard attention scales quadratically with sequence length — double the input, quadruple the compute. Existing fixes either compress the memory cache per layer (approaches like GQA and MLA) or swap in architectures that are cache-friendly but sacrifice long-range accuracy (like sliding-window attention). KSA takes a third route: it keeps a linear relationship between cache size and sequence length, but compresses at the semantic level using a tunable ratio, landing the model at O(n/k) memory cost instead of O(n).
That distinction matters because the trade-off in most long-context shortcuts is legibility — models start dropping distant details when the context window fills up. KSA's summary tokens are learnable, meaning the model is trained to decide what to keep rather than discarding by position or frequency. For applications like code agents, document reasoning, and recommendation systems — where a reference from 50,000 tokens ago can flip an answer — that retention is the whole point.
Kuaishou operates one of the world's largest short-video platforms, so its interest in recommendation systems at scale is not academic. The caveat: this is a technical report, not a peer-reviewed result, and summary-token approaches have appeared in prior research without becoming the dominant paradigm. Whether KSA's particular compression ratio and training recipe actually hold up across diverse workloads is a question the broader community will now get to stress-test.