[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-giving-linear-attention-a-better-memory":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},3440,"giving-linear-attention-a-better-memory","Giving Linear Attention a Better Memory","A new architecture called HOLA pairs a standard linear-attention model with a small exact key-value cache, cutting Wikitext perplexity by 16 percent.","Linear-attention models just got a memory upgrade borrowed from neuroscience.\n\nResearchers introduced HOLA (Hippocampal Linear Attention), a hybrid architecture that bolts a bounded exact key-value cache onto the standard linear-attention recurrent state. The problem it targets is well-known: linear-attention and state-space models compress everything into a fixed-size state, which means earlier facts get overwritten when enough new associations pile in. HOLA takes its cue from the brain's complementary learning systems — the idea that the hippocampus handles sharp, exact recall while the cortex handles slower, generalizable patterns. The cache writes selectively, retaining tokens where the prediction residual is large, and a decoupled normalization step ensures retrieval stays sharp rather than blurring into soft averages.\n\nThe benchmark numbers are the headline. At 340 million parameters trained on 15 billion tokens, HOLA drops Wikitext perplexity from 27.32 to 22.92 — a 16.1 percent improvement — and beats a full-attention Transformer++ at 26.88. It also holds up on needle-in-a-haystack recall out to 32,000 tokens, which is 16 times its training length, outperforming competing approaches like GDN.\n\nThe broader tension here is familiar: efficient models trade exactness for speed, and researchers keep finding ways to claw that exactness back. HOLA's trick — no learned eviction module, just a residual-magnitude heuristic — keeps the approach simple enough that the gains feel durable rather than brittle. Whether it scales past 340M parameters is the next question nobody has answered yet.","[\"ai\",\"machine-learning\",\"research\",\"language-models\"]","2026-07-03T04:00:00.000Z","2026-07-03T06:04:34.464Z","2026-07-03T06:04:37.420Z","published",null,[],"ai",[24,26,27,28],"machine-learning","research","language-models",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.02303",0,{"sections":35},[36,40,45,50,55,60,65,70,75,80,85,89,94,99],{"name":37,"slug":24,"count":38,"latest_published_at":39},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":41,"slug":42,"count":43,"latest_published_at":44},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":46,"slug":47,"count":48,"latest_published_at":49},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":79},"Dev Tools","dev-tools",59,"2026-07-07T04:00:00.000Z",{"name":81,"slug":82,"count":83,"latest_published_at":84},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":86,"slug":87,"count":83,"latest_published_at":88},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":90,"slug":91,"count":92,"latest_published_at":93},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":95,"slug":96,"count":97,"latest_published_at":98},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":100,"slug":101,"count":102,"latest_published_at":103},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]