[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-new-method-turns-in-context-learning-into-a-single-weight-update":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},9701,"new-method-turns-in-context-learning-into-a-single-weight-update","New Method Turns In-Context Learning Into a Single Weight Update","Researchers compressed in-context learning into a single analytic weight update, closing most of the gap with full few-shot prompting.","A new technique lets language models capture the benefit of in-context examples without re-reading them on every single query.\n\nResearchers analyzed how in-context learning actually works inside a transformer's forward pass. They found that each attention head's output, when given demonstrations, is just an affine transformation of what that same head would produce without them. The transformation's parameters stay consistent across different examples of the same task. Based on that, they built a method called Task Operator, which calculates this transformation once and applies it directly as an update to the model's attention output projection, instead of feeding the examples through every time. Tested on lexical, algorithmic, and reasoning tasks, it beat prior compression techniques and closed most of the performance gap between zero-shot prompting and full in-context learning.\n\nThat matters because in-context learning is expensive to run at scale. Every inference call has to chew through the full set of examples, which eats compute and latency budget in production systems. Task Operator also suggests models store task knowledge in a sparse, identifiable circuit rather than smeared across the whole network, and the researchers showed operators from separate example batches can be averaged together to add more effective examples without expanding the context window at all.\n\nEarlier attempts to compress demonstrations into fixed vectors broke down on anything more complex than simple pattern matching. Whether this one holds up outside of benchmark tasks and arXiv preprints is the next question worth asking.","[\"in-context-learning\",\"llms\",\"mechanistic-interpretability\",\"arxiv\"]","2026-10-02T04:00:00.000Z","2026-10-03T06:57:15.979Z","2026-10-03T06:57:20.148Z","published",null,[],"ai",[26,27,28,29],"in-context-learning","llms","mechanistic-interpretability","arxiv",[31],{"name":32,"url":33},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.01054",0,{"sections":36},[37,40,44,48,53,57,61,66,71,76,81,86,91,96],{"name":38,"slug":24,"count":39,"latest_published_at":18},"AI",5976,{"name":41,"slug":42,"count":43,"latest_published_at":18},"Security","security",842,{"name":45,"slug":46,"count":47,"latest_published_at":18},"Policy","policy",438,{"name":49,"slug":50,"count":51,"latest_published_at":52},"Deals","deals",317,"2026-10-01T22:00:00.000Z",{"name":54,"slug":55,"count":56,"latest_published_at":18},"Hardware","hardware",199,{"name":58,"slug":59,"count":60,"latest_published_at":18},"Science","science",173,{"name":62,"slug":63,"count":64,"latest_published_at":65},"Consumer Tech","consumer-tech",155,"2026-10-01T19:54:10.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Dev Tools","dev-tools",96,"2026-10-01T16:57:03.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Software","software",93,"2026-09-30T21:41:11.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Startups","startups",90,"2026-10-01T21:55:22.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"Gaming","gaming",53,"2026-10-02T02:50:39.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"General","general",50,"2026-09-30T21:37:54.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"Reviews","reviews",31,"2026-09-28T14:31:34.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"How-To","how-to",7,"2026-10-01T09:00:00.000Z"]