[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-ai-anti-forgetting-fix-should-target-layers-not-parameters":10,"sections":41},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":36,"feedback":40,"feedback_at":22,"cost_usd":40,"total_tokens":40},5408,"ai-anti-forgetting-fix-should-target-layers-not-parameters","AI Anti-Forgetting Fix Should Target Layers, Not Parameters","New research shows continual learning regularizers should protect early neural network layers more heavily, not spread protection evenly across parameters.","A new theoretical paper argues that a popular fix for AI models forgetting old tasks has been protecting the wrong layers all along.\n\nContinual learning regularizers like EWC try to stop a model from forgetting earlier tasks by penalizing changes to parameters that mattered before, scoring each parameter with something called diagonal Fisher information. The researchers show that under a block-diagonal assumption about the network's Hessian, forgetting actually splits into per-layer terms driven by each layer's top eigenvalue, a quantity the diagonal Fisher score cannot capture. Two layers can share the same average Fisher value and still have top eigenvalues that differ by a factor as large as the layer's width. Spreading regularization evenly across parameters, rather than by layer, costs new-task performance in proportion to how ill-conditioned that layer is.\n\nThe proposed fix is almost embarrassingly simple: protect early layers heavily and let deeper layers adapt more freely, instead of treating every parameter as equally important. Applied to EWC and the newer SLCA method, this layer-adaptive approach produced measurable gains in both average performance and forgetting metrics.\n\nIt is a useful correction to an assumption baked into a decade of continual learning work: that scoring importance per-parameter is automatically more precise than scoring it per-layer. Sometimes the coarser unit is where the real signal lives.","[\"continual-learning\",\"ewc\",\"neural-networks\",\"machine-learning\"]","2026-08-18T04:00:00.000Z","2026-08-18T19:01:31.278Z","2026-08-18T19:01:43.177Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"Remove or attribute the claim that 'this result...could explain why some EWC variants lag behind newer replay-based approaches in benchmarks' — that comparison isn't in the source and reads as an invented\u002Funsupported implication.","resolved","ai",[32,33,34,35],"continual-learning","ewc","neural-networks","machine-learning",[37],{"name":38,"url":39},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.15901",0,{"sections":42},[43,47,51,56,61,66,71,76,81,85,90,95,100,105],{"name":44,"slug":30,"count":45,"latest_published_at":46},"AI",3293,"2026-08-20T04:00:00.000Z",{"name":48,"slug":49,"count":50,"latest_published_at":46},"Security","security",435,{"name":52,"slug":53,"count":54,"latest_published_at":55},"Policy","policy",210,"2026-08-19T09:32:27.000Z",{"name":57,"slug":58,"count":59,"latest_published_at":60},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":62,"slug":63,"count":64,"latest_published_at":65},"Hardware","hardware",140,"2026-08-19T18:25:42.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Consumer Tech","consumer-tech",95,"2026-08-18T16:05:00.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Science","science",90,"2026-08-19T18:41:02.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Software","software",73,"2026-08-18T07:51:50.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":18},"Dev Tools","dev-tools",69,{"name":86,"slug":87,"count":88,"latest_published_at":89},"Startups","startups",47,"2026-08-19T19:13:46.000Z",{"name":91,"slug":92,"count":93,"latest_published_at":94},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":96,"slug":97,"count":98,"latest_published_at":99},"General","general",33,"2026-08-18T22:18:13.000Z",{"name":101,"slug":102,"count":103,"latest_published_at":104},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":106,"slug":107,"count":108,"latest_published_at":109},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]