[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-smarter-layer-removal-speeds-up-vision-transformers":10,"sections":41},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":36,"feedback":40,"feedback_at":22,"cost_usd":40,"total_tokens":40},3876,"smarter-layer-removal-speeds-up-vision-transformers","Smarter Layer Removal Speeds Up Vision Transformers","A new depth pruning method called HetDPT trims entire layers from vision Transformers without the accuracy collapse that has stalled earlier approaches.","Researchers have built a faster way to strip layers from vision Transformers while keeping accuracy nearly intact.\n\nMost compression work on vision Transformers targets width pruning — shrinking layers rather than removing them outright. Depth pruning, which cuts entire layers, offers bigger speed gains but has historically wrecked accuracy, so engineers avoided it. A team has now traced that failure to a overlooked cause: existing depth pruning methods ignore how much individual layers differ from one another, creating dimension mismatches when layers are removed. Their method, HetDPT, accounts for that heterogeneity explicitly. On ImageNet-1K, CIFAR-100, COCO, and ADE20K benchmarks, HetDPT delivers a 1.58x speedup for the DeiT-B model and 1.39x for DeiT-S with almost no accuracy drop.\n\nThe harder number is what happens at the extreme end. Paired with width pruning, HetDPT+ pushes the acceleration ratio for one standard configuration from 4.24x to 5.19x — a meaningful jump — while staying near-lossless on accuracy. That combination matters because most real deployments care about inference cost, not benchmark elegance, and prior joint pruning methods hit a wall at high compression ratios.\n\nDepth pruning has been the unloved sibling of model compression for years precisely because accuracy recovery was so unreliable. If HetDPT's results hold outside controlled benchmarks, it could shift how practitioners think about slimming down vision models — though the gap between an arXiv result and a production deployment is where most promising methods quietly disappear.","[\"machine learning\",\"computer vision\",\"model compression\",\"transformers\"]","2026-07-07T04:00:00.000Z","2026-07-07T10:38:30.504Z","2026-07-07T10:38:33.407Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"The headline and dek read as vague and informal — 'A Smarter Way to Slim Down' is a working-placeholder tone, not a publication-ready headline — and the dek references 'accuracy collapse that has plagued earlier approaches' without attribution, while the body omits two benchmarks (CIFAR-100, ADE20K) cited in the source without explanation, leaving the impression the results were selectively reported.","resolved","ai",[32,33,34,35],"machine learning","computer vision","model compression","transformers",[37],{"name":38,"url":39},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.03784",0,{"sections":42},[43,47,52,57,62,67,72,77,82,86,91,95,100,105],{"name":44,"slug":30,"count":45,"latest_published_at":46},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":48,"slug":49,"count":50,"latest_published_at":51},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":53,"slug":54,"count":55,"latest_published_at":56},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":58,"slug":59,"count":60,"latest_published_at":61},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":63,"slug":64,"count":65,"latest_published_at":66},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":68,"slug":69,"count":70,"latest_published_at":71},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":73,"slug":74,"count":75,"latest_published_at":76},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":78,"slug":79,"count":80,"latest_published_at":81},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":83,"slug":84,"count":85,"latest_published_at":18},"Dev Tools","dev-tools",59,{"name":87,"slug":88,"count":89,"latest_published_at":90},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":92,"slug":93,"count":89,"latest_published_at":94},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":96,"slug":97,"count":98,"latest_published_at":99},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":101,"slug":102,"count":103,"latest_published_at":104},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":106,"slug":107,"count":108,"latest_published_at":109},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]