[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-cut-gpu-wait-times-with-tens-of-lines-of-code":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},3752,"cut-gpu-wait-times-with-tens-of-lines-of-code","Cut GPU Wait Times With Tens of Lines of Code","A new paper shows that most web servers already have what they need to stop stalling on hardware offloads — no framework rewrite required.","Most servers handling AI inference or post-quantum crypto are wasting time waiting for hardware — and a new paper argues the fix is already sitting in the codebase.\n\nResearchers tested ten production-grade servers spanning every major concurrency model and found that overlapping hardware offloads with other in-flight requests requires adding 22 to 138 lines of code and modifying at most one existing line. The approach skips the usual remedies — async framework rewrites, new runtimes, custom dataplanes — and instead treats the problem as routing: submit the offload, suspend the request using the server's own deferred-response primitive, then resume on completion. On real hardware, that change recovered 1.2x to 5.4x throughput. A more aggressive variant using an LD_PRELOAD fiber runtime, which injects the reroute into an unmodified thread-per-connection binary, hit 17.3x — with caveats.\n\nThe insight matters because hardware accelerators — GPUs, FPGAs, hardware security modules, inference servers — are now on the critical path of online serving, and the standard options for handling the resulting stall are both bad. A context switch costs roughly as much as the offload itself; busy-waiting burns the CPU core. The paper's argument is that servers handling concurrent requests are already suspending and resuming them, so the concurrency machinery needed for overlap ships by default.\n\nThe safety asterisk is real: rerouting breaks run-to-completion atomicity, which can create hazards around unlocked shared aggregates. The authors address this with a page-protection detector, validated on stock Redis. Whether that taxonomy holds across messier production codebases is the question this paper cannot answer.","[\"performance\",\"servers\",\"hardware\",\"ai\"]","2026-07-07T04:00:00.000Z","2026-07-07T07:31:23.345Z","2026-07-07T07:31:26.296Z","published",null,[],"dev-tools",[26,27,28,29],"performance","servers","hardware","ai",[31],{"name":32,"url":33},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.02630",0,{"sections":36},[37,41,46,51,56,60,65,70,75,78,83,87,92,97],{"name":38,"slug":29,"count":39,"latest_published_at":40},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":42,"slug":43,"count":44,"latest_published_at":45},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":47,"slug":48,"count":49,"latest_published_at":50},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":52,"slug":53,"count":54,"latest_published_at":55},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":57,"slug":28,"count":58,"latest_published_at":59},"Hardware",122,"2026-07-14T19:46:26.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":76,"slug":24,"count":77,"latest_published_at":18},"Dev Tools",59,{"name":79,"slug":80,"count":81,"latest_published_at":82},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":84,"slug":85,"count":81,"latest_published_at":86},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":88,"slug":89,"count":90,"latest_published_at":91},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":93,"slug":94,"count":95,"latest_published_at":96},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":98,"slug":99,"count":100,"latest_published_at":101},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]