[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-smarter-sql-cuts-ai-inference-costs-with-streaming-cascades":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},4210,"smarter-sql-cuts-ai-inference-costs-with-streaming-cascades","Smarter SQL Cuts AI Inference Costs With Streaming Cascades","New algorithms route most database rows to cheap models and reserve expensive AI calls for uncertain cases, hitting F1 above 0.95 on six benchmarks.","A research paper proposes two algorithms that make AI-powered SQL queries significantly cheaper to run.\n\nModern data warehouses increasingly embed large language models directly into SQL queries, running inference on every matching row — an approach that gets expensive fast. Model cascades address this by sending most rows through a cheap \"proxy\" model and escalating only ambiguous cases to a pricier \"oracle\" model. The problem: existing cascade methods (specifically the SUPG family) require a full pass over proxy scores before any results flow out, which blocks pipelined query engines and forces a tradeoff between precision and recall rather than optimizing both. The new paper formalizes this as a streaming problem and offers two solutions. SUPG-IT adapts the existing SUPG approach to streaming by iteratively tightening two routing thresholds as oracle labels arrive in batches — the first streaming cascade with joint probabilistic guarantees on both precision and recall simultaneously. GAMCAL takes a different angle: instead of asking users to specify targets, it learns a calibrated model that maps proxy scores to true-positive probabilities and routes stochastically based on a single cost-accuracy tradeoff parameter.\n\nThe efficiency gains matter because semantic SQL is no longer a research curiosity — production data warehouses are already running it, meaning inference costs are real and recurring. GAMCAL reached F1 above 0.95 while making 58% fewer oracle calls than the LOTUS SUPG baseline on the same benchmarks, which translates directly to API spend. SUPG-IT posted a mean F1 of 0.989 across six datasets, the highest best-case accuracy of the two.\n\nNone of this solves the deeper issue that stuffing LLM inference into SQL is architecturally odd — but given that it is happening regardless, smarter routing is a reasonable harm-reduction strategy.","[\"ai\",\"databases\",\"sql\",\"machine-learning\"]","2026-07-07T04:00:00.000Z","2026-07-07T20:02:45.219Z","2026-07-07T20:02:48.110Z","published",null,[],"ai",[24,26,27,28],"databases","sql","machine-learning",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.00660",0,{"sections":35},[36,40,45,50,55,60,65,70,75,79,84,88,93,98],{"name":37,"slug":24,"count":38,"latest_published_at":39},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":41,"slug":42,"count":43,"latest_published_at":44},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":46,"slug":47,"count":48,"latest_published_at":49},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":18},"Dev Tools","dev-tools",59,{"name":80,"slug":81,"count":82,"latest_published_at":83},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":85,"slug":86,"count":82,"latest_published_at":87},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":89,"slug":90,"count":91,"latest_published_at":92},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":94,"slug":95,"count":96,"latest_published_at":97},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":99,"slug":100,"count":101,"latest_published_at":102},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]