[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-text-to-sql-models-break-easier-than-anyone-admitted":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},3885,"text-to-sql-models-break-easier-than-anyone-admitted","Text-to-SQL Models Break Easier Than Anyone Admitted","A new automated framework called SAGE finds that LLMs converting plain English to database queries fail in ways static testing never caught.","LLMs that turn plain-English questions into SQL queries are more fragile than their benchmark scores suggest.\n\nResearchers have published a framework called SAGE - Systematic Automated Guided Exploration - that hunts for failure patterns in text-to-SQL models without relying on hand-written rules. The system generates hypotheses about where a given model might stumble, then designs targeted perturbations to test those hypotheses, logging confirmed weaknesses into a growing \"Vulnerability Codex.\" Tests on several open-source LLMs turned up a substantial number of failure cases that existing diagnostic approaches had missed. The kicker: the failure patterns transferred across models, meaning the weaknesses are structural, not one-off quirks.\n\nThat cross-model transferability is the detail worth sitting with. It suggests the problem is not a single vendor shipping a sloppy model - it is a systemic property of how current LLMs handle the translation task. Any enterprise plugging an LLM in front of a database should treat that as a supply-chain concern, not a model-selection problem.\n\nThe paper also tests a lightweight fine-tuning pass using SAGE-generated samples and finds early improvements, though the authors frame this as preliminary. Text-to-SQL has been pitched as a way to give non-technical users direct database access; the gap between \"passes the benchmark\" and \"safe to run in production\" just got harder to hand-wave away.","[\"ai\",\"security\",\"llm\",\"databases\"]","2026-07-07T04:00:00.000Z","2026-07-07T10:52:45.597Z","2026-07-07T10:52:48.566Z","published",null,[],"ai",[24,26,27,28],"security","llm","databases",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.03833",0,{"sections":35},[36,40,44,49,54,59,64,69,74,78,83,87,92,97],{"name":37,"slug":24,"count":38,"latest_published_at":39},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":41,"slug":26,"count":42,"latest_published_at":43},"Security",294,"2026-07-15T19:59:48.000Z",{"name":45,"slug":46,"count":47,"latest_published_at":48},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":50,"slug":51,"count":52,"latest_published_at":53},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":55,"slug":56,"count":57,"latest_published_at":58},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":60,"slug":61,"count":62,"latest_published_at":63},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":65,"slug":66,"count":67,"latest_published_at":68},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":70,"slug":71,"count":72,"latest_published_at":73},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":75,"slug":76,"count":77,"latest_published_at":18},"Dev Tools","dev-tools",59,{"name":79,"slug":80,"count":81,"latest_published_at":82},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":84,"slug":85,"count":81,"latest_published_at":86},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":88,"slug":89,"count":90,"latest_published_at":91},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":93,"slug":94,"count":95,"latest_published_at":96},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":98,"slug":99,"count":100,"latest_published_at":101},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]