[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-system-1-checkers-aim-to-curb-ai-pentest-false-positives":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},7812,"system-1-checkers-aim-to-curb-ai-pentest-false-positives","System 1 Checkers Aim to Curb AI Pentest False Positives","A new paper argues that letting LLMs grade their own hacking attempts breeds inflated bug reports, and proposes small calibrated classifiers as a fix.","Autonomous hacking agents have a self-grading problem, and a new paper wants to fix it with a second, dumber model watching the smart one.\n\nAutonomous penetration-testing harnesses use large language models to scan for weaknesses, exploit them, and write up the results. The trouble is these systems often use the same LLM to double-check its own findings, decide how severe a bug is, and choose which sub-agent handles it next. Researchers describe this as a setup that produces false positives and inflated severity ratings, along with wasted compute. Their proposed fix is a \"System One\" layer: lightweight, non-generative classifiers that hand back typed, calibrated verdicts instead of another round of LLM guesswork. They tested one such model, called Jev, inside a harness named NeuroSploit against a web target seeded with 13 known vulnerabilities, comparing a run with Jev to one without it.\n\nThe interesting move here isn't the case study itself, which the authors are careful to call exploratory and not statistically significant. It's the framing borrowed from behavioral psychology: fast, cheap, rule-bound judgment (System One) checking the work of a slower, generative reasoner. That's a sensible division of labor for security tooling specifically, where a single overconfident false positive can send a human analyst chasing a bug that isn't there, or a missed one can slip past entirely.\n\nThis is essentially the same pattern showing up in AI-assisted code review and content moderation, where cheap classifiers triage before an expensive model gets involved, applied to a domain where getting it wrong has real consequences for what a security team trusts.","[\"ai-security\",\"penetration-testing\",\"llm-agents\",\"autonomous-systems\"]","2026-09-25T04:00:00.000Z","2026-09-26T00:28:19.520Z","2026-09-26T00:28:31.190Z","published",null,[],"security",[26,27,28,29],"ai-security","penetration-testing","llm-agents","autonomous-systems",[31],{"name":32,"url":33},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.28940",0,{"sections":36},[37,42,46,51,56,61,66,71,76,81,86,91,96,101],{"name":38,"slug":39,"count":40,"latest_published_at":41},"AI","ai",4536,"2026-09-25T17:16:30.000Z",{"name":43,"slug":24,"count":44,"latest_published_at":45},"Security",741,"2026-09-25T15:52:13.000Z",{"name":47,"slug":48,"count":49,"latest_published_at":50},"Policy","policy",390,"2026-09-25T16:24:59.000Z",{"name":52,"slug":53,"count":54,"latest_published_at":55},"Deals","deals",256,"2026-09-25T17:00:53.000Z",{"name":57,"slug":58,"count":59,"latest_published_at":60},"Hardware","hardware",185,"2026-09-25T15:00:22.000Z",{"name":62,"slug":63,"count":64,"latest_published_at":65},"Science","science",140,"2026-09-25T11:55:23.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Consumer Tech","consumer-tech",132,"2026-09-25T15:30:00.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Software","software",88,"2026-09-24T23:06:55.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Dev Tools","dev-tools",82,"2026-09-25T09:59:40.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"Startups","startups",76,"2026-09-25T18:33:59.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"Gaming","gaming",48,"2026-09-25T18:35:21.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"General","general",46,"2026-09-25T02:12:57.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"Reviews","reviews",30,"2026-09-24T20:07:31.000Z",{"name":102,"slug":103,"count":104,"latest_published_at":105},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]