The automated systems supposed to catch AI outputs that break the rules mostly aren't reading the rules at all.
A new audit tested activation probes and guard models - the compliance layers deployed to check chatbot outputs against written policies covering data protection, healthcare, finance, and platform rules. The researchers deleted, swapped, or permuted the governing rule in each test case and found detection accuracy barely moved for any detector, a pattern they call rule blindness. One guard model even correctly named the clause it was applying, then barely changed its verdict when that clause was swapped for its permissive opposite. A new benchmark crossing two rules with two scenarios, designed so neither factor alone predicts the right answer, confirmed the problem; only detectors that reasoned step by step escaped it.
The researchers also built their own lightweight detector, the Internal Compliance Score, calibrated from just ten labeled examples and cheap enough to audit four deployed guards, an 8B-parameter judge, and thirteen benchmarks at once. They held it to the same bar and it failed too: a pre-registered test for beating a trivial baseline came up short, and a plain bag-of-words model matched its results. Its one genuine payoff was raising the verified pass rate when it was used to rank candidate answers - useful, but narrow.
That gain vanished the moment researchers ran an adaptive attack designed to exploit the detector directly, a reminder that a compliance tool cheap enough to audit everything is not the same as one robust enough to trust.