A new research paper says the AI detectors conferences rely on are measuring the wrong thing.
The paper, posted to arXiv, argues that AI-use detection tools estimate whether a model wrote a given passage, not whether an author followed a venue's actual AI policy. Those policies vary by role, task, and disclosure requirement, but current detectors ignore all of that context. The authors propose what they call policy-conditioned AI-use detection: a framework that treats the venue's rule as an explicit input and outputs hypotheses, evidence, and a calibrated error rate instead of a flat "AI detected" verdict. They test the idea against peer review, building benchmarks from pipelines that generate both compliant and non-compliant writing workflows.
Applied to realistic violation rates, the paper finds that even a detector running at a strong operating point still flags more compliant authors than actual rule-breakers. That is a different problem than a detector simply missing cases, it is a detector whose false positives routinely outnumber its true positives. The framework's fix is procedural: structured disclosure, tool routing that does not break reviewer anonymity, and a formal way to contest a flag.
It is the same lesson plagiarism-detection software learned the hard way, catching the technology was always easier than defining the offense.