A new machine learning method can optimize a key classifier metric without ever seeing a labeled example of what it is trying to catch.
Researchers propose a way to train binary classifiers to maximize partial AUC (pAUC), a ROC-curve metric that tracks true-positive rates within a chosen false-positive-rate range, using only positive and unlabeled (PU) data. Normally pAUC maximization needs labeled positive and negative examples, but negative labels are often hard to collect in practice because of privacy limits or the expertise needed to annotate them. The team reworked the underlying math so pAUC, including its FPR-dependent thresholds, can be expressed using just the positive and overall data distributions, then derived an empirical estimator from that. They trained a classifier by maximizing a smoothed version of this estimator and tested it on ten real-world datasets.
This targets a real bottleneck in fields like cybersecurity, medical care, and ad fraud detection, where confirmed negative cases are expensive or legally fraught to label, while confirmed positive cases, like a known intrusion or a confirmed diagnosis, are comparatively easier to get. If PU-based pAUC training holds up beyond the paper's ten test datasets, it could let teams build FPR-constrained classifiers without the labeling pipeline that currently gates them.
The paper does not report how its pAUC scores compare against standard methods trained on fully labeled data, so the real test is whether skipping negative labels costs accuracy right where it counts - at the low false-positive end that security and medical teams actually care about.