Contrastive AI models throw away useful information every time they run — and a new theoretical paper explains why that information exists in the first place.
Embedding models trained with scale-invariant losses are typically evaluated using cosine similarity, a metric that measures the angle between vectors and ignores their length entirely. Prior empirical work had noted, somewhat puzzlingly, that those discarded lengths still seemed to correlate with meaningful properties: how specific a concept is, how often a token appears in training data, and how much uncertainty humans express about a term. Researchers now have a formal explanation. By working through the optimization dynamics of contrastive training, they derive an analytic formula showing that vector length encodes this semantic information as a natural side effect of how the model learns — not by design.
That matters because the signal is already there, in every model trained this way, for free. Rather than building separate calibration layers or uncertainty estimators, retrieval systems and downstream applications could read embedding norms directly — a cheap diagnostic that requires no additional training, no labeled data, and no architectural changes. The paper frames this as a grounded explanation for what had previously been a heuristic observation, which is a polite way of saying the field had noticed the pattern without understanding it.
Embedding models underpin search, recommendation, and retrieval-augmented generation at scale, so even marginal improvements in calibration have compounding effects — but it is worth noting that "free" signals discovered post hoc have a habit of behaving less reliably once practitioners start depending on them.