A new audit tool can catch an API quietly swapping the AI model it promised you for a cheaper one.
Researchers built KBF, a black-box auditing method that works by asking a model questions near the edge of what it knows, including ones it reliably gets wrong the same way every time. Those repeatable wrong answers act like a fingerprint. Across 16 production endpoints, the team ran 155 separate substitution attempts total, testing different swapped-in models and routing setups at each endpoint, and KBF flagged every single one. None of the 16 genuine same-model control checks triggered a false alarm. In mixed-routing simulations, it still hit a 95% detection rate even when only 15% of responses actually came from a swapped model.
The researchers then pointed KBF at the real world: auditing 28 live endpoints across six platforms. Seven came back statistically inconsistent with the model they claimed to be running.
This matters because the market for LLM access is full of middlemen: relays, resellers, and proxy APIs. Buyers have no reliable way to confirm they are getting the model they paid for, rather than a cheaper substitute routed in behind the scenes. KBF offers a receipt instead of blind trust: a repeatable check costing about $0.67 per audit, even when verifying against an expensive reference model like GPT-6 Astra.
Seven out of 28 flagged endpoints is the real headline. That is not a hypothetical risk - it is roughly a quarter of tested real-world providers potentially not delivering what they advertise. Whether anyone acts on it is another matter. Right now this is a research paper, not a certification regime any reseller has agreed to submit to.