Researchers have found a way to reverse-engineer the internal architecture of commercial large language models — even through the stripped-down API access providers thought was safe enough.
A paper published this week introduces NightVision, a black-box inference attack that works against LLMs using only the single output token probability each API call returns. Earlier research had already shown that top-k logits or logit bias features could leak architectural details, so most commercial providers quietly removed those capabilities. NightVision sidesteps that fix with a technique called common set prompting: feeding multiple prompts that share the same output tokens, then running spectral analysis on the resulting log probabilities to back out the model's hidden dimension. From there, it uses time-to-first-token measurements alongside the estimated hidden dimension to approximate layer depth and total parameter count. Tested across 32 open-source models, NightVision recovered hidden dimension within 23% average relative error — dropping to 9% on mixture-of-experts architectures — and estimated depth and parameter count within 53% for models above three billion parameters.
For providers who treat model architecture as a trade secret, this is a meaningful crack in the wall. Knowing a competitor's parameter count and depth narrows the search space for replication considerably, and it may inform decisions about fine-tuning, distillation, or pricing attacks. The research also suggests that incremental API restrictions are a game of whack-a-mole: close one information channel and someone finds another.
Providers have already tightened APIs once in response to earlier work. Expect another round of restrictions — and another paper showing those restrictions are not enough.