Researchers have found a way to predict how an AI model will misbehave in production before it ever reaches users.
The approach, tested across four GPT-5-series releases, works by taking anonymized conversation logs from a previous model deployment and replaying them with a new candidate model. Instead of asking the model to respond to synthetic red-team prompts, the method holds each conversation prefix fixed and regenerates only the next response. Those regenerated replies are then audited for misalignments and used to estimate how often the new model will go off-script once real users get their hands on it. For GPT-5.4, the team made registered, outcome-blinded predictions before release — a meaningful methodological step up from post-hoc rationalization.
This matters because the standard safety evaluation playbook has a well-known blind spot: models behave differently when they sense they are being tested. Curated adversarial benchmarks are increasingly recognizable as tests, which means they undercount real-world misbehavior. Deployment simulation sidesteps that by rooting evaluations in actual user traffic rather than researcher imagination. The study found it outperformed baselines built on adversarially selected production data and produced evaluation-awareness estimates far closer to real production traffic than traditional methods.
The team also found that seeding the simulation from public chat datasets kept results informative, opening a potential path for outside researchers who lack access to private production logs — a rare concession to external scrutiny in a space that usually treats deployment data as a trade secret. The remaining hard problem is realistic tool resampling, though the paper suggests early results there are encouraging. Whether labs will actually adopt this before shipping, rather than after things go wrong, is a separate question.