A new technique can reduce toxic output from large language models without touching their weights or requiring any training at all.
The method, detailed in a new paper, works by approximating how toxic a model's output is likely to be and nudging the input embeddings in a less harmful direction before generation begins. It uses zeroth-order optimization — meaning it only needs access to input embeddings and a toxicity scoring function, not gradients or internal model states. The researchers ran a small number of "descent steps" at inference time to steer completions away from harmful content. Tests showed consistent toxicity reductions across multiple models and prompt types.
This matters because most existing detoxification methods are expensive, model-specific, or both. Retraining a model to be less toxic costs compute and doesn't always transfer to a different model family. A technique that works at test time on black-box models is far more portable — it could, in principle, be layered on top of any model an operator can query, including ones where the internals are never exposed.
The approach won't satisfy critics who argue that safety guardrails need to be baked into training, not bolted on afterward — and it does add latency at inference time, which is already a cost center for anyone running models at scale.