OpenAI trained GPT-4 to write natural-language descriptions of individual neuron behavior inside language models, then scored how well those descriptions fit observed activity.
The project automatically generates and evaluates explanations for what each neuron in a neural network is responding to. The resulting dataset covers every neuron in GPT-2, the company's older and smaller model first released in 2019. OpenAI describes the explanations as imperfect, which is unusually candid for a field that tends to present incremental progress as a breakthrough. The dataset is being made publicly available.
Interpretability, the effort to understand what is actually happening inside neural networks rather than just what comes out, is one of AI safety's hardest open problems. Automated tooling that can explain neuron behavior at scale would give researchers a way to audit models for unexpected or dangerous internal patterns without requiring a human to inspect each neuron by hand. The recursion problem is worth naming, though: GPT-4 is not itself well understood, which means the approach relies on one opaque system to explain another.
GPT-2 is a comparatively simple model by current standards, with far fewer parameters than the systems now in deployment. Whether the method holds up on today's far larger models is the question this research raises but does not answer.