A new benchmark shows that today's top AI models are surprisingly bad at predicting how a single substituent swap changes a molecule's properties.
Researchers built MolSC, a dataset of 181,000 examples pulled from manually annotated bioactivity records, each capturing how attaching a specific substituent to a molecular scaffold shifts properties like structural-alert risk, target-specific bioactivity, and physicochemical traits. They paired it with MolSC-Bench, a 1,541-example holdout test built so the scaffolds, substituents, and molecules never overlap with the training data. When they ran existing molecular LLMs and general-purpose models, including GPT-5.2 and Gemini-3-Flash, against the benchmark, all of them struggled to reliably predict how these small structural tweaks change a molecule's behavior. Fine-tuning a model on MolSC closed that gap and also boosted performance on other, unrelated molecular tasks.
Substituent swaps are exactly the kind of edit medicinal chemists make every day while optimizing a drug candidate, so a model that can't reason about them isn't ready to assist with real design work. It's also a data point against the assumption that scaling general-purpose LLMs automatically buys domain expertise: two current flagship models, GPT-5.2 and Gemini-3-Flash, still lost to a model trained on a narrow, well-curated dataset.
More parameters didn't buy these models chemistry intuition. Targeted data did.