A new benchmark finds that AI models predicting molecule properties fall apart the moment they leave familiar chemical territory.
Researchers released BOOM, an open-source benchmark for testing how well machine learning models predict properties of molecules that fall outside their training data - the out-of-distribution problem that plagues drug and materials discovery. The team ran more than 150 combinations of models and prediction tasks. No model generalized well across the board: even the best performer saw its error rate triple on unfamiliar molecules compared to familiar ones. Chemical foundation models, the large pretrained systems many labs treat as general-purpose tools, did not extrapolate any better than smaller, more specialized models.
Molecular discovery is supposed to be one of AI's clearer wins - screening drug or material candidates faster than a lab ever could. That promise depends on models being right about molecules nobody has tested yet, not just molecules that resemble the training set. BOOM gives the field a shared yardstick for a weakness that, until now, was mostly anecdotal.
If a benchmark's headline finding is that nothing works consistently, that is not a knock on any one lab - it is a sign the whole approach needs a rethink before anyone hands these models a real discovery pipeline.