A research team has released PDFBench, the first unified benchmark for AI models that design proteins from scratch based on a target function.
Right now, labs building protein-design models pick their own metrics, test on their own data slices, and declare victory. PDFBench pushes back on that. The benchmark runs eight state-of-the-art models through 16 metrics across two settings: description-guided design, using a repurposed dataset called Mol-Instructions, and keyword-guided design, tested against a new holdout set called SwissTest. SwissTest was built with a strict date cutoff to prevent models from training on test data — a basic hygiene step the field has often skipped.
Protein design sits at the center of drug discovery and enzyme engineering, so sloppy benchmarking has real downstream costs. When every lab grades its own homework, it is genuinely hard to know which model to trust for a given task. A shared evaluation framework does not solve the underlying science, but it at least puts competitors on the same playing field.
The analogy here is ImageNet for computer vision: a common benchmark did not end the field's debates, but it did force everyone to argue about the same thing. PDFBench is an early attempt to do the same for protein design — though whether the community actually adopts it is a separate question from whether the authors built it well.