Researchers have found a way to trim large language models without ever running them.
A new paper describes Weight-Redundancy Pruning, or WRP, a technique for removing entire Transformer blocks from an LLM based purely on its saved weights. Instead of feeding calibration data through the model to see which layers matter, as most existing pruning methods do, WRP compares the attention-output and MLP down-projection weights across layers directly. It builds an all-pairs similarity matrix combining those comparisons with relative projection-scale data, then uses that map to group similar layers and decide which blocks are redundant enough to cut. The authors tested it across multiple model families, pruning settings, and downstream tasks.
Skipping the forward pass matters because calibration runs are not free. They need representative data, GPU time, and careful tuning to avoid biasing which layers survive the cut. WRP's authors report it beats prior forward-free pruning methods that score each block in isolation, and gets close to the accuracy of slower, calibration-based approaches.
Close but not equal: the paper says WRP "approaches" activation-based performance, not matches it, so there is still a tradeoff between how fast you can prune a model and how much quality you keep.