Picking the right LLM for a bank just got a methodology.
Researchers published a meta-benchmarking framework that organizes 452 publicly reported LLM benchmarks into 41 O*NET Generalized Work Activities, then rolls those up into 38 banking business domains drawn from the BIAN standard. A multiplicative weighting scheme, scoring each benchmark on discrimination, coverage, and recency, feeds a pairwise Elo tournament that produces work-activity scores without requiring raw score normalization. The framework was tested on a June 2026 snapshot covering 288 models from 25 organizations. The full methodology, taxonomy, and limitations are documented to let other institutions reproduce it.
Public leaderboards like MMLU-Pro rank models on global averages that say little about whether a model can handle compliance reasoning or multi-turn customer service. This framework argues that a model topping a coding benchmark might be the wrong pick for loan origination or risk management, and gives procurement teams a structured way to make that distinction.
The banking sector has been slow to formalize LLM governance; a reproducible, publicly documented scoring system could push that conversation forward, though its value ultimately depends on which benchmarks institutions choose to feed it.