Security/ llm-security · owasp · ai-risk · incident-analysis

OWASP's LLM Risk List Holds Up Despite Weak Data Agreement

A new analysis checks OWASP's expert-ranked LLM security risks against 6,639 real incidents and finds the rankings barely agree, yet still hold up.

OWASP's ranked list of the biggest risks to LLM applications barely matches what actually happens in the wild - and that's not necessarily bad news.

Two members of the OWASP working group built a corpus of 7,714 snapshotted LLM-security incidents pulled from CVE, GHSA, OSV, and the AIAAIC database, then labeled 6,639 of them against OWASP's 20-entry risk taxonomy. Using a Bayesian model that corrects each category's count for how often a classifier over- or under-flags it, they built a separate incident-based ranking and compared it to the expert consensus behind the official Top 10. The two rankings barely agree: Cohen's kappa lands around 0.20, and the 90% interval crosses zero, meaning the true agreement could be close to nothing. The 2026 candidate list still leans on expert judgment, blending it with incident data at a fixed 75/25 split.

That mismatch sounds like a problem for the Top 10's credibility, but the stress test says otherwise. A pre-registered bake-off of four frontier classifiers found none could beat the incidence-based ranking's baseline balanced accuracy of 0.863, and that same ranking's order tracked held-out ground truth with a Spearman correlation of 0.918. Expert intuition about which LLM risks matter most is holding its own against a large real-world incident dataset, even where the two rankings disagree on specifics.

Worth remembering: this is a side analysis from two working-group members, not an official OWASP release, so don't expect the published Top 10 to get reshuffled because of it. But it's a useful reminder that risk lists built by committee aren't automatically out of step with reality - sometimes the incident data just hasn't caught up yet.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →