An academic research framework called AnyPoC can take a raw LLM bug report and automatically generate a working proof-of-concept that triggers the suspected defect — no human in the loop required.
Most LLM-based bug finders stop at a list of candidates: plausible-sounding reports that still need a developer to verify them by hand. AnyPoC treats that gap as a test-generation problem. Given a candidate report, it builds and executes a proof-of-concept, collects execution traces, and then has a separate agent re-run and scrutinize the result — specifically to catch cases where the model convinces itself something worked when it didn't. The system was tested across 12 large codebases including Firefox, Chromium, LLVM, OpenSSL, SQLite, FFmpeg, and Redis, turning over two thousand noisy candidate reports into 121 confirmed new bugs, 92 of which have already been patched. Notably, 46 of the generated PoCs were accepted as official regression tests by the projects themselves.
That last number matters more than the headline bug count. Regression tests are a long-term artifact — they run on every future commit. When maintainers adopt an automated PoC as a regression test, they're not just closing a ticket; they're embedding external verification into the project's own safety net. The paper also reports that AnyPoC produces 37% more valid PoCs for true bugs than Claude Code or Codex, while rejecting 9.7 times more false positives — which suggests the independent re-execution step is doing real work, not just adding latency.
The recurring problem with LLM security tooling is the false-positive flood that makes triage worse than manual review. AnyPoC's architecture is explicitly designed around that failure mode, which puts it in a different category than most "AI finds bugs" announcements — though how it scales beyond the tested codebases is a question the paper leaves open.