AI-powered open-source intelligence tools are outpacing the frameworks meant to keep them honest.
A new survey covering 74 studies maps how large language models and agentic AI systems are being applied to open-source intelligence gathering and cyber investigations. Researchers found that while AI handles collection and analysis reasonably well, the later stages — verification, reporting, and decision support — remain largely unaddressed. More troubling: although more than 20 studies flag hallucination as a serious reliability concern, only one OSINT-specific system has ever empirically measured it end-to-end, and that study ran under conditions that cannot be reproduced.
That gap matters because OSINT feeds real decisions — criminal investigations, threat assessments, national security analysis. An AI that confidently fabricates a connection between two entities is not just wrong; it is potentially dangerous in the hands of an analyst who trusts it. The survey's conclusion — that a human-AI co-pilot model, where analysts retain responsibility for verification, is the most defensible near-term approach — is sensible, but it also implicitly acknowledges that the field is not yet ready to operate autonomously.
The research also draws a meaningful line between agentic AI systems and simple LLM prompting, treating the former as a distinct analytical category capable of multi-step reasoning and tool use. That distinction is increasingly important as vendors market every chatbot wrapper as an "agentic" platform.
The ten-point research agenda the authors propose includes better benchmarking, adversarial robustness testing, and dark-web coverage — a reminder that the publicly available web is only part of what investigators actually need to search.