Researchers at Microsoft have published a framework that treats LLM output as a draft, not a deliverable.
A paper from Microsoft describes a three-part system for generating reliable structured artifacts — database queries, threat-mapping schemas, and entity relationships — using large language models. The core idea is blunt: LLMs generate; humans (or their coded proxies) validate. When an artifact fails a test, the error message goes back to the model, which then revises its output. The system layers two kinds of tests — deterministic checks for syntax and schema, and LLM-based judges trained to replicate expert human decisions — and has been deployed in production inside Microsoft Sentinel, the company's cloud-based security platform.
Most enterprises trying to use LLMs for structured output find themselves trapped between two bad options: trust the model and accept errors, or keep a human in every loop and lose the efficiency gain. This framework offers a third path — codified expert judgment that scales without a dedicated reviewer on each ticket. The security domain is a meaningful proving ground, since a malformed KQL query or a bad MITRE ATT&CK mapping can send a threat investigation in the wrong direction.
The paper is careful not to claim the model gets it right; it claims the system catches it when the model gets it wrong — a meaningful, if less marketable, distinction.