A new paper wants AI agents to prove, cryptographically, that they actually checked the rules before taking an action.
Researchers built NiyamAI, a runtime guardrail for autonomous LLM agents that call external tools. At the start of a session, the allowed tools and constraints get locked into an Intent Contract, sealed under a SHA-256 commitment. Every tool call passes through an authority gate and a small neural network judge (an 11-8-2 layer network), and for each approved action the system generates a zk-SNARK proof that the policy check ran correctly against that contract - the action only executes once the proof verifies. Tested on 2,000 scenarios from AgentSafetyBench, NiyamAI scored 88.8% F1 at a 1.0% false-positive rate, well ahead of Llama Prompt Guard 2 (66.8%), GPT-OSS-Safeguard (46.2%), and NeMo Guardrails (40.4%).
Most guardrails today - prompt filters, semantic classifiers, policy engines - run on the same machine as the agent they're supposed to police, so a compromised host can quietly wave through a dangerous action and leave no trace of the failure. NiyamAI's proofs are portable: an 18.6 KB proof verifies in 51 milliseconds, and any third party can check it without touching the underlying model, turning "the system checked the rules" from a claim into something auditable.
The catch is overhead - 1.7 seconds of proof generation per approved action - and the team's own adversarial testing turned up two exploitable holes in the enforcement layer, a useful reminder that verifiable isn't the same as unbreakable.