How do you audit an AI agent?

Why this differs from auditing ordinary software

Ordinary software audits check a fixed code path against a spec. An AI agent's behavior can vary run to run, its underlying model or prompt can change without a version bump a customer sees, and its own description of its reasoning can be narrated in natural language rather than logged as a verifiable trace. That gap between a plausible-sounding explanation and a verifiable one is exactly where an audit has to focus.

The five-point checklist

Each of these should be checkable by someone other than the vendor, on the running product.

• Identity and authority ceiling: does the agent have a distinct identity from the humans using it, and a documented ceiling (observe, propose, act) rather than an undefined "it can do a lot"?
• Source scope: is the agent's read access tenant-bound and enumerable, or can it reach data outside the boundary a customer expects?
• Action scope and approval: can you see exactly which actions require a human approval, and can you confirm the approver is never the same identity as the proposer?
• Receipts: does each finding or recommendation cite the specific record, field, or event it came from, so a claim can be traced back to evidence?
• Abstention: when the agent lacks the data to answer confidently, does it say so, or does it produce a plausible-sounding number anyway?

A red flag worth naming directly, and a sourced counterexample

Launch narration in this category often describes an operator console where every step the agent takes is visible. That is a legitimate design goal. But a narrated claim of visibility, with no published screenshot, no definition of what "every step" means and no way for a third party to check it, is not the same thing as an operable log. That difference is exactly what an audit checklist exists to catch, for any vendor. Ask to see the log itself, for an incident you choose, rather than a description of it.

By contrast, Datadog's Bits Investigation renders an agent trace listing every tool call the agent made, and Rootly ranks its hypotheses with a visible, confidence-scored reasoning chain. Both are examples of a claim being made checkable rather than only narrated, which is the standard worth holding every vendor to, including us.

What to demand in a proof of concept

Ask for read-only access to the agent's own audit log, not a curated export. Force a scenario where the agent should not know the answer, and check whether it abstains or fabricates. Try to get one identity to both propose and approve the same action, and confirm the system refuses it rather than merely discouraging it in a policy document.

HELM by LockedIn Labs exposes this on /command#investigations as a replayable trace of event chronology, ranked hypotheses, and counterevidence, and on /control-plane as an agent-observability section that states "not measured in the reference deployment" where the record does not hold a real metric, instead of asserting one.

Auditing over time, not just at a point in time

A single audit is a snapshot; an agent's behavior can shift as its underlying model, prompt, or configuration changes, sometimes without a version number a customer would notice. A durable audit practice re-runs the same five checks on a schedule, and specifically checks whether the agent's authority ceiling, source scope, or approval requirements have changed since the last review, not only whether the agent is currently behaving as expected.

It is also worth auditing what the agent chooses not to say. An agent that always sounds confident, even on questions where the underlying data is thin, is a bigger risk than one that occasionally admits uncertainty, because confident wrong answers are the ones that get acted on. Testing abstention deliberately, by asking a question the record cannot support, is one of the few checks that catches this before a real incident does.

Finally, audit the audit trail itself. A log that only the vendor can query, in a format only their support team can export, is not independently verifiable no matter how complete it looks on screen. Ask for a raw export you can inspect with your own tooling before accepting any agent's audit log as evidence of what it did.