Auditing an AI: judgment without method is not enough

Most organizations that today claim to audit their artificial intelligence simply check a questionnaire against the standard, tick boxes, and file the result. That is not an audit: it is a documentary inspection. The difference is not one of form. An inspection confirms that the document exists; an audit confirms that the system does what the document asserts, and leaves evidence that it does.

The misunderstanding arises from treating ISO/IEC 42001 as if it were, at once, the what and the how. It is not. The standard describes the criterion; the audit method lives elsewhere.

ISO/IEC 42001:2023 sets out the requirements of an artificial intelligence management system: what must be governed, which risks must be treated, which controls must exist. It is criterion. ISO 19011:2018 — Guidelines for auditing management systems — sets out how an audit is planned, conducted, documented, and reported. It is method. And ISO/IEC 17021-1:2015 establishes who may certify a management system and under what rules of impartiality and competence.

An auditor who knows 42001 but not 19011 knows what to look at and does not know how to look at it. They will produce an opinion without traceability: defensible in a conversation, indefensible against a challenge.

The breaking point is almost always the same: the control exists in the policy and leaves no trace in operation. The organization declares that it supervises its models, that it evaluates bias, that it retains decision records. The auditor requests the evidence for the last twelve weeks, and the void appears.

Evidence traceability is what separates an assertion from a verifiable fact. Without it, an AI management system is a statement of intent with a standard for a cover.

ISO/IEC 42001 is young, and that carries a practical consequence worth stating without euphemism: much of what is offered today as 42001 certification falls outside the published accreditation scope. It is not necessarily fraud; it is a standard whose recognition is still being built.

Hence the distinction the buyer must demand. One thing is a certification within a body's published accreditation scope; another is a complementary institutional validation, which confirms standard, market, and applicable scope but does not amount to accreditation. IAC registers and verifies that evidence; it does not accredit or certify. Confusing the two planes is the most common way to buy trust that does not exist.

Auditing an AI with rigor is no harder than auditing any other management system. It demands what it always has: clear criterion, disciplined method, and evidence that survives a second look. The novelty of the technology does not change the rule; it only makes it more urgent.