What Goes Into an AI System Impact Assessment

AI system impact assessments go beyond policy. They evaluate how an AI system is used, who it affects, the risks it creates, the controls in place, and how those risks are monitored over time.

A one-page AI use policy is common. An actual impact assessment is much rarer, and it's the document ISO 42001 expects to see before a system goes anywhere near production.

Here's the gap I keep running into: a use policy tells employees what they're allowed and not allowed to do with AI tools. It doesn't ask what could actually go wrong once the system is live, who's on the receiving end of its decisions, or how anyone would even know if something failed. That's a completely different document, and most organizations don't have it.

A proper impact assessment covers the system's intended purpose and scope, what it's actually meant to do and just as importantly, what it isn't meant to do. It covers who's affected by its outputs, which is a wider group than just the employees using it: customers, applicants, and anyone downstream of a decision the system influences. It looks at potential harms across accuracy, bias, safety, and rights, assessed for that specific system rather than in the abstract. It traces the data feeding the system back to where it actually came from. And it spells out the human oversight built into the workflow where, specifically, a person can step in before a bad outcome reaches someone.

Timing matters more than people expect here. An impact assessment written after a system is already live and something's already gone sideways is a much weaker document; both practically and in front of an auditor, it starts to read like damage control instead of governance.

If your organization's AI governance right now consists of a policy document and not much else, this is the next thing to build. It's also, in my experience, the document most likely to get asked for directly in an ISO 42001 audit, not because it's a formality, but because it's the clearest evidence that someone actually thought through the risk before deployment rather than after.