Governance7 min read

"Human In The Loop" Is Not A Control

What governance actually has to say, on one page

I run a company inside a publicly traded parent, and I have sat through the security reviews, the compliance questionnaires and the audits that come with that. So I read a lot of AI governance documents.

Most of them contain the phrase "human in the loop, as appropriate." That phrase is doing no work at all.

Why it fails

A control has to be specific enough to test. "As appropriate" cannot be tested, which means it cannot be audited, which means it will not protect you when someone asks what happened.

Worse, it creates the appearance of a control while permitting the opposite. A reviewer approving two hundred outputs an hour satisfies "human in the loop" and reviews nothing. Everybody involved knows this and the document does not.

What to write instead

Name the decision. Name the reviewer by role. Name what they must see. Name where the review is recorded.

That is four specifics instead of one platitude, and it takes an afternoon.

The five lines

Governance fits on one page. If you cannot complete a line, that gap is your work item.

A human must review before: specific decisions, named.

This data may never enter a public tool: written in language your staff will actually follow, not legal abstraction.

Accountable when output is wrong: one person. Not a committee, not a function.

The audit trail contains: input, output, model version, reviewer, and what they changed. Retained to match your existing records policy rather than a new invented one.

We disclose to: customers, regulators, shareholders, each at a defined trigger.

The part that cannot wait

Four of those five can be added later. The audit trail cannot. It has to exist from the first deployment, because it is nearly impossible to reconstruct retroactively and it is the only thing that lets you answer the question that actually gets asked: what did your system do, and why?

The organizations that get into trouble with AI are not the ones that used it. They are the ones that could not explain it.

The reframe for regulated sectors

If you are in financial services, healthcare, legal or government, you have been told to be careful by everyone. Here is the other half: the discipline is not a tax on adoption. Being the organization that can prove what its systems did is a competitive advantage, and most of your competitors cannot.

Related

This is the framework from my keynote Trust At Scale, built for regulated organizations.