Human approval that scales

Executive summary

The first week an AI agent goes live, approvers read every card. By the third month, the same people clear a queue of two hundred items before lunch. The gate is still there on paper. In practice, it has become a click.

This is the quiet failure of human oversight. A gate that nobody reads gives you the cost of a manual process and the risk of an automated one. The fix is not more approvers. It is fewer, better approval requests: routed by risk, shown with the right evidence, checked for fatigue and loosened only when the log says it is safe. This part of our series sets out how to do that for operations and risk leaders who own the queue.

1. Why approval gates decay

Approval gates fail in a predictable way. Volume rises, most items are fine, and people learn that "approve" is almost always right. Soon they stop checking.

Hospitals learned this with drug safety alerts. A review in the Journal of the American Medical Informatics Association found that clinicians overrode these alerts in 49% to 96% of cases, largely because too many alerts were irrelevant. People tune out a signal that is usually noise.

The opposite failure is just as real. Researchers call it automation bias: people accept a system's output without checking it. Parasuraman and Manzey's review in Human Factors found that it affects experts as well as novices and cannot be prevented by training or instructions alone. You have to design against it.

Regulators now name the problem directly. The EU AI Act requires that people overseeing high-risk AI stay aware of automation bias and are able to override or stop the system (Article 14). The UK Information Commissioner's Office is blunter. A decision does not stop being automated just because a human has rubber-stamped it. Its guidance says reviewers need the authority and competence to go against the recommendation, and must not fear being penalised for doing so.

HMR insight: An approval nobody reads is worse than no approval, because it looks like control.

2. Route by risk, then design each lane

In Part 2 we set out four risk tiers for agent actions and one gateway that enforces them. The tiers also tell you how much human attention each action deserves. Spend it where the risk is, and spend almost none elsewhere.

The biggest gain usually comes from tier 1. Many teams put reversible writes behind a gate out of caution. That fills the queue with items that don't need a person, and it trains approvers to click. Let those run, keep a working undo, and review a sample instead.

Show the approver what they need, and nothing else

An approver can only judge what they can see. A card that says "Agent requests approval: process refund" asks for trust, not judgement. Every tier 2 and tier 3 card should show four things.

Keep the card short. If the approver has to open three systems to check it, they won't. Put the agent's reasoning in one or two lines, and let the approver expand the full trace if they want it.

Batch alike items, never mixed ones

Batching is how a tier 2 queue stays manageable. It is safe when every item in the batch follows the same rule, sits in the same value band and has no flags. Twenty identical address corrections can be one decision. A refund, a credit note and a message to a customer cannot.

Two rules keep batching honest. First, any flagged item leaves the batch and gets its own card. Second, the approver can still see and reject single items within it. Tier 3 is never batched.

Put limits on delegation

Approvers go on leave, and queues can't wait. Delegation is fine if it follows the same authority matrix your finance and risk teams already use:

• A delegate must hold at least the same approval limit.
• Nobody approves an action from an agent they built or own.
• The risk approver on a tier 3 item can only delegate to someone else in risk.
• Delegations expire on a set date, and the gateway enforces that.
• The audit trail records who approved on whose behalf.

3. Measure the approvers as well as the agent

Most teams track what the agent does. Fewer track what the approvers do. Yet approver behaviour is the best early warning that a gate is decaying.

The most useful measure is the override rate: how often an approver rejects or edits what the agent proposed, per action type. Read it with care. A falling rate can mean the agent is improving. It can also mean nobody is looking.

To tell the two apart, seed the queue. Every so often, the gateway inserts a test case with a known flaw, such as a payee that doesn't match the invoice. It is clearly marked in the log but looks normal on the card. If approvers catch it, the gate works. If they don't, you have learned something no dashboard would have told you. Keep seeded cases rare, never let them execute, and share the results with the team rather than using them to blame anyone.

We cover how these feed wider risk and return reporting in Part 5.

4. Loosen a gate only with evidence

Every gate should have a path to loosening. Without one, volume grows until approvers stop reading, and the gate fails anyway. With one, you move attention to where it is still needed.

We suggest a simple test before any action moves down a tier. Has it run for long enough to cover a full business cycle, such as a month end? Is the override rate low and steady? Have approvers caught every seeded case? Is there a working undo? Have there been no incidents? If the answer to all five is yes, the business owner and risk can agree to move it. Write the decision down, with its evidence.

Move in small steps. Take one value band first, such as refunds under a set amount, not the whole action type. Set a tripwire at the same time. If overrides or errors rise, the gateway puts the gate back without waiting for a meeting. And if something does go wrong, Part 6 covers how to stop, contain and recover.

5. Illustrative scenario: a freight company's payables desk

This scenario is illustrative. The company is fictional.

Sutlej Freight runs an agent that matches carrier invoices to delivery records and proposes payments and credit notes. At launch, every proposal went to the payables team for approval. Within weeks, the queue ran to hundreds of items a day. Approvers cleared most of them in seconds, and a duplicate invoice slipped through.

The redesign starts with tiers. Matching and tagging become tier 1, running on their own with a weekly sample. Credit notes and payments within a set limit are tier 2, batched by carrier and rule, with flagged items pulled out. Payments above the limit and any new carrier bank account are tier 3: a payables lead and someone from risk both approve.

Each card now shows the invoice, the matched delivery and what is unusual. The gateway seeds a small number of test invoices each month. After a clean quarter, risk agrees to move small credit notes for known carriers down to tier 1, with a tripwire. What changes is where people spend their attention. The queue is shorter, and the items left in it are the ones that need a person.

Moving forward

Pick one live or planned agent and list every action it can take. Mark which ones truly need a person today, and what that person should see. If you want help tiering the actions and designing the gates, our three-week Readiness Audit covers it. You can see how we govern our own use of AI in our AI governance policy, or book a readiness audit to start.

Sources

1. H. van der Sijs and others, Overriding of drug safety alerts in computerized physician order entry, Journal of the American Medical Informatics Association, 2006
2. R. Parasuraman and D. Manzey, Complacency and bias in human use of automation: an attentional integration, Human Factors, 2010
3. EU Artificial Intelligence Act, Article 14: Human oversight
4. UK Information Commissioner's Office, How do we ensure individual rights in our AI systems?, guidance on AI and data protection