The wrong question

Most conversations about AI in business operations get stuck on a yes-or-no question: should a person be in the loop or not?

Put that way, there are only two answers, and both are wrong. Keep a person on every step and the system adds cost without removing any. Remove people entirely and the business takes risks it never chose to take.

The better question is narrower. For each decision in a piece of work: who owns it, what would it cost to get it wrong, and what does the owner need to see to decide well?

What human in the loop means

Human in the loop AI is a way of designing work so that systems carry out what is routine and well understood, and people keep the decisions that carry real consequence. The person is not there to watch the system. They are there to own specific decisions the system is not allowed to make alone.

In practice, every action in a process falls into one of three groups:

  • The system acts on its own. The action is routine, the rules are clear, and a mistake is easy to catch and undo. The system completes it and records what it did.
  • The system proposes, a person approves. The action matters more, or the rules leave room for judgment. The system prepares the work and a recommendation; a person approves, changes or rejects it.
  • The system stops and escalates. Information is missing or conflicting, the request falls outside the rules, or the case is new. The system does not guess. It hands the case to a named person with everything it knows.

A related idea, sometimes called human on the loop, describes people who do not approve individual actions but watch the overall results and can step in or stop the system. Most well-run operations use both: approvals for consequential actions, and oversight of the whole.

Deciding where people belong

Where to draw the line is a design decision, and it should be made explicitly, one decision at a time. Five questions help:

  1. What is the consequence of a mistake? A wrong tag on a ticket is cheap. A wrong refund, price or contract term is not.
  2. Can it be undone? Reversible actions can usually be left to the system with checks. Irreversible ones, such as sending money or messages to customers, deserve a closer look.
  3. How clear are the rules? Where the business can write the rule down, the system can follow it. Where the answer depends on judgment, relationships or context the system cannot see, a person should decide.
  4. How much evidence is there? A step that has run correctly across many measured cases has earned more trust than one that is new.
  5. What do customers, contracts and regulators expect? Some decisions must stay with a person regardless of how well a system performs.

The answers will differ between businesses, and between steps in the same process. That is the point: oversight should match the risk of each decision, not be applied evenly across all of them.

This step belongs inside the redesign of the work itself, which we describe in Business process redesign, from first principles.

Designing the handoff

An approval is only as good as what the approver sees. If a person is asked to approve something they cannot check, the approval protects no one.

A good handoff gives the person:

  • The request, in plain language. What was asked, by whom, and when.
  • The recommendation. What the system proposes to do, and why.
  • The evidence. The records, rules and earlier cases the recommendation is based on, with anything uncertain clearly marked.
  • The consequence. What happens if they approve, and what happens if they do not.
  • The choices. Approve, change, reject or ask for more information, each with one clear action.

It also needs a clear owner and a time limit. A decision sitting in a shared queue with no owner is not oversight; it is a delay.

The rubber stamp problem

The most common failure of human oversight is quiet. People are asked to approve so many routine items that approval becomes a reflex. The review still happens on paper, but the protection it was meant to provide is gone.

Two signs point to it:

  • Almost everything is approved, quickly. If a step is approved nearly every time without change, it may not need a person. Or the person may no longer be looking.
  • Problems are found later, not at review. If errors reach customers after being approved, the review is not catching what it was designed to catch.

The fix is usually to ask people for fewer, better decisions. Move routine, reversible actions to the system with checks, and concentrate human attention on the cases where it changes the outcome.

Accountability that holds

Oversight only works if responsibility is clear after the fact as well as before it. For every consequential action, the business should be able to answer:

  • Who or what made the decision?
  • On what information, and under which rule?
  • Who approved it, and when?
  • Can it be reversed, and how?

Some limits should be enforced by the systems themselves, not left to anyone's judgment in the moment: spending limits, permissions, separation of duties, and approval rules that cannot be skipped. These controls are part of a good design, not friction to remove.

Exceptions are where much of this is tested. We look at that in more detail in The exception is the real process.

Moving the line over time

The first version of a redesigned process should be cautious. As results build up, the line between system and person can move, one decision at a time, based on evidence:

  • Start with the system proposing and people approving.
  • Measure how often people change or reject the proposals, and why.
  • Where a type of decision is consistently approved without change, and mistakes are cheap and reversible, let the system act on its own with checks.
  • Where people often change the proposal, improve the rules or keep the decision with a person.

Autonomy should be earned this way, not assumed. And it should be possible to move the line back if results change.

Common mistakes

  • One rule for everything. Applying the same level of review to every action either slows the work or misses the risks.
  • Approvals without evidence. Asking people to sign off on decisions they cannot check.
  • No named owner. Shared queues where everyone can approve and no one is responsible.
  • Measuring speed only. Fast approvals are not a success if the wrong things are being approved.
  • Treating oversight as temporary. Some decisions should stay with people permanently, because of what they mean to customers or the business.