The notion that there must always be a "human in the loop" has become de rigueur in any discussion of AI implementation in business. The cynical side of me suspects that part of the reason is to reassure everyone that AI won't lead to mass layoffs, but there are also plenty of practical reasons to have humans check the work of AIs.
Or maybe not. At least not for long.
A recent article in Harvard Business Review argues that human supervisors quickly become numb and simply approve the recommendations of the AI as it becomes increasingly accurate. And I think the author has a point, based on what I've seen of human/AI interactions in semi-autonomous cars.
There needs to be a better way.
Let's have a look.
Early in the development of its driverless cars, Google gave demo models to employees. Google warned them that the AI wasn't perfect and instructed them to have their hands on the wheel, always ready to take control. That worked... for a little while. A very little while.
As the AI proved to be reliable, the drivers began to relax their hold on the wheel, then started to take their eyes off the road, then began reading or looking at their phones. When the car signaled that the drivers had to take over, they needed perhaps 10 seconds to fully reengage. Given that a car going 60mph travels nearly 300 yards in 10 seconds, Google realized its hybrid approach wouldn't work and committed to producing only cars that could be fully autonomous.
Other makers of autonomous vehicles, most notably Tesla, continue to offer cars that take a hybrid, human/AI approach, but I think that's irresponsible (as I wrote at length a year ago).
Now, an article in Harvard Business Review offers a similar view about the dangers of having a "human in the loop" to supervise work done by AIs. The author, a senior fellow at Cal, writes:
"Decades of research on automation show that better automation systems lead to worse human safety layers; as automation grows more reliable, the people monitoring it grow complacent, their skills atrophy, and they become less attentive.... AI compounds the problem. Its failures sound as confident, fluent, and well-structured as its successes....
"Even when validation works and people catch errors, every correction is captured as training data. Each exception a person handles teaches the system to handle that exception; the better the data, the smaller the scope of human validation needed in the future."
The article says the lack of reliability of the humans in the loop is actually just the start of the problem. He writes:
"The larger problem with the human-in-the-loop organization, though, is that the entire blueprint reads the transformation backwards. It organizes the company around the shrinking frontier of what machines can’t yet do. But there’s a larger opportunity here. Because AI removes constraints, it enables new combinations of work, new decision rights, and new forms of value. Organizations, accordingly, have an opportunity to reorganize and 'rebundle' around those new decisions rather than the shrinking residue of the old work."
He goes on to lay out how to think about rebundling: "Instead of asking which tasks the machine can take over, leading organizations should ask which new roles the technology makes possible, and then redesign the organization around them." He lists several steps that can lead you to new approaches, which seem intriguing and which you may want to explore.
For now, though, I just want to make the point that "human in the loop" isn't a panacea, certainly not in the long term. The reviewers of AI work can quickly become as reliant on the AI as those Google employees who found themselves with their hands off the wheel, their eyes off the road and their minds far away from driving when a problem popped up in front of them.
Cheers,
Paul