I come from a long line of people who scream at the TV when our Steelers are playing. My Dad taught my siblings and me well, and I've done my fatherly duty by passing the passion along to my two daughters. When the younger one was watching a Steelers game in college, a fellow student knocked on her door, said he'd heard a lot of yelling and wanted to be sure "you guys are all okay." My daughter was watching the game by herself.
I'm proud of you, Clare.
As you might imagine, I can be a tad critical of what the announcers say. But I was especially struck by the analysis by the TV pundits following the Cleveland Browns' upset of the San Francisco 49ers on Sunday.
The pundits praised the Browns and denigrated the 49ers as though the outcome had never been in doubt. But, but, but... the 49ers had a 41-yard field goal attempt with six seconds to go that would have won the game, and NFL kickers are about 85% successful from that distance. The Browns didn't just need a coin flip to win the game; they needed the equivalent of calling a coin flip right three times in a row.
Yet, given how the human mind works, everything about the game was viewed by the pundits based on the fact that the 49ers' kick leaked slightly outside the right goalpost. Had that kick been a yard or so to the left, the pundits would have had a completely different interpretation -- those gritty 49ers, etc.
I don't mean to trash TV football pundits -- well, maybe a few of them... but not more than half... two-thirds at the most -- but wanted to point to the faulty analysis of the game because it demonstrates a serious cognitive bias that's baked into us humans.
It's called survivor bias, and it's a real threat to innovation.
As much as it hurts me to be kind to the Browns (Pittsburghers call them "the Brownies"), their defense was awesome, and they mostly outplayed the 49ers. The win also makes for a great story. The Browns were without their injured $230 million quarterback Deshaun Watson and instead started a journeyman who had been in the XFL four years ago. Yet the Browns beat a team that was generally considered to be the best in the league.
After the game, the TV pundits could only sing the praises of the Browns. For instance, former Dallas head coach Jason Garrett said on "Football Night in America": “The guys on the defensive line for Cleveland were owning the line of scrimmage.... [49er quarterback Brock Purdy] certainly came down to Earth today. Struggled with the conditions, a lot of balls were slipping out of his hands, he wasn’t accurate…he just didn’t play as we’ve seen him play to this point.”
Everything he said is true, but neither he nor any of the other analysts said anything about how lucky the Browns had to be to win the game. Even before the field goal attempt by the 49ers that gave them an 85% chance of winning, the Browns benefited from a phantom unnecessary roughness penalty on what would be their final drive of the game (a clear error demonstrated by exhaustive replay analysis). Rather than face fourth-and-10 against a ferocious defense, the Browns picked up 15 yards and a first down, on their way to the field goal that provided the 19-17 margin of victory.
Just imagine how the take on Purdy would have differed if the 49ers had made the field goal. All his struggles would have been noted, but the narrative would have been about how the second-year quarterback, the last pick in the 2022 draft, had survived a brutal challenge from the Cleveland defense. He had marched the 49ers 52 yards down the field with time running out and had set up a field goal that kept his team undefeated. Why, he might be the new Tom Brady.
Purdy for MVP! Purdy for President!
There is certainly some smart analysis in football. (Looking at you, Ryan Clark and Bill Barnwell.) And that kind of analysis needs to be brought to insurance and innovation.
Barnwell actually accomplishes a lot based on just a single, simple adjustment to win-loss records. Rather than decide that some teams have a talent for winning close games, he believes that games decided by seven points or less are essentially coin flips. So, all a team's close wins become half-wins for analytical purposes, as do close losses. Based on that adjustment, he predicted, for instance, that the Minnesota Vikings would tumble from their 13-4 record last season, because they were 9-0 in close games. Sure enough, the Vikings began this season 1-4. (Barnwell also predicted that my Steelers will have their first losing season in Mike Tomlin's 17 years as head coach, but he can be wrong sometimes... I hope.)
The problem for corporate innovation efforts is that they are typically evaluated based simply on a win-loss record, without a Bill Barnwell on ESPN explaining the sorts of other issues that should be incorporated into the analysis.
Venture capital firms are built on the idea that nine out of 10 startups fail. You just have to be sure that the one success is so big that it covers for the others. In corporations, though, the people who keep moving up the corporate ladder are the ones with the unbroken string of wins, so the strivers don't want a one-in-10 chance of major success. They want a 10-in-10 chance of success, even if that success is small.
That's why, I believe, telematics adoption in cars is still so low even though its potential has been apparent for decades -- I, personally, first wrote about the technology almost 25 years ago -- and even though Progressive introduced its Snapshot capability 15 years ago. This need for guaranteed success is why take-up of the IoT has been so cautious, why companies are being so careful about the switch from the traditional "repair and replace" approach in insurance to a "predict & prevent" model, why so many companies are talking about the potential for generative AI but so few in insurance are actually exploring it in any significant way.
Guarantees of small successes don't lead to breakthroughs.
The survivor bias is hard to shake, though. The first time I encountered it was in a great book on risk that talked about a ship full of Romans who were caught in a storm that was about to sink them but who prayed to the gods and survived. No one ever heard from the Romans who prayed to the gods and drowned, so how could anyone dispute the claims that the gods saved that one ship?
A very smart friend of mine once wrote a book about self-made billionaires and reported that they tended to be single-minded and made big bets. Great, but what about all the folks who were single-minded, made big bets and lost those bets? You could do a book about the benefits of buying lottery tickets if you just interview the winners.
We all fall victim to the sense that rich people must be smart. How else did they become so successful? But I've interviewed more billionaires than I can count and... hmmm. Lots are very impressive, but plenty just aren't. And I've interviewed loads of people who didn't make it big who struck me as much smarter than most of the billionaires -- such as the fellow who invented the iPad (but before the technology was quite ready) and launched an early form of eBay (but took ownership of the goods being auctioned, rather than just facilitating the exchange between private parties).
The survivor bias has even reared its head in academic research and has required corrective action. Papers on, say, the potential for new drugs get published when they find a statistically significant benefit, which generally means at least 95% likelihood. But research that doesn't find the desired result tends to not be published. Non-results are boring. So those reading the academic research see the survivors -- the studies that are 19 out of 20 certain of a positive link -- but not the potentially many other studies that found no link.
Government sponsors are increasingly requiring that all research be published, whether or not it produced an exciting result. And that sort of discipline needs to be brought to business.
Yes, those who have a sterling record of success might well be CEO material. But how big were their successes? Did they take any actual risks? What about the noble failures, the people who did take big risks and failed through no fault of their own?
If we can get away from the survivor bias that people who succeed must be winners and can focus on rewarding noble efforts, we'll be much less cautious and more innovative as an industry.
Go, Steelers.
Paul
P.S. If you must know, my pet peeve with TV football analysts is their years-long claim that a receiver catching a 50-yard pass by outleaping a defender grabbed the ball "at its highest point." Paying even the slightest attention to physics, it's obvious that the ball reached its highest point when it was halfway to the receiver -- 25 yards or so upfield. At that point, the ball was 40 feet or more above the ground. While NFL players are exceptionally impressive athletes, no, it isn't possible for someone to catch a ball at its highest point, 40 feet or more in the air.
After years of my yelling at the TV, most announcers have changed their terminology and say a receiver "high-pointed" the ball when he makes a leaping catch on a long pass over a defender. That terminology doesn't make a lot of sense, but at least it's an improvement over grabbing "at the highest point."
Next, we'll get announcers to stop talking about "negative yardage" when they mean a loss and "positive yardage" when they mean a gain.
Here's hoping.
Go, Steelers.