Why Customer Surveys Are Passe

Ever since a college statistics class in which I learned about the polling debacle in the 1936 U.S. presidential election, I’ve cast a wary eye at surveys. The respected Literary Digest got fully 2.4 million people to respond to a poll on their preferences that year and reported that Alf Landon would easily defeat incumbent Franklin D. Roosevelt. Or maybe not. While the poll found Landon leading 57% to 43%, in fact, FDR won 61% of the popular vote. He pitched a near shutout in the Electoral College, winning 523 to eight.

It’s not that consumer surveys don’t have value. They do, especially as long as we’ve all learned to avoid pitfalls such as the wild sampling bias that led Literary Digest so far astray. The surveys can be especially valuable when they track changes in attitudes over many years or even decades.

But customer surveys have always been rather crude. They’re highly sensitive to how questions are phrased, and they’ve always operated at a remove from what we really care about: Surveys tell us what customers say they’ll do, not necessarily what they’ll actually do.

There’s now a better way. And it’s especially important at a time when so many insurers are trying to reinvent the customer experience — they need to know what really matters and what doesn’t, in fact, affect customer behavior.

continue reading >