The Move to Voice of the Customer

Posted on Dec 30 in Featured, Issues in Customer Research, VoC News & Issues by robert gerst

Increasing numbers of organizations are abandoning traditional customer survey research in favour of Voice of the Customer (VoC).  Moves by Toyota, Ford, Hewlett-Packard and Toshiba to VoC has driven some of this trend, but there are other interesting factors at work as well.

Outside of the leadership displayed by the likes of Toyota, abandonment of traditional research methodologies has been driven in large measure by criticism arising from the scientific/statistical community. Much of this criticism has focused on the claims that traditional customer research (such as polling) is scientific.

Yet the methods used, including misuse of statistical significance testing and the sheer nonsense of ranking organizational performance (i.e.; 100 best) or of using statistical models to predict complex outcomes (i.e.; 12 factors that determine loyalty) are anything but scientific. It’s more cargo-cult science – using the tools of science for appearances sake without the knowledge required to use the tools effectively.

1. Criticism from the Scientific/Statistical Community

A big reason for the move to VoC is attributable to recent, well founded, attacks by the scientific/statistical community on the widespread misuse and overselling of traditional customer survey research. Traditional research based on enumerative methods (such as polling) simply can’t deliver what is promised. The enumerative emperor has no clothes.

Misuse of statistical significance as an indicator or measure of practical  importance.

Traditional customer research typically uses tests of statistical significance to highlight differences among departments, branches,  products, customer segments, time periods or geographical regions. Differences between customer segments, for example, are said to be important if the differences are statistically significant to some confidence  level such as 95% (the infamous 19 times out of 20 statistic). Management is expected to pay attention and to respond to these statistically significant results.

Of course, statistical significance has nothing to to do with importance and is of little economic or scientific relevance. Unfortunately, that hasn’t stopped some  from selling the fiction that it does. Recently, a detailed and brilliant attack from the scientific community has come from authors Stephen T. Ziliak and Deirdre N. McCloskey  in their highly recommended: The Cult of Statistical Significance: How the Standard Error Cost Us Jobs, Justice and Lives. The title says it all. Organizations that use statistical significance as a measure of importance are literally flaying at windmills and destroying their relationship with customers in the process.

Fortunately, the efforts in the scientific/statistical community are having an impact. In realizing the shortcomings of enumerative approaches, businesses are turning to analytic research methodologies such as Voice of the Customer with its sound scientific basis and focus on practical, real world or economic significance, to properly identify what is, and is not, important to customers.

Flagrant overreaching.

The scientific/statistical community is also taking direct aim at many of the claims made by those selling traditional customer research. Among the two most outrageous:

• that  comparisons can be made among companies to produce a ranking of performance (i.e.; 50 best or top 100),
• that statistical models can predict customer loyalty, satisfaction or engagement (i.e.; 12 things that predict loyalty).

Unfortunately, the simple truth is that both claims are well beyond the ability of the statistical sciences. What is being sold here is pure snake-oil.

Corrupt Comparisons  (i.e.; 50 Best).
Creating corrupt comparisons, occurs when the results of one company are compared with the results of a group of companies (i.e; 50 best). For example, comparing your customer loyalty score with the customer loyalty scores for a comparison group. Conclusions are drawn that usually take the form of  a ranking, where one company is said to be higher or lower in the ranking than others. (We are in the highest quartile!). Such conclusions are examples of overreaching and are complete nonsense, but they sell.

Some of the snake oil involved is in having you believe that a single number, your customer loyalty score in the current year for example, can be taken as an indicator for organizational performance generally. Comparing your measures to similar measures of comparative organizations compounds the error while promoting the delusion that scientific comparisons are being made. (Such comparisons also make use of statistical significance testing which adds a whole new layer of nonsense to the process.)  This is the scientific equivalent of deducing a trend from a single data point – a mathematical miracle of immaculate conception.

Basing branding, product and service decisions on such fanciful conclusions is dangerous. It leads the organization to pursuing directions that have no potential for improving the customer relationship while ignoring others that do. The net result being wildly misdirected efforts. Small wonder research by the GAO in the United States has indicated that the least effective customer strategies are built on a foundation of organizational comparisons (see Benchmarking Customer Satisfaction Doesn’t Work).

Meaningless Models (i.e.; twelve factors that predict customer loyalty).

Meaningless models are those that use statistical correlations to predict  customer engagement, loyalty or satisfaction. Examples are identifying the ten factors determining customer loyalty or the twelve drivers of customer satisfaction. Here again if a statistics course was part of your education, there is a good chance you can recall one phrase your instructor pleaded with you to remember: correlation is not causation! It was good advice.

Customer research claiming to determine or predict levels of customer loyalty or satisfaction confuse statistical models of correlation (enumerative models) with models of causation (analytic models). Like shell game artists, they take advantage of the confusion to pretend one thing is another.  Such models of customer loyalty, engagement or satisfaction are as scientific as models linking declines in polar bear populations with decreases in the number of accordion players in Missouri. The two are correlated, but the relationship is a fantasy and the model, meaningless.

Edward Tufte, former Professor of Public Affairs at Princeton and now Professor Emeritus at Yale University, recently coined a word to describe such flagrant overreaching – economisting. (See Beautiful Evidence, Graphic Press LLC, 2006.) Derived from a German root, the word equates to the deliberate act or process of converting limited evidence into grand claims … see also the German meaning of mist. Such claims he notes, are evidence of mediocrity. We couldn’t agree more.

2. The Growth of Analytic Methods in Business

Another driver behind the move to Voice of the Customer research is the rise and acceptance of other analytic research methods in business, especially Lean, Six Sigma and Continuous Improvement. As companies gain experience with these improvement methodologies, their understanding of the differences between enumerative and analytic  methods has grown. With this comes the recognition that traditional customer research, which is enumerative, is simply incapable of answering the questions that are typically put to it.

For example, Lean Six Sigma improvement models call for improvement teams to gather the voice of the customer at some point in the improvement process – usually early on in the improvement model. Some companies equated this call with the traditional customer research being used. As experience with Lean Six Sigma grew, however, these companies began to realize that voice of the customer meant just that –  analytic research methods that properly capture customer requirements embedding them into product and process improvement activity.

At the same time, understanding of the distinction between enumerative and analytic methods has also grown. Organizations are simply doing a better job of appreciating the appropriate application and limitations of each research approach and doing a better job at aligning the research question with the research methodology used to answer it. (For more on the enumerative-analytic distinction in the research science see; VoC: Answering the Important Questions).

The Move to Voice of the Customer

The move to Voice of the Customer then has been prompted by three factors:

1. Strong criticism from the scientific/statistical community of the widespread misrepresentation and overreaching in traditional enumerative customer research. This is evidenced by the misuse of statistical significance and the making of corrupt and meaningless comparisons and models.

2. Growth and acceptance of other analytic methods through Lean, Six Sigma and Continuous Improvement. The use of these techniques has increased awareness of the distinction of enumerative and analytic research methods generally along with an increased appreciation for  the importance of aligning the research methodology with the research question at hand.

3. The performance of organizations that have adopted Voice of the Customer. Why do some companies simply do a better job at understanding their customers than other organizations? Why do some companies continuously develop products and services that people actually want to buy? One factor in this success is the ability of these organizations to capture the voice of the customer and embed it into products and services. You can’t do that with enumerative methods or measures of statistical significance. It requires the use of real world economic significance and Voice of the Customer.