Benchmarking Customer Satisfaction Doesn’t Work – GAO

Research conducted by General Accountability Office of the United States Federal Government, comparing customer service improvement strategies in the Internal Revenue Service , confirms what users of Voice of the Customer (VoC) and Voice of the Employee (VoE) users have known for some time — comparing data across organizational units (other organizations or branches) leads to declines in performance.

In what amounts to a sophisticated controlled experiment, the GAO divided several branches of the IRS into two groups. The first used a benchmarking strategy.  This group used customer satisfaction research to build improvement strategies based on comparative areas of strengths and weakness with other branches of the IRS.

The second group was the control group. These branches of the IRS used only their own customer satisfaction data – no comparisons with other branches were made. In essence, this voice of the customer approach focuses improvement activity directly on the expressed concerns of the customers.

The findings of the research were as clear as they were overwhelming. Branches of the IRS that used the benchmarking approach saw customer satisfaction go down and complaints rise across the board. In contrast, branches in the control group saw performance, measured in terms of customer satisfaction, go up. Voice of the customer branches also saw other performance metrics, including recoveries, rise as well.

The GAO research recommended that the IRS end the practice of benchmarking - measuring customer service across various field units and using the comparative  information to identify strength and weakness areas. Rather, it recommended government organizations adopt the control group strategy of using customer satisfaction data to drive improvement directly to the organization unit to which the feedback was directed.

We could have told them so. Users of VoC and VoE from Converge have heard us repeat this message many times. Comparing your customer satisfaction or employee satisfaction data to benchmarks may satisfy managerial curiosity, but decisions based on these comparisons always, always drives declines in performance. The more the voice of the customer is manipulated, summarized, benchmarked and generally messed with, the more the meaning in the message is lost or misinterpreted. The results are decisions that degrade performance.

Listening to the voice of the customer means listening directly to what is being said (without all manipulation) and making interpretations that increase understanding as a basis for action. The whole point of customer (or employee) research is to identify what improvements to product or service need to be made.  Comparing data with others is only good for public relations purposes. Your customers (or employees) won’t buy feel good messages when their actual experience with you is different than what is being promoted. Just the opposite in fact. They will come away believing you are disconnected or disengaged with what is really going on..

The bottom line is this — gathering data from your customers (or your employees) should identify specific issues and concerns of practical importance to the business. Addressing these issues of practical importance will improve performance.

It really is, just that simple