economisting

Economisting: (e kon’ o mist’ ing) 1. The act or process of converting limited evidence into grand claims by means of rhetorical ploys, especially punning. 2. The belief or practice that empirical evidence can only confirm and never disconfirm a favored theory. 3. Conclusions that are theory-driven, not evidence based. See also confirmation bias, painting with a broad brush, Iraqi weapons of mass destruction, post-modern critical theory, marketing. 

Edward Tufte provides us with a new word to describe an old idea–using and presenting data in a manner designed to mislead the consumer of research.  In consulting, including areas of employee and customer engagement engagement research, this is reflected in:

• using statistical significance as a measure of practical importance (material significance),
• making grand claims that go beyond the evidence, such as (i) engagement models claiming that a single number measures engagement, (ii) that cherry-picked factors can predict engagement, (iii) that engagement measures can predict corporate performance or that (iv) rankings can determine much of anything let alone best in consumer satisfaction,
• using data to support decision-based evidence-making rather than evidenced-based decision-making.

We concur with Dr. Tufte that data analysis & presentation is an intellectual and a moral act. At Converge we endeavor to keep our work, in all areas of practice, economisting free.  Providing consumers of research, information without window dressing or spin.

Taguchi Loss Function (Why Goal Posts and Service Standards Don’t Work)

Taguchi Loss Functions are essential to representing the voice of the customer properly. The detailed calculations may not always be required in a VoC study, but a little Taguchi thinking when interpreting results is essential.

Taguchi thinking challenges typical goal post (specification) thinking. You know the type, believing that anything within specifications are good, while anything outside of the specifications is bad. In manufacturing there typically two specification limits, one on either side of a desired or nominal value. In services, usually faster is better so we have only one specification limit such as we will serve customers within ten minutes. Anything less than ten minutes is good service, anything more is seen as bad service. Read More

Qualitative/Text Analysis

Converge VoC uses advanced computer-based text acquisition and interpretation software that can read volumes of text data and provide unbiased sentiment analysis of the meaning behind the words. Relationships between words and phrases provide insight to the thinking behind what was said, improving our understanding of what really matters to our people.

This text analysis capability is important in our analysis of survey data and the qualitative information arising from focus groups and other sources of employee data. You have probably experienced having to read through mountains of employee comments. As entertaining as some of these may be, where’s the value? How do you move from what people have said, to evidence worthy of driving change? Even when summarized into themes, results are heavily influenced by the biases of those writing the summaries.

That’s where solid Qualitative/Text analysis comes in. By providing computer based text analysis, bias is removed and leaders can review the data free of interpretative spin. The Voice of the Customer is heard loud and clear and without bias. We can use a number of analytical tools to help. Simple word clouds provide a nice summary of text data and work very well as a communication feedback tool. People ‘get them’ instantly making them the perfect vehicle for quickly disseminating ‘what was said.’

sentiment analysis…

. . . uses computer-based semantic reading engines that “read” the qualitative statements and interpret employee sentiment. In the example, areas of strong positive sentiment are listed in green while negative sentiments are expressed in red, and here, sorted by frequency of mention. This provides an unbiased way of interpreting and summarizing qualitative/text data.

Sentiment analysis requires the development of unbiased models of concepts and phrases to ensure the computer-based reading technologies return meaningful and relevant results. Converge have these models pre-built for qualitative data analysis in human resource and employee survey applications meaning we can run sentiment analysis for your with interpretative models that have already proved themselves in the field.

concept mapping . . .

. . . results in powerful graphical representations of how people are actually thinking about critical concepts.

Understanding What People Value

Shouldn’t your customer research understand there is a difference between having your choice of beef or chicken and not crashing the plane? 

A Three Dimensional Understanding

only a difference in degree, measurable on a five point scale. 

Material Significance versus Statistical Significance

Improved accuracy and reliability . . .

.  .  .  because VoCAl uses tests of material significance over statistical significance.

Traditional customer research is based on polling techniques using statistical significance to identify important findings. When examining the differences between customer groups or segments for example, statistical significance is used to identify important differences, or used to distinguish important and unimportant product or service features. Similarly with results over time where statistical significance is used to identify important increases or declines in year to year results.

The problem is that statistical significance has nothing to do with practical or economic importance. That means most of what your research highlights as important, isn’t important at all.

• Those important product features, aren’t.
• Important differences between customer segments, meaningless.
• That significant increase in customer engagement, a fantasy.

Statistical significance gets the answers wrong about 90% of the time (false positive rate).

The misuse of statistical significance has devastating consequences for organizations in fields as diverse as law, health care, employee relations, economics and of course–customer engagement. (For a discussion on the disastrous effects of using statistical significance, see: The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice and Lives Highlight Lecture by Stephen T. Zilak and Deirdre N. McCloskey, posted on the Converge-Net website with permission of the authors.)

VoCAl’s use of material of material significance means the answers you get, are right, better than 99% of the time. That means marketing and operations can stop chasing statistically significant ghosts, and can back to the business of building solid relationships and improving product and services by focusing on what really matters to customers.

Statistically Significant Nonsense

If you don’t believe it, why are you using it?

Almost all customer survey research, probably the research you are using now, assumes that what is important to people can be determined by the formula for standard error – significance testing. Read More

Answering Questions Important to Business as Opposed to Politics

Voice of the Customer (VoC) was developed by Yoji Akao in conjunction with Quality Function Deployment (QFD)–a product and service design methodology.  The intent was to embed concrete and clearly specified customer requirements directly into the design specifications of new products–VoC was to identify customer requirements; QFD was to connect them with design specifications. The strategy changed the face of customer research. Read More

VoCAl: Supporting the Customer Driven Organization

Interested in customer research that actually makes a difference? Voice of the Customer (VoC)  is a research methodology designed to support customer driven change — where hard evidence is used to drive improvement.  Read More