Economisting: (e kon’ o mist’ ing) 1. The act or process of converting limited evidence into grand claims by means of rhetorical ploys, especially punning. 2. The belief or practice that empirical evidence can only confirm and never disconfirm a favored theory. 3. Conclusions that are theory-driven, not evidence based. See also confirmation bias, painting with a broad brush, Iraqi weapons of mass destruction, post-modern critical theory, marketing.
Edward Tufte provides us with a new word to describe an old idea–using and presenting data in a manner designed to mislead the consumer of research. In consulting, including areas of employee and customer engagement engagement research, this is reflected in:
- using statistical significance as a measure of practical importance (material significance),
- making grand claims that go beyond the evidence, such as (i) engagement models claiming that a single number measures engagement, (ii) that cherry-picked factors can predict engagement, (iii) that engagement measures can predict corporate performance or that (iv) rankings can determine much of anything let alone best in consumer satisfaction,
- using data to support decision-based evidence-making rather than evidenced-based decision-making.
We concur with Dr. Tufte that data analysis & presentation is an intellectual and a moral act. At Converge we endeavor to keep our work, in all areas of practice, economisting free. Providing consumers of research, information without window dressing or spin.
We love automated text analysis and use it extensively. See Integrated Text/Qualitative Data Acquisition for some details.
The key here though is the word Integrated. Text/qualitative analysis used in isolation of quantitative techniques can lead to disaster. Beware of automated text analysis software claiming push button VoC–if it seems to good to be true, it probably is.
That said, the analysis qualitative/text data can truly take your customer research forward. Not only can the meaning behind qualitative statements on surveys be analyzed, but textual data in databases, on the web or across social networking sites can also be gathered and analyzed.
Converge VoC uses advanced computer-based text acquisition and interpretation software that can read volumes of text data and provide unbiased sentiment analysis of the meaning behind the words. Relationships between words and phrases provide insight to the thinking behind what was said, improving our understanding of what really matters to our people.
This text analysis capability is important in our analysis of survey data and the qualitative information arising from focus groups and other sources of employee data. You have probably experienced having to read through mountains of employee comments. As entertaining as some of these may be, where’s the value? How do you move from what people have said, to evidence worthy of driving change? Even when summarized into themes, results are heavily influenced by the biases of those writing the summaries.
That’s where solid Qualitative/Text analysis comes in. By providing computer based text analysis, bias is removed and leaders can review the data free of interpretative spin. The Voice of the Customer is heard loud and clear and without bias. We can use a number of analytical tools to help. Simple word clouds provide a nice summary of text data and work very well as a communication feedback tool. People ‘get them’ instantly making them the perfect vehicle for quickly disseminating ‘what was said.’
. . . uses computer-based semantic reading engines that “read” the qualitative statements and interpret employee sentiment. In the example, areas of strong positive sentiment are listed in green while negative sentiments are expressed in red, and here, sorted by frequency of mention. This provides an unbiased way of interpreting and summarizing qualitative/text data.
Sentiment analysis requires the development of unbiased models of concepts and phrases to ensure the computer-based reading technologies return meaningful and relevant results. Converge have these models pre-built for qualitative data analysis in human resource and employee survey applications meaning we can run sentiment analysis for your with interpretative models that have already proved themselves in the field.
concept mapping . . .
. . . results in powerful graphical representations of how people are actually thinking about critical concepts.
Size of the ‘spheres’ represent the frequency of mention of critical concepts and lines represent connection points among these. You can also see less important concept points (in grey) operating within the spheres to which they are related. Even color is used to convey characteristics of employee sentiment towards the concepts expressed.
This capability supports our advanced data gathering technologies that can read through internal and external data bases and even scan social media, news and other sites on the web to capture what people are saying about you and gauging general sentiment.
Improved accuracy and reliability . . .
. . . because VoCAl uses tests of material significance over statistical significance.
Traditional customer research is based on polling techniques using statistical significance to identify important findings. When examining the differences between customer groups or segments for example, statistical significance is used to identify important differences, or used to distinguish important and unimportant product or service features. Similarly with results over time where statistical significance is used to identify important increases or declines in year to year results.
The problem is that statistical significance has nothing to do with practical or economic importance. That means most of what your research highlights as important, isn’t important at all.
- Those important product features, aren’t.
- Important differences between customer segments, meaningless.
- That significant increase in customer engagement, a fantasy.
Statistical significance gets the answers wrong about 90% of the time (false positive rate).
The misuse of statistical significance has devastating consequences for organizations in fields as diverse as law, health care, employee relations, economics and of course–customer engagement. (For a discussion on the disastrous effects of using statistical significance, see: The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice and Lives Highlight Lecture by Stephen T. Zilak and Deirdre N. McCloskey, posted on the Converge-Net website with permission of the authors.)
VoCAl’s use of material of material significance means the answers you get, are right, better than 99% of the time. That means marketing and operations can stop chasing statistically significant ghosts, and can back to the business of building solid relationships and improving product and services by focusing on what really matters to customers.
Almost all customer survey research, probably the research you are using now, assumes that what is important to people can be determined by the formula for standard error – significance testing. Read More
Voice of the Customer (VoC) offers a whole new world for the Marketing professional because VoC identifies what really matters to your customers. VoC replaces tests of statistical significance with tests of real world or material significance. This allows the Marketing professional to design and redesign products, services, programs and processes that will actually contribute to improved customer satisfaction and loyalty.
There is no statistical gobbledygook. Only hard science applied to the problem of identifying issues of real and practical importance to your customers.
That’s the new world of VoC. Developing sustainable, practical improvement solutions for business. Your business.
Voice of the Customer (VoC) was developed by Yoji Akao in conjunction with Quality Function Deployment (QFD)–a product and service design methodology. The intent was to embed concrete and clearly specified customer requirements directly into the design specifications of new products–VoC was to identify customer requirements; QFD was to connect them with design specifications. The strategy changed the face of customer research. Read More