Author: Frank Buckler, Ph.D.
Published on: September 3, 2021 * 9 min read
What if you have thousands of NPS scores to analyze? How will you do the customer feedback analysis? How will you draw impactful conclusions from new batches every week?
One way is to read every response, but it will not be possible to read thousands of them. So, how can AI help you in automating your task to gain actionable insights? If you don’t know what to do, don’t worry. You are at the right place where I will be guiding you in the right direction.
FREE for all client-side Insights professionals.
We ship your hardcopy to USA, CA, UK, GER, FR, IT, and ESP.
You can experience frequency illusion when the information you recently learnt, noticed or experienced crops up everywhere. It is a frequency bias that gives you the feeling everyone is talking about the subject you noticed or is swiftly swirling you. But, in actual, there is no increase in the occurrence of that subject. It’s just because you’ve started to notice it.
So, the impact-frequency illusion is just a paradox that we now ask the customers, why did you rate our product this way? We expect that they should tell us their rating criteria so that we can know and count the most often mentioned topics that drive the rating. That’s why everyone in the world counts what people are saying and then acts on it to improve their customer satisfaction levels. But, when you do the math and the analysis of how predictive the most mentioned topics and the information are in explaining what people do while giving their feedback, there arises a mismatch.
There are certain reasons for mismatch, and they are as:
Here the examples show that:
The new or low performing categories have a low frequency. So, if your products are not so good and are at a reliable performance, then, of course, they are not so often mentioned. But people may be experiencing your good reliability or be satisfied with the status your products have.
The promoters are your loyal customers who are most likely to suggest your services or products to others. A person most likely tries a service when it is suggested by a friend or an acquaintance instead of being suggested through promotions or advertisements. So, you need to keep your promoters happy once they are identified.
On the contrary, detractors are the customers who are dissatisfied with your services or products and are most likely to give negative feedback. So, you need to improve their experience to avoid a domino effect of bad referrals.
We need an approach to find out which topics are important. So, the first thing we may think of is comparing promoters with detractors. This is because we as humans learn by correlating concepts and ideas. So, we compare promoters with detractors as we are interested in finding the difference between the two. However, different problems arise when we try to find the difference between the two, and they are as:
Consider the above examples of Sonos. In the first example, we see that 61% of promoters mention that Sonos has a great sound. Also, there are 25% of detractors who believe the same. It’s something we don’t expect from detractors as we generally believe that they always give negative feedback. But the example proves that we may go in the wrong direction. Further, the customers choose our product because they like it as it performs well. But, we should know that improving the product features doesn’t always contribute to increasing the loyalty of the customers, and it becomes clear when we do the predictive analysis.
In the second example, we see that there are 4% of promoters and 3% of detractors who give negative feedback regarding the Sonos sound system. So, why do the promoters have larger complaints about it? Does it mean bad support is good?
No, it doesn’t. Actually, the reason is that the detractors are the new customers who have more support calls as compared to promoters. They also have an initial excitement of trying a new product or a service, so they have a smaller percentage in terms of giving bad support. In short, there is a confounding effect of the new customer property that distorts the correlation and gives counter intuitive results. It does not allow you to measure the importance of service support. So, you do not need to do that.
As a solution of impact frequency illusion, what we need is something like a key driver analysis. So, the basic idea of key driver analysis is that there is a set of possible drivers, and each of them has a mutual impact. It implies across a number of different responses; we can separate out a different impact of each driver.
The basic form of key driver analysis is multiple regression. Let’s suppose rating is the outcome to be predicted, so the regression formula is as:
Rating = D₁α₁ + D₂α₂ + D₃α₃ +… DNαN + c
Here D₁ denotes driver 1, and α₁ is its weight. We add another impact that causes the rating, and that is driver 2. This driver gets its respective weight α₂. In the same way, there are more drivers with their weights, and we name them DN. We also add a constant number c that has zero impact.
The driver D indicates the customer response and can be a stated quality, or a sentiment, etc., and α is the parameter. The drivers come from the survey, and parameters are found by analysis using regression to find those values of α that best predict the rating of every single respondent. The weight α has a lot of significance in the regression equation because if it becomes zero, it doesn’t matter what the customer is saying. So, the parameter is also an analogy to the impact.
So, now you know that if the value of α1 is high, driver 1 will contribute significantly to predicting the rating. Let’s have a look at the idea proposed by Granger.
According to Granger:
It means if we assume that the driver 1 and driver 2 are causing the rating and there is nothing else influencing it, then the impact we measure is the causal impact.
For key driver regression, we have some more assumptions like:
Closed World – According to this assumption, no external factor or parameter other than α influences the rating.
Let’s summarize what we discussed so far. So, we have an impact frequency illusion which means that the most often mentioned topics are, most of the time, not the most important ones. That is why we need drivers. They have a mutual impact, and they are represented in terms of a regression formula. According to Granger’s idea, you can find causal to your outcome by keeping in mind the three assumptions, i-e; linearity of drivers, closed world assumption, and independence of drivers. Further, we discussed that comparing the frequency between promoters and detractors can be quite misleading as the difference can not be found due to the high correlation.
Our Group: www.Success-Drivers.com