Founder of CX-AI.com and CEO of Success Drivers
// Pioneering Causal AI for Insights since 2001 //
Author, Speaker, Father of two, a huge Metallica fan.
Author: Frank Buckler, Ph.D.
Published on: October 29, 2021 * 9 min read
It belongs to the fundamentals that every insights professional knows. Biased samples can lead to biased results. Nothing new.
Nothing new? When Hiram Bingham discovered Machu Picchu in 1911, it was not new either. The locals knew this spot well, but they underestimated the role and importance of this spot.
Same with ‘filter bias’. It is not only all around us. It has a -sometimes- devastating effect on what we believe about the world.
Take social media. The term ‘filter bubble’ or ‘echo chamber’ are widely known for the impact that a filter can have. Social feed algorithms learn which content leads to engagement. I then only show the content that is engaging for you.
Engaging content is most likely in line with your opinion, and it is most likely negative and alarming. The result is a polarization of unbalanced views and a fearful worldview.
In this LinkedIn article, I am describing the background in more detail, and I am proposing how feed-algorithms can be optimized to stop filter bubbles.
But for now, I like to shed light on the impact to businesses when underestimating the filter bias.
FREE for all client-side Insights professionals.
We ship your hardcopy to USA, CA, UK, GER, FR, IT, and ESP.
What would happen if you ask non-churner a similar question like “what needs to be improved”. What if they mention the same topics like churners? Would you still believe what churners most often mention as a reason is the true motivator for churn?
Churner feedback provides you a bias feedback by design. It builds on the assumption that churners know and articulate the true reason for churn.
This assumption is more or less broken.
It can only be validated with an unbiased sample of churners and non-churners.
Most companies do it. Asking an NPS or Satisfaction question and then asking WHY. Then companies assume what customers say is unbiased and can be taking as-is.
They assume customers are willing and able to articulate why they are loyal reliably.
This assumption is broken too.
Human brains are notoriously lazy. If there is no strong incentive, customers’ brains spit out instant associations instead of well-thought-out replies.
This is why restaurant customers most often mention “great taste”, insurance customer “great service” and speaker customer “great sound”.
It’s often an instant reaction without deeper rational processing.
The process and context of interviewing itself provide a bias that can turn results upside down. Here is an article that goes deeper on this and how to unbias results.
The “feed” of customer feedback is biased like a Facebook feed.
Most companies are sending this feed to the frontline. It should enable the frontline to learn what customers think.
But it does not do the job. Instead, the frontline will learn something else. It will learn what customers spit out by instinct when asked. But not necessarily what will make them happy or more loyal.
CX feedback needs a causal analysis (or some kind of driver analysis) to judge its importance reliably. This article discusses how to solve the issue.
Public ratings on Amazon, Google Maps, Google Play, Trustpilot, Capterra, and much more are a great and free source of feedback for many businesses and local branches.
Yes, sure, it is biased in multiple ways
This means the sampling of rating feedback is largely biased.
What is not biased by the 3 filters is the relationship between rating and explanation. It enables you to understand still the impact of topics using causal or driver analysis.
Social listing uses public conversions to measure what people talk about to indicate what is going on.
But Social data is even more biased.
80% of social conversions are of non-human origin. Those conversations that are human, are biased by the feed algorithm. This algorithm determines the reach of posting. Some opinions may be predicted to be less engaging and thus will get fewer eyeballs.
Compared to the Rating feedback, it’s harder to debias social feedback, because the reference is missing.
One approach is to use existing brand tracker and machine learning algorithms to find the unknown link between social conversations (as an aggregate per time and region) onto brand tracking results. This website gives more details.
Are you managing a team? Certainly, you ask for feedback regularly. Did you ever realize how biased this is?
I made this observation myself in my previous life as a Marketing Director. My team reported to me what great things they did and what mistakes the other departments were making.
The information flow to upper management is like a Facebook feed. It is optimized for engagement. It’s certainly not optimized for you to learn the truth.
A biased feed of facts will inevitably result in wrong opinions.
Managers without a direct line to the front line (or other elaborate ways of truth discovery) will entirely lose grounding.
I know many people who would support this with anecdotal evidence. I am sure you do too.
The phenomenon is also one reason why companies like McDonald’s have Upper Managements work in the front line regularly.
Besides this, it takes an Employee Experience Feedback program that deploys the same rigor in analytics and understands the true impact of topics out of unstructured feedback.
The most important step has already been done: you read this article. Being aware of the phenomenon will give you healthy doubts and awareness for the biases all around us.
Practical ways to tame the bias effects are typically modeling analysis work. It uses biased feedback as predictors and objective outcomes as a benchmark. Now, with machine learning, we can find the link between input and output. We can find this link that is unknown because the bias is unknown.
If you want to dive into more cutting-edge CX thinking, the CX Analytics Masters Course is for you. It’s free for enterprise insights professionals. If you are looking to discuss some of the advanced technics mentioned above with an expert, reach out at www.cx-ai.com
Now I have a question: Was this article helpful? Please DM me directly with any comments or questions
Each month I share a well-researched standpoint around CX, Insights and Analytics in my newsletter.
+4000 insights professionals read this bi-weekly for a reason.
I’d love you to join.
“It’s short, sweet, and practical.”
