Founder of CX-AI.com and CEO of Success Drivers
// Pioneering Causal AI for Insights since 2001 //
Author, Speaker, Father of two, a huge Metallica fan.
Author: Frank Buckler, Ph.D.
Published on: April 14, 2023 * 7 min read
ChatGPT is making a splash because it’s so simple, so universal, and yet so “human.” Even my ten-year-old son writes stories and solves problems with ChatGPT and teaches his afternoon tutors a thing or two.
Experts talk about Large Language Models (LLM), because besides OpenAI’s solutions (ChatGPT, GPT4 , etc.) there are already other providers such as Google’s BART.
In early March, an article was now published by researchers at Harvard University who were able to reproduce the results of a simple conjoint survey using GPT3 – WITHOUT polling. The researchers instructed the machine to imagine it was a consumer and would go shopping for toothpaste and would see two brands with specific prices. “Would you buy one of them, and if so, which one?” The resulting price-demand function across hundreds of purchases (the machine can behave as erratically as hundreds of different consumers) strongly resembled the results of market research.
Will we still need market research in the future? Is the apocalypse here after all?
The answer, as always, is “yes”.
FREE for all client-side Insights professionals.
We ship your hardcopy to USA, CA, UK, GER, FR, IT, and ESP.
LLM are already extremely useful today in giving improvements to questionnaire formulations by suggesting wording. For sure, in DIY platforms like Surveymonkey there will soon be a functionality besides survey templates, where a virtual market researcher (i.e. an LLM) builds a questionnaire automatically, completely according to user wishes.
However, the few months of experience with LLM also show that it depends centrally on the question that one asks the AI. These questions or instructions to the LLM are called “prompts”. A separate profession called “Prompt Engineering” is already forming. Whoever masters the art of prompts – detailed instructions – can create entirely new qualities.
Not surprising, really. Every service provider knows the purpose of a good briefing only too well. Without a good briefing, no craftsman can build anything useful.
When designing a questionnaire, and especially when a driver analysis (the analysis of what drives success) is due, the question “what should I ask?” arises. So, for example, what influences a restaurants customer satisfaction? LLMs can help build a list of possible drivers that is comprehensive.
In addition, time can be saved and mistakes avoided during the creation process. Especially LLMs enable even market research beginners to achieve good results.
A technically sound driver analysis is always a causal analysis. This includes the consideration of indirect causal effects and the influence of context variables. The reality is that most market researchers are overwhelmed with setting up a causal analysis. This is another area where LLM can help.
This software NEUSREL, for example, has already announced to make the widespread dream come true: “just upload an SPSS or Excel dataset and a causal driver analysis is ready and the result is written out in complete sentences”.
AI-based text analysis of open-ended mentions has been growing in popularity for a shorter period of time. But categorizing open-ended mentions in a way that is equivalent to manual categorization has so far required manual training of the AI. In addition, the market researcher must define the codebook himself.
Both of these will be made redundant by LLM in the medium term. LLMs already build 90% perfect codebooks in seconds. Without training, LLMs can perform so-called “single shot” coding: i.e. they can accurately tell whether a verbatim belongs to a category without being trained in advance.
A “megatrend” in market research is what I call “synthetic” market research. AI trained on data can predict what another market research study would reveal. This already exists in the field in some areas:
In eye tracking, there are already many solutions that analyze posters or entire commercials and predict with 90% precision how people would actually react to an eye-tracking device.
The same exists in the area of word and sense association. Today we can predict what people associate with certain words, phrases and advertising slogans, whether it fits to a positioning and makes them want to buy – without any questioning at all.
It remains open to what extent LLMs can be used for synthetic market research. Logically, it is obvious that the information is implicitly contained in other information given to the LLM for training.
The LLM will probably only be able to answer a Sunday question by prompting it with all necessary current information.
How the area of the synthetic surveys will develop, remains speculation, that here still much development work is in it. However, all indications are that there will be specialized solutions that can perform synthetic interviews with the help of elaborate prompts or customized training of the LLM.
In this way, LLMs can also be taught additional information individually by the user with so-called “embedded models”. For example, it would be conceivable to give the machine certain current news or social media information and thus enable the machine to answer current questions.
Dashboards are becoming more and more popular and partially replacing PowerPoint decks. What they can’t do is summarize the quintessence of the situation in full sentences. This is exactly what LLMs can now do and thus again partially replace the market researcher.
But LLMs can do more than just speech. They can already generate images and videos today. In the future, AI will be able to deliver a packaging design or advertising poster draft that most closely matches the market research results.
This creates the opportunity to link market research more closely to implementation and thus increase its relevance.
Through LLMs, education (including market research education) will change completely in my opinion. The fact is that almost all professionals working as market researchers have not learned this “profession” and probably would not pass a university exam in market research.
LLMs make the cramming of data and information obsolete. What it takes is a healthy curiosity and problem sets available through application practice. Through a chat conversation with LLMs, learners can acquire core knowledge in a very short time.
This in turn, means that anyone who spends a few weeks immersed in the subject can very quickly become a “market researcher”. LLM brings the democratization of knowledge and education – worldwide at almost ZERO cost.
Sure, there is no guarantee that the LLM speaks the “truth”. But they honestly don’t have that guarantee in textbooks either. Just take two and compare what they say. Furthermore, the learner has it in his own hands to optimize the knowledge quality with good prompting.
Market researchers will always exist. There was the profession of the shoemaker 500 years ago, and it still exists today. At that time, every 100th person was a cobbler, today, it is every 100,000th person.
This will also be the case for market researchers. AI will automate his expertise step by step, and soon, everyone can do his job.
You can choose, row against the current or become the sailor and master of “AI winds” of time.
Each month I share a well-researched standpoint around CX, Insights and Analytics in my newsletter.
+4000 insights professionals read this bi-weekly for a reason.
I’d love you to join.
“It’s short, sweet, and practical.”
