It is not to blame anybody, this is to open our eyes. The “plausibility” superstition has a long history that actually roots in social sciences.
When it comes to applied statistics, still today, students learn to proceed theory-led. It will take another article to clearly prove that this whole research approach is more harmful than helpful in today’s world.
It’s practical to publish scientific papers. But it is not helpful to make a relevant practical impact in real life nor gain genuinely unique, thus valuable insights.
“Nothing is more practical than a good theory” (Kurt Levin) was the professor’s mantra who thought me marketing and statistics. While certainly, the point is valid, it is abused by science and applied researchers.
The problem that we have is NOT that our “good theories” are not used. The problem that we have is that we do not use proper methods to DISCOVER good theories (=insights).
I know. This now violates the existing beliefs of most of you. You are skeptical. That’s fine. Make up your own mind. (And challenge me if needed to write a more in-depth article about this)
Today’s statistical practice of causal modeling is built on this unpractical theory-led approach and works like this:
Collect all hypotheses that are backed up by theory. If it’s just speculation, leave it out from the model. Then test the model with a statistical modeling approach (can range from regression, econometric modeling, to structural equation modeling) to validate the relationships where you already have the theories.
The only meat on the bone you are getting is the linear strength coefficient of the relationships. No wonder that researchers are starving for richer insights.
And they roam to those who promise “richer insights” — the story and fortune-tellers.
The reason why not many are using conventional causal modeling in practice is not mainly because it is clunky and complicated. They don’t use it because it simply validates what you already know it is not that helpful.
Now, if a plausibility check is a numb sword, how can we test theories and potentially new insights?
The 2 types of insights: There are just facts (descriptives) and relationships between facts (cause-effect relations). The latter is what businesses are unknowingly asking for: “What action A will lead to outcome B”.
The art and science to gain those insights can be labeled as “causal modeling”. This talk here discusses this in more detail.