DJS Research Ltd

Confirmation Bias, Market Research and Dominic Cummings: Don’t worry, I’m not going to re-visit the recent Dominic Cummings “driving to test my eyes” debacle or debate whether Boris is completely incompetent or “the best prime minister since Churchill” (you can find plenty of this material on social media).

Instead I’d like to briefly look back to those happy times before all this, specifically the Andrew Sabisky debacle.  Sabisky resigned as a Government adviser following criticism of alleged past remarks on pregnancies, eugenics and race. A self-proclaimed “superforecaster”, he was appointed by the aforementioned Dominic Cummings as part of his call for "misfits and weirdos" to apply for jobs in Downing Street.

During the furore, our Dom made a cryptic comment suggesting that the reporters camped outside his house should “listen to superforecasters rather than ignorant pundits”.

What was he on about?

I guess he was referring to Superforecasting: The Art and Science of Prediction; a book by Phil Tetlock, Professor of Psychology and Political Science at the University of Pennsylvania.

I read it a while ago, and one of the most interesting bits was about confirmation bias. Put simply, this is not having an open mind – the concept goes that “experts” and opinionated people often prefer to read articles and accept evidence that support their strongly-held opinions; they are likely to ignore or dismiss evidence that contradicts their view.  Consequently, they don’t tend to make good forecasters, especially when compared to people who take a balanced view based on weighing up all of the available evidence.

There are lessons for us as researchers here. We need to work hard to be impartial and balanced across the research process. We need to work hard to uncover and hammer home the insights that surprise clients, as well as the ones that confirm what they already think (how dispiriting is it when, at the end of a research presentation, a client says "it pretty much confirms what we already knew”?)

There are also lessons for everyday life. It feels like there has been an epidemic of confirmation bias recently - Brexit (sorry to remind you of it) is a good example.

I won’t go in to my opinions on Brexit apart from saying that I was surprised by the outcome. Why? Because the papers that I read and my Facebook bubble of like-minded friends made me confident in what the outcome would be, wrongly.

As market researchers we might benefit from, for example, reading the Guardian Online and the Mail Online (even if it makes us retch a bit) if only to get a balanced view of what people of different persuasions and tribes might be thinking.

Maybe if we all try our best to avoid confirmation bias, we might even be a bit more understanding and tolerant of one other, as well as being better researchers.

Yep, I haven’t really done a great job of explaining what Dominic Cummings was referring to. I should really read his blog to understand his point of view properly. It might also give me some insights into his dress sense and approach to optical care.

And I won’t go into whether a pandemic of confirmation bias has impacted on recent policy and management of Covid-19, you can make up your own mind!