Cognitive Biases in the Wine Industry, Part 2: Confirmation Bias
Cognitive Biases in the Wine Industry, Part 2: Confirmation Bias
This is the second article in a series that focuses on how our brains, which are still wired to deal with Neolithic tasks, often mislead us into making poor business decisions. Though we have learned to gather and utilize facts, data, evidence and logic, we do it poorly for innate, biological reasons. Those of us in the wine industry are not exempted from this.
These human hardware glitches are called cognitive biases and I am writing about them for three reasons. Foremost is that I have been learning about these anachronistic mental processes to improve my decision-making. Second, I see how these biological pitfalls lead to poor decision-making in this industry on a regular basis and hope that, by sharing, others can benefit, too. Finally, I am hoping readers find these posts to be a pleasant break from my more data-driven blog posts, which often seem like inscrutable explorations of numerology. Comments on past posts have often been positive, but have also included these gems:
“That is literally the dorkiest thing I have ever read, I think it has retroactively made me a virgin again.”
“Your wife needs to give you more to do around the house.”
“There you go again, cooking up a big pot of people repellent.”
The Confirmation Bias
The confirmation bias is one of the easiest cognitive biases to understand, since we see it in others every day and, if we have any degree of introspection, recognize that we suffer from it. We humans gather, interpret and recall information selectively to confirm our preconceived beliefs. If we believe that we should plant Chardonnay on our Monterey County vineyard, but decide that first we should make sure that this truly is a good business decision, we will almost invariably go learn what we can, analyze the information and then decide that we are even more sure this is the way to go.
How Confirmation Bias Affects Our Information Gathering
What makes cognitive biases particularly insidious is that they are unintentional, so we have trouble correcting for them. The confirmation bias is rarely detectable, but permeates our most basic information-gathering processes. To understand this, pretend you are playing a game where you need to find out what number is written on the back of a piece of paper and you can ask me questions about it. You guess for whatever reason that I would have put the number 3 on the piece of paper. You will not, therefore, ask me if the number is even, but if the number is odd. This is due to a natural tendency to confirm beliefs, instead of dispelling them. Either question returns the same information in this case, but not in more complex, real-world settings.
When we try to find out if Chardonnay is the right variety for our vineyard, we do the same thing, but this time we get genuinely different answers than we would if we phrased our questions in a more neutral manner. Pretend now that we can ask some all-knowing robot about your planting decision. He will give us answers that are correct, but has no stake in our vineyard and so won’t take the initiative to expound on his beliefs.
We ask, “Robot, should I plant Chardonnay?” The robot answers, “Yes.” But it fails to tell us that we should also plant Pinot Noir to mitigate risk. The robot, of course, is not a robot. It represents the whole universe of information we can gather and its answers represent what we will get from our natural line of questioning.
Let's make the example a bit more concrete. Maybe we ask your neighbors if they think Chardonnay prices will rise and they say yes. That is good news, but, due to the way we phrased the question, they may well not even consider how much Pinot will rise or whether Chardonnay prices will keep up with the pace of inflation or how volatile Chardonnay prices may be. They probably would consider those issues, if we had asked them.
How Confirmation Bias Affects the Way We Interpret Information
We do not limit this bias to our research, but also extend it to our analysis. A slew of scientific studies have shown that if you have two people with opposing beliefs read the same body of evidence, each will claim and believe that the evidence supports his belief. We don’t need to spend much time or money on figuring this one out, though. Just put my family in front of a presidential debate and ask them who won afterwards and you’ll find a strong correlation between perceived candidate performance and previously held viewer beliefs.
And this is not limited to topics that are emotionally important to people. One study had people read a brief story about a theft and then hypothesize as to who committed the theft. Then they were given more information and asked what they believed after reading that. Though everyone received the same information, there was a strong tendency to interpret the second, more informative story as confirming the reader’s initial hypothesis. If we are honest with ourselves, I think few of us could deny that we suffer from this. Don’t feel bad, though, as this is not a sign of stupidity. Studies show that such responses do not vary with measures of intelligence. We are all, it seems, still cavepeople on some level.
It is not very difficult to imagine how this hurts us in the wine business. Imagine a winemaker at a young winery, with a lot of direct sales and small accounts, who is asked by her boss if she should add Cabernet Franc to her product mix. She thinks that this is a poor idea, a distraction from their core portfolio of Rhone blends, but she knows she should look at this as a business decision, do the right research and come back with the right answer. She looks at Nielsen data and sees that Cab Franc is nowhere to be found. She checks the Crush Report and sees that prices are rising and are almost on par with Cabernet Sauvignon. She asks tasting room managers if their Cab Franc is selling and hears that it is. She weighs the facts and thinks, costs are rising, quantitative data indicates no real demand and the only hints that it is doing well are subjective and from a small sample size of people with their own biases. The answer is no, we should not start making Cab Franc.
Meanwhile, the owner looks at the same information. He sees that Cabernet Franc is still not making it into the Nielsen reports, which only cover around 40% of the market, and includes no direct-to-consumer sales channels or small wine shops. But he hears about how well it is doing in tasting rooms. Sounds like the next hot variety. This is confirmed by rising grape prices, which indicate increasing demand down the supply chain. Best of all, it is not quite as expensive as Cabernet Sauvignon, but could still grow into a role as the winery’s high-end, small-production price leader and set the brand apart from the crowd. He decides, wow, this is perfect.
I am sure that most of you have experienced several situations like this. On top of all this, we also remember information selectively, but that bias isn’t as strong as the past two and I don’t think I need to keep beating this horse. The real question is what do we do about it?
Correcting for the Confirmation Bias
I am not a psychologist, but I will share what I am doing to correct for this innate bias.
First, a checklist for gathering data:
Consider what you are really trying to find out and base your questions around this. Are you trying to find out if you should plant Chardonnay? No, your real goal is to figure out what you should plant, period.
When you plan out how to gather data – and you should plan it out, even if you only intend to talk to your neighbors – write out a few representative questions in advance.
When writing the questions, ask if there is a way to phrase them neutrally. If so, start by trying to answer the neutral questions. For instance, ask your neighbor “What do you think I should plant and why?”
Next, see if you can ask the positively-phrased question you originally started with, but in a more neutral manner, like “What do you think about planting Chardonnay?”
Finally, ask in a more negative manner, such as “What do you think the risks are?” or “Do you think there is a better way to do this?”
Next, a set of standards for interpreting data:
Define what you want to measure and what data sources you will use to measure it. In the Cabernet Franc example, you may want to use the NASS acreage numbers to compare planting trends to price trends and get an idea of where your costs will end up. You should decide in advance the relative importance of the Nielsen data and the survey of friends at other wineries.
How are you going to measure the information you gather? Think through the units you will use to measure it. If you are analyzing NASS pricing data, you may want to think of supply-demand dynamics in terms of percent increase in grape price versus percent increase in planted acres. How will you interpret data from tasting rooms? Will you ask friends to rate their bullishness on Cabernet Franc on a scale of 1 to 5 or find out how much they are selling and at what prices?
Come up with standards. What is the bar you are setting for giving the project a go? For the above example, maybe you say that, if you see that 75% of tasting rooms claim to be selling out their Cab Franc inventory in a timely fashion with an average bottle price of over $30 and you expect acreage to rise at least as much over the next 3 years as prices have risen over the past 3 years, then you will go for it.
Don’t let your standards for gathering and interpreting data make the process inflexible. If you find new sources of data or want to take your line of questioning in a different direction, to explore new findings, that’s great. If you genuinely feel that some data is unreliable and should be discarded, do that, but make sure it is in accordance with an objective standard that you can and will apply to all data sources. Most of the time, these things do happen. Just make sure that you’re not secretly trying to justify your position and always put the new processes through the same checklist or set of standards you’re already working off of.
Finally, though I did not go into it at all, to ward off biased recollection of information do what you do in the cellar and on the vineyard. Keep notes on any data gathered and keep those notes well-organized and accessible.
I know this was a long one but, hopefully, it’s been useful and justifies this last minute pitch. If you are making a really important decision, you may want a knowledgeable, unbiased opinion, backed by objective facts, data and evidence. If so, I hope you would consider getting in touch with me.
Listened to while writing this blog:
Do That Guitar Rag, Album by Big Bill Broonzy