Cognitive biases
Base rate neglect: why a vivid description misleads us
Alice is on a train. Across from her sits a quiet man. He neatly arranges his things on the little table and spends the whole evening reading a thick book. They get talking, and he offers a riddle: “Guess where I work: in a library or as a salesperson in a store?” Alice has almost no doubt: “In a library, of course.”
The answer seems obvious. But Alice did not think about one thing: there are far more salespeople around than librarians. Even among quiet book lovers there may be more salespeople, simply because there are so many more of them.
This is base rate neglect: when we estimate a probability, we look at the vivid details of a case and give too little weight to how common such a thing is in general. Below, we look at what experiments showed, why the description outweighs the numbers, and when people do get it right. In the middle of the article you will help Alice with the riddle yourself.
Contents
In short: how base rate neglect works
What a base rate is
A base rate is how common something is in general, before any details. How many librarians and how many salespeople there are in a city. How often a rare disease occurs. How many new cafés survive to a second year.
When you need to guess which group a person or case belongs to, a good estimate has two parts. The first is how well the description matches the group. The second is how big the group itself is. We usually have a good feel for the first and remember the second poorly. If one group is many times larger than another, even a very “librarian-like” description may belong more often to a salesperson.
Try it yourself: the fellow passenger’s riddle
This takes a couple of minutes. Here is what will happen.
Step 1. You read a short description of Alice’s fellow passenger and use a slider to mark what you think the chance is that he works in a library rather than as a salesperson in a store.
Step 2. We show you something that may help with the answer. What exactly, you will find out in the game. After that you can answer once more.
At the end we compare both of your answers with what you get when you do the math. The city and the numbers in the game are made up for illustration; this is not real statistics. All players see the same thing. The sliders start out empty so as not to hint at any number.
What usually happens. The description “quiet, neat, loves books” pulls the answer toward the library, and the first estimate may come out high. When you can see how many salespeople there are in the city, the answer often shifts down. If you set a low chance right away, maybe you knew this puzzle or had read the explanation above. Then the game can’t show the effect on you. But in real life nobody warns you: “Don’t forget how many salespeople there are around.”
What experiments showed
A similar problem about engineers and lawyers was devised by the psychologists Daniel Kahneman and Amos Tversky. They described it in a 1973 paper on how people make predictions, and then retold it briefly in a well-known 1974 review. We could not read the 1973 paper itself, so we give the details of the experiment as that review retells them.
Participants were given short descriptions of people and told that each person had been picked at random from a group of 100 that included engineers and lawyers. They had to estimate the chance that the person was an engineer. Some participants were told the group had 70 engineers and 30 lawyers. Others were told the opposite: 30 engineers and 70 lawyers.
By the rules of probability, the answers in the two groups should have differed noticeably. But participants gave almost the same estimates: they looked at how much the description resembled an engineer and barely looked at the makeup of the group.
The most interesting thing happened with a description that said nothing about the profession, something like “married, capable, liked by colleagues.” The right answer here is simply the share of engineers in the group. But participants put it at 50-50 in both groups. When there was no description at all, they correctly said 70% and 30%. The authors concluded that people behave differently when there is no information and when the information is useless. A useless description seems to shut out the big picture.
Why the description outweighs the numbers
We judge by resemblance. Kahneman and Tversky explained it this way: we judge probability by how much a case resembles a familiar image. The fellow passenger’s description looks very much like a librarian. And resemblance is not affected in any way by how many librarians exist. A neighboring rule is the availability heuristic: there we judge how common something is by how easily examples come to mind.
A general number feels “not about this person.” The psychologist Maya Bar-Hillel suggested in her paper that we sort information by how directly it concerns this very case. A description of a specific person seems important, while the share in the population seems like a random general number. So the second gives way to the first. This is her explanation; we could not read the paper itself.
But we don’t always forget about frequency. The researcher Jonathan Koehler gathered many such experiments in a large review. His conclusion: people almost always take the base rate into account somehow, it is just often not enough. And a lot depends on how the problem is presented. The base rate is used better when people learn it from their own experience or see it as “how many out of how many,” rather than in percentages.
Other experiments confirm this. When similar problems were given not in percentages and probabilities but as “so many people out of so many,” there were noticeably more correct answers. That is why our game shows the city as people rather than percentages.
Where it gets in the way
Judging people. “He’s quiet and wears glasses, so he must be a programmer.” “She’s bright and loud, so she must be an actress.” The description fits the image, and we forget to ask how many people of that profession there are in general. This is how confident but often wrong guesses about strangers are born. This is an everyday observation.
Medical tests and screenings. Say a screening for a rare disease came back with a worrying result. It seems the matter is almost settled. But the logic here is the same as in the fellow passenger’s riddle. No screening is infallible: sometimes it also triggers in healthy people. If the disease is rare, healthy people for whom the screening was wrong may outnumber the sick. The rarer the disease, the more often such a result turns out to be a false alarm. This does not mean a worrying result can be ignored. What it means in a specific case is for a doctor to explain; here we only show why a first impression can be too frightening.
News and rare events. A vivid story about a rare event sticks in memory, and it seems that such things happen all the time. The question “How often does this really happen?” rarely comes to mind at such a moment. This is an everyday observation.
Work and plans. A new project feels special, and we don’t ask how similar projects usually end. There is more on this in the article on the planning fallacy: there, too, Kahneman and Tversky suggested taking into account how similar undertakings usually end.
How not to fall for it
Base rate neglect is one of the biases on the cognitive bias map, in the section on what we notice. Next to it is confirmation bias: once we decide “he’s a librarian,” we more readily notice whatever agrees with that.
Frequently asked questions
So a description of a person is worth nothing?
It is worth something. A good estimate takes into account both the description and how common such cases are. The description shifts the chance, but you should start from the overall frequency. The bigger the difference in group sizes, the stronger the description has to be to outweigh it.
Is it true that people don’t look at frequency at all?
No. That is a common exaggerated version. According to a review of many experiments, people almost always take frequency into account somehow, but less than they should. And they do much better when the problem is presented as “how many out of how many.”
Does a worrying test result always mean a false alarm?
No. The article only says that when a disease is rare, a worrying result from an imperfect screening is more often a false alarm than it first seems. What it means in your case is for a doctor to explain.
