Base Rate Neglect: Why Your Intuitions Are Wrong

A doctor tells you that you've tested positive for a rare disease. The test is 99% accurate. How worried should you be?

Most people say very worried. The test is 99% accurate, after all. But if the disease affects 1 in 10,000 people, the real probability that you actually have the disease is roughly 1%. The other 99% of positive results are false alarms.

This is base rate neglect — our systematic failure to account for how common or rare something is before updating our beliefs with new evidence. It is one of the most pervasive and consequential cognitive biases, and it distorts decisions in medicine, investing, hiring, criminal justice, and everyday life.

If you've read about thinking in probabilities, you already know that our brains are poorly calibrated for risk. Base rate neglect is a specific mechanism behind that miscalibration, and understanding it will sharpen every probabilistic judgment you make.

Let's work through the classic example with real numbers. Suppose:

• A disease affects 1 in 10,000 people (the base rate)
• A screening test is 99% sensitive (correctly identifies 99% of people who have the disease)
• The test has a 1% false positive rate (incorrectly flags 1% of healthy people)

Now imagine we test 1,000,000 people.

People who actually have the disease: 100 (that's 1 in 10,000 of one million)

• Of these, the test correctly identifies 99 as positive

People who don't have the disease: 999,900

• Of these, the test incorrectly flags 9,999 as positive (1% false positive rate)

Total positive results: 99 + 9,999 = 10,098

Probability you actually have the disease given a positive test: 99 / 10,098 = 0.98%

Read that again. A 99% accurate test, and the chance you're actually sick is less than 1%. The base rate of the disease (1 in 10,000) is doing almost all the work. The test's accuracy is almost irrelevant when the condition is rare enough.

This is precisely why mass screening programmes for rare conditions are so problematic. The UK's National Screening Committee explicitly considers base rates when evaluating whether to roll out population-wide tests — because a flood of false positives causes anxiety, unnecessary follow-up procedures, and wasted resources.

This is Bayesian thinking in action: the prior probability (base rate) matters enormously, and no amount of evidence quality can overcome a sufficiently low prior.

Base rate neglect is rampant in investing, and it costs people real money.

Consider the base rates that most investors conveniently ignore:

• Most stocks underperform the index. A 2023 study by Hendrik Bessembinder found that just 2.4% of listed US stocks accounted for all net wealth creation in the stock market since 1926. The majority of individual stocks delivered returns below Treasury bills over their lifetime.
• Most actively managed funds underperform passive indexing. The SPIVA scorecard consistently shows that over a 15-year period, roughly 90% of active large-cap funds trail the S&P 500.
• Most day traders lose money. A study of Brazilian day traders found that 97% of those who persisted for more than 300 days lost money. The median loss was significant.

Yet people routinely pick individual stocks, choose active fund managers, and try day trading. Why? Because they focus on the vivid stories — the friend who bought Tesla early, the fund manager profiled in the Financial Times — while ignoring the base rate of failure.

The base rate for stock-picking success is abysmal. If you're picking individual stocks, you're not competing with average investors — you're competing with quantitative hedge funds, PhD researchers, and algorithms that process information in microseconds. The base rate of outperformance for a retail stock-picker is somewhere in the neighbourhood of 10-20% over a decade, and much of that is luck.

This is directly connected to the concept of expected value. Even if you can identify a stock that will 10x, the expected value of your stock-picking strategy depends on the base rate of your picks being correct multiplied by the payoff. For most people, the expected value of passive indexing is simply higher.

The Kelly Criterion makes this even more explicit: optimal bet sizing depends on your actual edge, and if your edge is zero or negative (as the base rates suggest for most stock-pickers), the Kelly-optimal allocation to individual stock bets is zero.

Startup Survival

The base rates for startups are well-documented and sobering:

• Roughly 90% of startups fail
• Of venture-backed startups, about 75% never return their investors' capital
• Only about 1% of startups achieve a billion-dollar valuation

Yet every founder believes they're in the 10% that will succeed. This is textbook base rate neglect combined with overconfidence bias. When Y Combinator surveys its applicants, the average founder estimates their chance of success at around 60% — six times the actual base rate.

Does this mean nobody should start a company? Not at all. But it means you should calibrate your confidence to the base rate and then adjust based on specific evidence. If you have domain expertise, a proven co-founder, early revenue, and a defensible market position, your odds are genuinely better than 10%. But if you're a first-time founder with an untested idea, your starting estimate should be much closer to the base rate than your gut tells you.

Hiring

Companies routinely fall victim to base rate neglect in hiring. An interviewer spends 45 minutes with a candidate who is articulate, confident, and charming. They come away convinced this person will be a top performer.

But what's the base rate? Research suggests that unstructured interviews predict job performance with a correlation of about 0.20 — barely better than chance. Even structured interviews achieve only about 0.40. The base rate of any single hiring signal being predictive is low.

The implication: rather than trusting your gut feeling from an interview, you should weight the base rate of candidates from similar backgrounds performing well in similar roles, then modestly update based on what you observe. A structured process with multiple independent signals (work samples, references, structured questions) gives you more data points to update away from the base rate — but the base rate should always be your starting point.

Everyday Life

Base rate neglect shows up in mundane decisions too:

• Restaurant choices: That new restaurant has glowing reviews on Instagram. But the base rate of new restaurants surviving past three years is about 40%. The hype might be real — or it might be novelty bias.
• Crime fear: After reading about a burglary in your neighbourhood, you feel unsafe. But the base rate of being burgled in any given year in England and Wales is about 2%. One vivid story shouldn't update your estimate by much.
• Project planning: Your team estimates a project will take 6 weeks. The base rate of similar projects? They typically take 1.5-2x the original estimate. This is the well-documented planning fallacy, which is essentially base rate neglect applied to time estimation.

Kahneman and Tversky identified base rate neglect in the 1970s, and decades of research have clarified why it happens:

1. The representativeness heuristic. We judge probability by how well something matches a mental prototype. If someone 'looks like' a successful entrepreneur (confident, articulate, Stanford MBA), we assign high probability to their success — ignoring that the base rate of startup success is low regardless of how someone presents.

2. Vivid evidence crowds out statistics. A single compelling story (a friend who beat cancer after a positive test, a startup that defied the odds) is psychologically more powerful than dry statistics. Our brains weight narrative evidence far more heavily than base rate data.

3. Base rates are abstract. The number '1 in 10,000' doesn't feel like anything. A positive test result feels like something. We process feelings faster than statistics, and in the absence of deliberate analysis, feelings win.

4. We anchor on the most available number. When told a test is '99% accurate,' that 99% figure becomes an anchor. We adjust insufficiently from it, even when the base rate should dominate our calculation.

5. Ego protection. Accepting base rates means accepting that we're probably average — that our stock picks probably won't beat the market, that our startup probably won't succeed, that our judgement probably isn't as good as we think. Base rate neglect is partly motivated reasoning: we want to believe we're the exception.

The good news is that base rate neglect is one of the more correctable cognitive biases. Here's a practical framework:

Step 1: Always Ask 'What's the Base Rate?'

Before evaluating any specific case, find the base rate. This is your prior probability — the starting point before any specific evidence.

• Considering an investment? What percentage of similar investments have delivered this return?
• Evaluating a job candidate? What percentage of hires from this channel have been top performers?
• Worried about a health risk? What's the actual incidence rate?

Make it a habit to never form a judgement without first establishing the base rate. Write it down. Put a number on it.

Step 2: Use the Bayesian Update Framework

Once you have the base rate, update it with specific evidence — but do so properly. The framework from Bayesian thinking for everyday decisions applies directly:

1. Start with the base rate as your prior
2. Ask: how much more likely is this evidence if my hypothesis is true vs. false?
3. Update proportionally

In the medical test example: start with 1/10,000, then update with the positive test result. The update is significant (the test is 99% accurate), but it only brings you to about 1% — nowhere near the 99% that naive intuition suggests.

Step 3: Use Natural Frequencies Instead of Percentages

Research by Gerd Gigerenzer shows that people reason about base rates much better when information is presented as natural frequencies rather than percentages.

Instead of: 'The test is 99% accurate and the disease prevalence is 0.01%'

Think: 'Out of 10,000 people, 1 has the disease. If we test all 10,000, we'll get 1 true positive and about 100 false positives. So 1 out of 101 positive results is real.'

Natural frequencies make the base rate visible. The numbers become concrete and countable rather than abstract.

Step 4: Keep a Reference Class Library

Build a mental (or physical) library of base rates for domains you care about:

• Startup success rates by stage and sector
• Investment return distributions
• Hiring success rates by channel and signal
• Project completion times vs. estimates
• Conversion rates for your industry

The more base rates you have memorised, the harder it becomes to neglect them.

Step 5: Practise with Calibration Exercises

Calibration training — where you make probabilistic predictions and track your accuracy — is one of the most effective ways to overcome base rate neglect. Over time, you develop better intuitions about prior probabilities because you've been forced to confront the gap between your estimates and reality.

Philip Tetlock's research on superforecasters found that the best predictors were distinguished not by intelligence but by their willingness to start with base rates and update incrementally. They resisted the temptation to anchor on vivid narratives.

Here's the optimistic conclusion: because most people neglect base rates, those who don't have a genuine edge.

In investing, simply accepting the base rate that most stock-picking fails — and indexing accordingly — puts you ahead of the majority of investors over a 20-year period. You don't need to be smarter. You just need to accept the base rate.

In business, founders who honestly assess the base rate of failure and plan accordingly (maintaining runway, testing assumptions cheaply, having a plan B) survive at higher rates than those running on pure optimism.

In medicine, doctors trained in Bayesian reasoning order fewer unnecessary follow-up tests, reducing patient anxiety and healthcare costs.

Base rate thinking is not pessimism. It's calibration. It means starting with reality and adjusting from there, rather than starting with hope and ignoring the evidence. The goal isn't to never take risks — it's to take risks with your eyes open, understanding the true probabilities rather than the ones your intuition fabricates.

As we explore in Thinking in Probabilities, the world rewards those who see it clearly. And seeing clearly starts with base rates.

For a deeper dive into the mathematical foundations of updating beliefs, see our guide to Bayesian Thinking for Everyday Decisions. And for more on how to apply probabilistic reasoning to practical decisions about money and risk, explore Expected Value Explained and The Kelly Criterion.

For further reading on the psychology of judgement and decision-making, many of the best books on probabilistic thinking cover base rate neglect in depth — particularly Kahneman's Thinking, Fast and Slow and Gigerenzer's Reckoning with Risk.