The Prosecutor's Fallacy: How Courts Get Statistics Wrong

The prosecutor's fallacy is the most expensive arithmetic error in the history of courtroom statistics. It happens when an expert witness — or a prosecutor reading from one — quotes the chance of seeing some piece of evidence given that the defendant is innocent, then lets the jury hear it as the chance the defendant is innocent given the evidence. Those two numbers can differ by a factor of millions. Innocent people have gone to prison because of it.

The prosecutor's fallacy is the mistake of confusing P(E | I) with P(I | E), where E is the evidence and I is the proposition that the defendant is innocent. They are not the same number, they are usually wildly different, and confusing them is the same arithmetic error you'd make if you confused 'the chance of a positive test given you have the disease' with 'the chance you have the disease given a positive test'. We covered this inversion in detail in our conditional probability explainer, but the courtroom version of the mistake has its own name because the stakes are uniquely high.

A typical setup looks like this. A forensic scientist testifies that the probability of finding this particular DNA profile, or this combination of fibres, or this rare blood-type pattern in a randomly chosen person from the population is one in a million. The prosecutor then summarises: 'so there is only a one-in-a-million chance the defendant is innocent.' That summary is the fallacy. The one-in-a-million number describes how often the evidence appears among innocent people, not how often a defendant who shows the evidence is innocent.

Imagine a city of one million people. A crime is committed and the police find a partial fingerprint at the scene. A forensic expert testifies that the fingerprint pattern occurs in roughly one person in ten thousand. The defendant matches.

The prosecutor's framing: 'There is a one-in-ten-thousand chance an innocent person would match. Therefore the chance this defendant is innocent is one in ten thousand.' That sounds compelling.

But run the numbers. In a population of one million, a one-in-ten-thousand pattern matches roughly 100 people. The defendant is one of them. Without any additional evidence pointing to this specific person, the chance they are the one true culprit is closer to one in a hundred — not one in ten thousand. The prosecutor's number was off by a factor of a hundred. In real cases, the factor is often a factor of millions.

The correct number depends on the base rate: how plausible is it, before this evidence, that any given person in the suspect pool is the perpetrator? If the prior is one-in-a-million, even a one-in-ten-thousand match leaves the posterior probability of guilt around one percent. Base-rate neglect is the deeper bias under the prosecutor's fallacy — the courtroom is just the highest-stakes venue.

The most cited example of the prosecutor's fallacy in modern legal history is the case of Sally Clark, a Cheshire solicitor convicted in 1999 of murdering her two infant sons. Both children had died suddenly in their cots. Clark maintained both deaths were Sudden Infant Death Syndrome (SIDS).

At trial, the paediatrician Sir Roy Meadow testified that the chance of two SIDS deaths occurring in a single affluent, non-smoking family was one in 73 million. He arrived at the number by squaring an estimate that the chance of one SIDS death in such a family was about one in 8,543. The squaring assumed the two deaths were independent events.

That assumption was wrong — siblings share genes and home environment, so a second cot death is more likely after a first, not the same probability as a fresh draw. But the squaring was only the smaller error. The larger error was the prosecutor's fallacy: even if the chance of two unrelated SIDS deaths really were one in 73 million, that is the probability of seeing the evidence given innocence. It is not the probability of innocence given the evidence. To compute the second number you would need to compare it against the prior probability of two infanticides, which is also extremely rare. When statisticians did that calculation properly, the posterior probability that Clark was innocent was substantially higher than the prosecution's framing suggested.

The Royal Statistical Society publicly criticised the testimony in 2001, noting it had 'no statistical basis'. Clark's conviction was overturned on appeal in 2003 after she had served more than three years in prison. She died in 2007. Two further wrongful convictions of mothers (Angela Cannings, Donna Anthony) were quashed on similar grounds.

Long before Sally Clark, the California Supreme Court reversed a robbery conviction in People v. Collins (1968) for the same arithmetic mistake. A witness in the original trial described the assailants as a Black man with a beard and moustache and a white woman with a blonde ponytail, leaving the scene in a yellow car. A prosecution expert assigned probabilities to each of these features in the general population — yellow car: 1 in 10; man with moustache: 1 in 4; girl with ponytail: 1 in 10; girl with blonde hair: 1 in 3; Black man with beard: 1 in 10; interracial couple in car: 1 in 1,000 — and multiplied them together to get one in twelve million.

Two errors stacked on top of each other. The features were not independent (a man with a beard is much more likely to also have a moustache), so the multiplication was invalid. And the resulting one-in-twelve-million figure described how rarely a random couple would match the description, not how unlikely it was that an innocent couple matching the description had been arrested. In a population of millions of couples, even a one-in-twelve-million combination would on average appear in several couples. The Supreme Court of California reversed the conviction, calling the testimony 'a veritable sorcerer' in the courtroom and warning future prosecutors against using probability without grounding it in base rates.

Modern DNA evidence has not made the fallacy go away — if anything, the larger numbers make it more dangerous, because juries are more likely to take a one-in-a-billion figure as case-closed.

The number forensic labs report is the random match probability (RMP): given a particular DNA profile, what is the chance a randomly chosen unrelated person would match. A typical autosomal STR profile RMP in casework is around one in a billion. The prosecutor's fallacy is to translate that as 'one in a billion chance the defendant is innocent.'

Two things make that translation wrong. First, the suspect pool is rarely a random sample of the global population — close relatives have far higher match probabilities, and forensic databases of millions of profiles produce expected matches purely by chance (this is sometimes called the database search fallacy). Second, RMPs say nothing about laboratory error, contamination, or sample mix-ups, which on a per-case basis can be far higher than one in a billion. The 1990s case of Andrew Deen in the UK was overturned partly because the Court of Appeal accepted the trial judge had effectively encouraged the jury into the prosecutor's fallacy on a one-in-three-million RMP.

The proper way to present DNA evidence — endorsed by the Royal Statistical Society and the National Research Council in the US — is as a likelihood ratio in a Bayesian framework, never as a direct posterior probability of guilt.

The prosecutor's fallacy has a mirror twin known as the defendant's fallacy (or the defence attorney's fallacy). It runs the inversion the other way: the rare evidence narrows the field to a small group, but it doesn't pinpoint the defendant, so it shouldn't increase our belief in guilt at all.

The most cited example is from O.J. Simpson's 1995 murder trial. Simpson had a documented history of domestic violence against Nicole Brown Simpson. The defence pointed out that of all men who batter their wives, only a tiny fraction (something like 1 in 2,500) go on to murder them. Therefore, the argument went, the history of abuse was almost no evidence of guilt at all.

That reasoning ignores the conditioning. The relevant question is not 'given that he was a batterer, what is the chance he was a murderer?', it is 'given that he was a batterer and his wife was murdered, what is the chance he was the murderer?'. The statistician I.J. Good showed in Nature that conditioning on the murder having happened, the prior history of domestic violence raises the probability the husband is the killer to roughly 80 percent — a strong piece of evidence, not a negligible one. Both sides can mishandle conditional probability; the courtroom rewards whichever framing the jury hears more clearly.

The correct conversion uses Bayes' theorem. We covered the full mechanics in Bayesian thinking for everyday decisions, but the courtroom-specific version is short.

Let G be the proposition the defendant is guilty and E the evidence. Bayes says:

P(G | E) = P(E | G) × P(G) / P(E)

The key term the jury is rarely given is P(G), the prior probability of guilt before the new evidence. In a city of a million people with no other information about who committed the crime, P(G) for any single person is around one in a million. P(E) can be expanded as P(E | G) × P(G) + P(E | not G) × P(not G).

For a one-in-ten-thousand match in a city of a million: P(E | G) is roughly 1 (the perpetrator obviously matches their own evidence), P(G) is 1 in a million, P(E | not G) is 1 in 10,000, and P(not G) is essentially 1. Working it through gives P(G | E) ≈ 0.01 — about one percent — not the prosecutor's framing of 99.99% guilty.

The same calculation done with the Sally Clark numbers, conservatively assigning a prior probability of double infanticide and a prior probability of double SIDS, yields a posterior probability of innocence many orders of magnitude higher than the one-in-73-million figure the jury heard.

You don't need to be a statistician to catch the fallacy in the wild. A small checklist suffices.

1. Which way is the conditioning running? If the number is P(evidence | something), ask what the something is. If it's 'innocence' or 'random chance', the speaker has not yet given you a probability of guilt.
2. Is there a stated prior? Without a prior, a likelihood cannot be converted into a posterior. Listen for any mention of base rates, suspect-pool size, or how the defendant came to be a suspect at all.
3. Is the evidence one-in-N independent of itself? Multiplying probabilities for several features (eye colour, blood type, fibre match) is only valid if those features are statistically independent. Most of the time they aren't, especially for hereditary or culturally correlated traits.
4. Is database trawling involved? If the suspect was identified by searching a database of a million profiles for a match, the relevant chance is not the per-person RMP but the chance of any match in a database that size — usually orders of magnitude higher.
5. Has anyone considered laboratory error? A one-in-a-billion theoretical match is irrelevant if the lab's per-sample mix-up rate is one in a thousand. The dominant error usually wins.

The prosecutor's fallacy is just the highest-profile version of a much wider failure mode. Whenever you read that 'a 99% accurate test for a one-in-a-thousand condition came back positive,' the headline-grabbing number is P(positive | sick), not P(sick | positive). The two are connected by base rates and Bayes. We've worked through the medical-test version of this trap in the false positive paradox.

The same shape appears in airport security alarms, fraud detection, polygraph testing, performance reviews, school admissions tests, hiring screens, and any system that flags rare events with imperfect signals. The pattern is always: an impressive sounding probability conditioned on the wrong thing, treated as if it were conditioned on the thing the audience cares about. Once you see the pattern you cannot un-see it.

The prosecutor's fallacy is one swap of two letters — the difference between P(E | I) and P(I | E) — but the consequences have ranged from overturned convictions to years of wrongful imprisonment. The fix is not advanced statistics; it is the discipline to ask, every time you hear a courtroom probability, which way is the conditioning running, and what is the prior. Once you have those two numbers, Bayes does the rest. Without them, no impressive-sounding ratio tells you anything about whether the defendant did it.