Base Rate Fallacy
Imagine a Townsville Policeman has developed a youth criminal detector that we shall call the YCD. The YCD is so advanced that just by taking a saliva sample it can tell if youths aged 10-24 years old are either a criminal or not. Now let’s say the YCD has a 5% false-positive rating amongst youths who are not criminals (say’s they are a criminal when they are not). However, the YCD never fails to detect a true criminal youth. One in a thousand youths are criminals. Suppose then the policeman stops a random youth and forces them to take the YCD. The YCD indicates that the youth is a criminal. How high is the probability that youth is really a criminal?
Many would answer as high as 95%, but the correct probability is about 2%
An explanation for this is as follows; on average, for every 100 youths tested:
- 1 youth is a criminal, and it is 100% certain that for that individual there is a true positive test result, so there is 1 true positive test result
- 999 youth are not criminals, and among those youths there are 5% false positive test results, so there are 49.95 false positive test results
- Therefore, we have a total of 50.95 positive results for the YCD, with 49.95 results being false positive
- Resulting in the probability being calculated as (1/50.95) x 100 = 1.96%
The above example is adapted from a Wikipedia article on the base rate fallacy, (https://en.wikipedia.org/wiki/Base_rate_fallacy)
The base rate fallacy is to ignore base rate information (one in a thousand) and infer a conclusion based on specific cases. For the case above, everybody living in North Queensland has been overwhelmed with news reports on a high prevalence of youth crime in the Townsville region for the last 3 years. Therefore, we often judge the likelihood to be higher that more youths are criminals in Townsville and disregard the base rate information. Now imagine if I had stated Aboriginal Australian youths, would it be likely that we would have estimated the probability to be higher?
The above rates are not to far off actual figures as of 2015 the population for the age group of 10-24 years was estimated at 52,275 in the Townsville region (ABS, 2017). Figures for robberies in 2017 for that age group were estimated at 44 by the Townsville Police (ABC, 2017). By ignoring base rates, we are left to make judgments on an individual case basis and this leads to erroneous conclusions and errors in decision making. Therefore, research, census data and surveying populations is critical for us to gain a true understanding of the likelihood of events.
This is a major cognitive error that leads onto many other ares, particularly the representativeness heuristic which we will discuss more in coming posts. The major issue that comes from this error is the conclusion that human probabilistic thinking is fundamentally flawed.
ABS, (2017) http://stat.abs.gov.au/itt/r.jsp?RegionSummary®ion=318&dataset=ABS_REGIONAL_ASGS&geoconcept=REGION&datasetASGS=ABS_REGIONAL_ASGS&datasetLGA=ABS_NRP9_LGA®ionLGA=REGION®ionASGS=REGION