Simpson’s Paradox

Alice farms carrots and corn. She plants 10 carrots (harvests 90%) and 100 ears of corn (harvests 75%).

Bob farms carrots and corn. He plants 1,000 carrots (harvests 85%) and 100 ears of corn (harvests 70%).

Though Bob harvests a lower percent of carrots and corn, than Alice, his total harvest is higher. This is Simpson’s Paradox.

	Alice	Bob
Carrots	9/10 = 90%	850/1000 = 85%
Corn	75/100 = 75%	70/100 = 70%
Total	84/110 = 76%	920/1100 =84%

Wikipedia has examples of Simpson’s Paradox: UC Berkeley gender bias and batting averages, but it’s farming that grows my insight. Picture Alice with her backyard garden. She has a two acre lot. There’s a house, a shed, and maybe a pond. For her 110 seeds of carrots and corn Alice probably has some raised beds. Now picture Bob’s homestead with a thousand carrot seeds.

Another example.

There are two girls in the same high school, Kim and Abby. They take English 101, but with different teachers. Kim does well, earning 88/100 on the homework portion and 80/100 on the exam portion of the class, and Kim’s final grade is 84%.

Abby’s teacher assigns a lot(!) more homework. She earns 860/1000 on the homework portion and 75/100 on the exams, and Abby’s final grade is 85%. Kim did better on each section, but Abby’s final grade was higher.

…

Okay. Put a pin in that paradox and consider this:

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.

Which is more probable: Linda is a bank teller [or] Linda is a bank teller and active in the feminist movement?

Okay. Last one.

There are 100 persons who fit the description above (that is, Linda’s). How many of them are:

• Bank tellers? [___] of 100
• Bank tellers and active in the feminist movement? [___] of 100.

…

What we’ve done is reframe problems. We’ve changed the story around the numbers and our understanding.

Simpson’s paradox is easier to understand thinking geometrically in terms of farm space or using familiar examples like school. Linda, via the conjunction fallacy, is easier to understand in percentages rather than absolutes. Rory Sutherland suggests solving life’s problems like sudoku puzzles. Look at it from one direction but if that doesn’t work try another. Use your current experience to find a future answer.