Survivor Explains Sampling

One of the nice parts of distance learning and social distancing has been extra family time. Without commutes, commitments, and the common-chaos, things are kinda quieter. So we’ve been watching Survivor.

I was a huge fan the first season. I was in college, online, and this was new. I kinda grew out of the show, losing touch with the premise, but now with kids that are twelve and ten we are ready.

We’ve watched as a family, working backwards from season thirty-four. Our favorite contestant of season thirty-three was Ken who played a straight version of the game; forming alliances, keeping his word, and winning challenges.

In this case it was the wrong way, as Adam took the final vote. Unanimously.

Ken was liked by all, played well, won challenges, and made it to the final three. What happened?

Two guesses.

Option 1: Survivor is a television show that’s edited a certain way. This is good. A time lapse or documentary or Instagram version of Survivor is worse. Television is a certain medium that excels with a certain message.

The producers know it’s sweeping panoramic of Bali islands, difficult-but-not-impossible challenges that make people at home say I-could-do-that, with some interpersonal drama mixed in. People are edited a certain way so there could have been a lot we didn’t see.

Maybe Ken wasn’t as sharp as he looked. Maybe Adam was even better.

Option 2: A sampling bias. The jury didn’t vote for Ken because they weren’t like him. They were there to play the game a certain way which is what Adam did. The people who want to go on Survivor want to play the game.

Sarah Tavel told Patrick O’Shaughnessy that in the early days of Pinterest there were a group of power users who wanted a specific feature to rearrange their pins. It would take a lot of work, but people really wanted it. So the engineering team built the feature and it largely went unused. What happened?

Sampling bias.

The power users weren’t a good sample.

The same thing was said by Ken Jennings about his run on Jeopardy. Everyone, Jennings said, that makes it to Jeopardy is really smart. That means they compete on something besides smarts. Competing against Ken was really about mastering the buzzer.

In his SSAC talk, Ken said that the producers didn’t know if his run was good or bad. Would this move Jeopardy to, “this is the spirit of the age” or repulse the loyal audience. After watching Ken rip off another week of winnings in a single day, the producers started to let the other contestants have longer buzzer practice. Jennings had mimed and timed the pattern and that was his key to winning.

Samples are fun to think about. With a good selection, a thousand people can explain the world. With a bad selection, and selection is often bad, we get things that may appear one way, but are not.

Want more? Check out this pay-what-you-want placebo prescription pdf.

2 thoughts on “Survivor Explains Sampling”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.