Tag: sampling

Survivor Games

MIKELeave a comment

December 13 is the Survivor finale and a chance to highlight the different ideas of the blog.

Sampling. These people are not “representative samples“. Survivor hosts a casting call for people with good stories. Like Bob Iger’s big lesson, entertainment isn’t about reporting so much as stories.

Incentives. Like the many games of Jeopardy, Survivor has layers of games. When there are layers of games it’s difficult to judge actions. Is someone trying to win the game or claim later fame?

Business models. Thanks to MTV in 1981 and then Real World in 1992, one entertainment business model is to create value by editing rather than crafting (unlike Seinfeld). Put regular people with backstories (hence sampling) in interesting situations and things will happen. Edit a month of island living to a few dozen hours and viola.

Customer acquisition costs. Sequels – it’s season 43 for Survivor! – have lower CACs. Consumers don’t need to be educated.

Sampling wine

MIKE1 Comment

“Let’s say you want to learn about trade in the ancient world. If you’re using archeology I have good news. We know a lot about trade in olive oil and wine in the Roman world. Olive oil and wine both moved in clay (Amphora) vesicles. They were disposable. When a giant pot of wine reached its endpoint, you poured it into bottles and chucked the big transport pot. Pottery survives, you can shatter it but the pieces are still there.” – Bret Devereaux, EconTalk, August 2021


By Ricardo André Frantz

Grain meanwhile was transported in sacks. “Imagine what parts of your life,” Devereaux asks, “are archeologically visible.”

This example highlights a bigger question: how do I know something is true? Big right? We can’t know everything, so we use samples. And Amphora highlights two caveats.

First, is that easier-to-find data may overstate the case. Covid19 in 2020 had a lot of early data that was easy to collect but not necessarily a good sample. During the first six months we talked a lot about the number of cases, maybe we still do depending on when you read this. Cases are okay, but the best predictors in the early days looked at hospitalizations and deaths. Not to stop there, Covid19 also affected ages quite differently, a fifty year old was fifty times more likely to die than a fifteen year old.

Second, easy data is usually ‘expensive’ because many people use it. If they use it in the same way then, like an auction for a Beanie Baby, prices rise. If the information turns out to be wrong, then prices fall. The heart of moneyball was to find data that was also good, but ‘cheap’ because fewer people used it.

Archeological visibility is a neat analogy to use for thinking about sampling. Though an ancient effect, we can use it still today. The next time we see data, we can ask, is this data more like a clay pot or a cotton sack?

–
The fifty/fifty/fifteen ratio was an estimate based on CDC data. Also, “Today’s persuaders don’t want you to stop and think,” writes Tim Harford in The Data Detective, “They want you to hurry up and feel. Don’t be rushed.”

Survivor Explains Sampling

MIKE2 Comments

One of the nice parts of distance learning and social distancing has been extra family time. Without commutes, commitments, and the common-chaos, things are kinda quieter. So we’ve been watching Survivor.

I was a huge fan the first season. I was in college, online, and this was new. I kinda grew out of the show, losing touch with the premise, but now with kids that are twelve and ten we are ready.

We’ve watched as a family, working backwards from season thirty-four. Our favorite contestant of season thirty-three was Ken who played a straight version of the game; forming alliances, keeping his word, and winning challenges.

In this case it was the wrong way, as Adam took the final vote. Unanimously.

Ken was liked by all, played well, won challenges, and made it to the final three. What happened?

Two guesses.

Option 1: Survivor is a television show that’s edited a certain way. This is good. A time lapse or documentary or Instagram version of Survivor is worse. Television is a certain medium that excels with a certain message.

The producers know it’s sweeping panoramic of Bali islands, difficult-but-not-impossible challenges that make people at home say I-could-do-that, with some interpersonal drama mixed in. People are edited a certain way so there could have been a lot we didn’t see.

Maybe Ken wasn’t as sharp as he looked. Maybe Adam was even better.

Option 2: A sampling bias. The jury didn’t vote for Ken because they weren’t like him. They were there to play the game a certain way which is what Adam did. The people who want to go on Survivor want to play the game.

Sarah Tavel told Patrick O’Shaughnessy that in the early days of Pinterest there were a group of power users who wanted a specific feature to rearrange their pins. It would take a lot of work, but people really wanted it. So the engineering team built the feature and it largely went unused. What happened?

Sampling bias.

The power users weren’t a good sample.

The same thing was said by Ken Jennings about his run on Jeopardy. Everyone, Jennings said, that makes it to Jeopardy is really smart. That means they compete on something besides smarts. Competing against Ken was really about mastering the buzzer.

In his SSAC talk, Ken said that the producers didn’t know if his run was good or bad. Would this move Jeopardy to, “this is the spirit of the age” or repulse the loyal audience. After watching Ken rip off another week of winnings in a single day, the producers started to let the other contestants have longer buzzer practice. Jennings had mimed and timed the pattern and that was his key to winning.

Samples are fun to think about. With a good selection, a thousand people can explain the world. With a bad selection, and selection is often bad, we get things that may appear one way, but are not.

Want more? Check out this pay-what-you-want placebo prescription pdf.