Tag: base rates

Base Rate Neglect(or)

MIKELeave a comment

In Thinking Fast and Slow, Daniel Kahneman writes about the planning fallacy. He’s in a group of professors tasked with writing a textbook. Each proposes a timeline. Each is confident. These are well-established professors after all.

But then Kahneman asks a group member who actually contributed to a textbook: How long did that take? Hmm, he thinks, never less than our longest guess.

It’s a ‘textbook’ planning fallacy. We error to optimism. Michael Mauboussin thinks of a home remodel. The neighbor’s project has delays and cost overruns but ours?

I know this. I’ve written about this for more than seven years. I make this mistake.

Our daughter’s high school is planning choir trips. The possibilities include Pigeon Forge, New York City, and Northeast Ohio.

Ohio, I exclaimed, that really stands out.

Yeah, my oldest said, we might go to Cedar Point.

Cedar Point? That’s awesome! How is that on your teacher’s radar?

Maybe, my wife said, he’s from Ohio or a member of the coaster club.

Oh, I said, If you’re going to Cedar Point I bet he’s a member of a coaster club.

…

I started with my inside view. Another approach is starting with the outside information and shifting from there: Kahneman’s textbook author, Mauboussin’s neighbor.

I grew up in Cedar Point. We know people in coaster clubs. That was my inside information. Upon inspection, it looks like <10,000 people are members of such clubs.

But 16,000-30,000 people leave Ohio for Florida each year.

Thinking in base rates (or like Fermi) prevents my error. Are there more people from Ohio who move to Florida than people in coaster clubs?

Change the framing, change the process.

…

I can still hear the ‘thank you for visiting America’s roller coast’ before clacking up the Magnum hill, feeling excited, enjoying the view and the breeze, marveling that something so tall can be so narrow, and feeling my stomach lift through my torso.

This = that

MIKE1 Comment

A short list of our fast thinking.

Expensive = good. This is many places, one of which is the opening story in Robert Cialdini’s book Influence.

Action = progress. Used well when Headspace ‘embodied’ meditation. Used less well when helicopter parents hover.

The category = the JTBD. For example, the Leatherman was not a knife it was a tool, not a tool but a gadget. Similar, the 3M Command Strips solve the job rather than fill the category.

1×20,000 != 20,000x 1. This is the idea that averages don’t always convey the best information. A small bit of something regularly is not the same as all of something at once. The Credit Karma Save program for instance. Travel too, writes Rory Sutherland is like this.

Argument = dislike. Part of the obstacle to arguing well is our social and evolutionary norms. (Added 3/30/2022)

…

Unrelated, we can update the base rates on March Madness. The cumulative Final Four seedings continues its trend, this year the sum is 13. The first post on March Madness matchups is here.

TTID: Canadian software

MIKE

Our this time is different examples have noted that when the overall system (airline regulation) or when the technology within the system (high jump pads) changes then this time is different. While confirming evidence isn’t a perfect indicator, it’s nice to note:

“If you have a mining business then base rates can tell you something. But over the past ten years there’s a new crop of businesses that have no historical analogs. This sounds like ‘this time is different’ but sometimes it kind of is. These businesses (software SAAS) grow fast, they grow organically. In a couple years they have global reach and no capital requirements. They can click a few buttons on AWS and suddenly they have more servers. They have expensive stocks so they can hire the best engineers, all around the world because everything is remote.” – LibertyRPF, Infinite Loops, October 2021

Often, TTID is used to support a narrative claim, and in general it pays to ‘short the narrative’. But sometimes TTID is right, the world changes. It’s a bit like finding a needle in a haystack (if there even is one).

The base rate for TTID is low. But when systemic rules or new technology allow the job to be done we can look closer.

–
Liberty has a nice Substack.

Will Novak win the most?

MIKELeave a comment

Considering the recency effect of inevitability, practicing with base rates, and reducing the solution space.

In early 2022, the men’s career grand slam standing were tied at twenty wins each between Roger Federer (age 40), Rafael Nadal (35), and Novak Djokovic (34).

For tennis fans, the big question is: Who will end their career with the most? For a while it looked like Novak Djokovic who was the youngest and playing the best. Djokovic had also won three out of the four majors in 2021 and his dominance looked inevitable.

But few things in life are inevitable, no matter how they look in the moment. Sports provides an example for other parts of our life and the Wharton Moneyball crew (in the 1/26/2022 episode) provide a way to think through probabilistic, inevitable, and recency related issues that come in any part of life.

“I was convinced Djokovic was going to end up with the most majors,” Wharton professor Eric Bradlow begins, “but let’s talk about what’s happened in the last six months.” Novak lost the last tournament of 2021 and didn’t play in the first tournament of 2022. Without getting vaccinated, Novak will not play in the French Open (and maybe U.S. Open) either.

But Djokovic not playing also means that someone else, like Nadal can increase his total. Rafael made the semi-finals at WON the Australian and plays best at the French.

“Six months ago I would have said Djokovic would be the top of all time. Now it’s 50/50. I give Nadal a legitimate chance to have the most majors of all time,” Bradlow says.

Tap the brakes, host Cade Massey says, remind us of the base rates. That is, what’s the oldest that someone great tends to win these major tournaments. “Federer hasn’t won one since he was 37,” Eric explains.

Ah. Now we have something to work with. Let’s follow a page from Zeckhauser and simplify.

Djokovic and Nadal each have about 12 chances left. But Novak isn’t vaccinated so subtract the ’22 Australian, ’22 French, and ’22 U.S. Open. “You’ve gone from twelve good chances to nine. And you still have (to play) Nadal at the French!” Bradlow bellows. Plus, it’s not just Nadal and Djokovic but a field of players and like something is always happening, someone unexpected will win.

Rather than overreact to news, we found the base rate (oldest age to win a major) and opportunities left. Though men’s tennis is top heavy, there’s a lot that can happen.

But wait, that’s not all.

“I’m bummed for folks like him (Novak), like Kyrie Irving,” says Massey, “like any high profile athlete, that takes a no vax stance. There is so little room for changing your mind. Once you take that high profile a stance, with the politics of it, it really diminishes the chance of him shifting his position.”

Put another way, a strong public stance creates a restricted action section.

Who will end up with the most majors? We don’t know. But we do know that using base rates, avoiding recency bias, finding simple examples, and not reducing our solution space are all good processes.

…

For football fans, there’s a section of fervor about the Allen-Mahomes game that speaks to the inevitability and our reactions to recent events.

Fire causes

MIKELeave a comment

There’s a few ways to be paid for work. One is monetary. Another is pleasure. A third is knowledge through experience. Framing work this way means someone always ‘gets paid’.

A friend posted on the neighborhood Facebook group that he cleaned his dryer vent and it was full of stuff *after only three years*. ‘Oof, I thought, and ordered a cleaning kit on Amazon. But do I need it?

About one in twenty-five homes has a fire each year. Half of all fires are cooking related and another sixth are from heating equipment. About one in ten are electrical related and 3% of those are due to dryer related fires. So all told, about 0.012 percent of homes in the United States have a dryer related fire each year. That’s an already low base rate, mix in a newer home and the satisfaction I get from cleaning the lint trap and our odds are pretty dang low.

Though unlikely, this was still a good exercise. For one, cooking fires are more common than I thought. One in fifty homes? Each year? The damage may be small, these are fires reported and responded to, but it’s high enough to be a bit more careful in the kitchen. Second, I’ll be paid in experience, one of the best teachers a guy can ask for.

Underdog or tie game?

MIKELeave a comment

It’s important to highlight base rates in the wild as a way to keep them top-of-mind. What’s available is what we think about. Nowhere is this better than sports. We tend to overreact rather than “short the narrative” and base rates are a useful thinking tool to practice.

Talking about the last week of the NFL season and potential playoff situations, otherwise sane Shane Jensen offers up, “I think it’s less likely the Colts lose to the Jaguars than that other game ends in a tie.” Okay, what’s the base rate? Since 2017 there have been 5 games that ended in a tie, and 7 games where a big underdog won.

What’s probably happening is recency bias. The Jaguars lost by forty points to the Patriots, fired their coach mid-season, and maybe aren’t the best run organization. All of those things can be true as well as still being more likely to win than a tie game.

Part of the Wharton Moneyball podcast is to be entertaining. That’s fine. But part of it is to be analytical, and for that this was a miss.

Does the bundle explain it?

MIKE4 Comments

Defaults are a design tool to frame thinking. One designed-default is mean reversion. For most situations, said Cade Massey, “Try regression to the mean on for size and see if that can explain it.” Another is to start with the base rate: what typically happens in situations like this? During the Summer of 2021 there were many comparisons of vaccinated and unvaccinated Covid infection rates. This was a case of base rate neglect.

Mean reversion and base rates are good starting ideas because they prevent our Narrative Spin Drives from jumping into high-output mode. For instance, there’s an annual NFL video game known as Madden NFL. There’s also a Madden curse. If someone appears on the cover they have a terrible season after. It’s happened to eighty-two percent of the athletes!

Or it is base rates and mean reversion. To earn the cover rights, a player must have an excellent season, and their “success equation” benefited from a few lucky bounces. That happens. But bad luck happens too.

To add to the value of starting with base rates and mean reversion we can add “The Bundle”: the idea that a JTBD is a collection of things.

Marc Andreessen talked about the bundle of education: a dating scene, knowledge, social interactions, signaling, potential professional connections, cheap financing, and so on. Part-of-the-reason education innovation hasn’t gained distribution is that online only addresses parts of the bundle. It’s hard to date or build friendships on a video call.

Another bundle is the meal. Every meal is a combo meal: social interactions, nutrients, calories, taste, and so on. We can see bundles further yet. Food is more than the sum of its vitamins and nutrients. Eating an orange is more than theVitamin C, fiber, and sugar.

Work is a bundle too. Economist Tyler Cowen often notes that part-of-the-problem with Universal Basic Income is that it doesn’t address The Bundle. From NPR:

“Companies, like those in the tech industry such as Google and Apple, built enormous offices and put them all right next to each other in Silicon Valley and the office expanded what it was in people’s lives. They became like a second home. They had fancy food, concerts, dry cleaning, free meals.” – Stacey Vanek Smith, Planet Money, August 2021

Okay, a confession. I love Ted Lasso. It’s my favorite show since Parks and Rec. What I admire about Lasso is that he sets a tone (assuming for a moment it’s a real football club but this ethos may exist in the real production). Players begin the day and “Believe”. That’s what starting with base rates, mean reversion, and the bundle does too. Starting with those prompts prevents the Narrative Spin Drive from generating primarily palatable explanations.

–
One thing I’ve changed my mind on is reading fiction. Fiction, like Ted Lasso, appeals to us because it is a fake premise sharing a human truth.
Also, the idea of online education needing distribution is from Alex Rampell, a colleague of Andreessen, who asks: Will disruptors gain innovation before innovators gain disruption? This is the “TiVo Problem.”

…

April 2022 update. Taylor Pearson highlights Kris Abdelmessih’s post.

Apples to apples in Iceland

MIKE

The basic base rate question is: what should I expect in situations like this? Most often we have looked at base rates through the lens of projects. We have an optimistic tendency to think, “yeah but…”. Sometimes it is! Sometimes it’s not.

but it might work for us

The general advice for using base rates has been to start with them, rather than our impressions, and then adapt from there.

Another way to think about base rates is as sampling. It’s important to get the “situations like this” part right, right? This is tricky, and this came up during the summer of 2021 as more and more covid vaccinated people became infected with the covid virus. At one point 67% of Iceland’s cases were among the vaccinated.

“When you look at Iceland and graph out (cases) by who is vaccinated, who is not, and where the cases are, you can see that there are more cases in the vaccinated group than the unvaccinated group.” – Dr. Kat, NPR Planet Money, August 2021

That sounds like the vaccine doesn’t work, or doesn’t work as well, or never-worked?! Maybe, but maybe our conclusions are muddied by an initial assumption that’s wrong.

Rather than jump right to Iceland, let’s pull a Zeckhauser and simplify everything. Imagine in Indiana there is a group of 100 people, half are vaccinated and half are not. In the vaccinated group there are five infections and in the unvaccinated group there are five infections. Putting aside “long-infection”, hospitalization, and death, it-looks-like, in-this-case, that the vaccine is meh.

Okay, now in Nevada there is another group of 100 people. This time there are 90 which are vaccinated and 10 are not. In the vaccinated group there are five infections and in the unvaccinated group there are five infections. Putting aside the same other-factors, in this case the vaccine is doing a lot of work! This was the case in Iceland too. Six of every thousand vaccinated people caught covid while fifteen of every thousand unvaccinated people caught covid. And all of the other-factors were much worse for the unvaccinated group. Vaccination reduced someone’s risk by more than half.

This idea is known as the “base rate fallacy” but really it’s comparing apples to apples which will make the idea stick better anyway(another bit of Zeckhauser advice is to keep explanations simple). BRF is good for talking with economists and behavioral scientists but for implementing this idea it’s an apple-to-apples question a day that will keep the bad decisions at bay.

Creative Operations

MIKE1 Comment

Creativity according to John Cleese is “A way of operating.” This smart 1991 YouTube talk, is full of lightbulb jokes and advice on creativity. How many socialists does it take to change a lightbulb?

The problem with creativity is that it seems difficult. It’s like running a 5K for someone who doesn’t run. Like, c’mon, I can’t do that. Cleese nips this complaint right away and offers two helpful pieces of advice.

First, is to be a designer, and we are all designers. We are all designers because designs influence actions. Some designs tightly constrain action, like this Mario 1-1 walkthrough on YouTube. Other designs constrain loosely.

To design for creativity requires two things: space and time. Set the phone to DND. Sit at the desk. As Steven Pressfield notes, put your ass where your heart wants to be. Like a chef ready for the dinner rush Cleese offers his next piece of advice: think.

Rather he says ‘to play’. That’s the second step. Creativity is the subconscious bubbling up and it’s the conscious shutting up.

“As a general rule, when people become absolutely certain that they know what they’re doing, their creativity plummets.” Jon Cleese

Without interruption, think widely.

This will be hard. Most people, says Cleese, don’t like it. It’s hard to just sit or walk or be. It’s hard to just think. Annie Duke faced this. When she coached poker players they wanted to act, to do, to play the hand. But a lot of poker is not playing. Duke’s challenge was to get players to feel like they were poker players while also making good decisions. So, she reframed the actions.

Rather than playing hands as the action, Duke explained that deciding was the action. Thinking through the hands, the outcomes, the pot odds, the base rates and the game-theory-optimal case was what good players did. That was the secret for being a good poker player. This is the secret too, according to Cleese, for operating creatively.

Creative people are comfortable with the lulls. They understand that the time of play is time working on the problem.

There aren’t good metrics for this. There’s no word count. There’s no investment return. There’s no miles or dollars or calls made. There’s nothing to count which means no numbers which means no comparison which implies no value.

Do not fall into this trip says Cleese. Trust that the moments of wide-open thought matter.
After the play it’s time for work.

How many socialists does it take? Five, but they don’t change it and instead insist that it works.

3 prompts to being Bayesian

MIKELeave a comment

How much should new information matter? Or, is this time different? Because sometimes it’s not.

“Merlin (Heidemanns) said that essentially the polls gain more weight. It’s not that we construct a model and weight the polls. We don’t take a weighted average of the polls, we estimate latent parameters and the polls are data. That said, you can roughly approximate the estimate as a weighted average.”

Andrew Gelman

According to Gelman, priors like “the economy, stupid” never exit the model. According to Nate Silver, the final poll removes all prior data.

In college sports priors matter more than Gelman allows for politics. Nearly two-thirds of college basketball teams who start a season ranked in the top-25, finish in the top-25. College football is the same.

How much new information should matter is a tricky question, but it’s helpful and why the Wharton Moneyball co-hosts encourage each other to become more Bayesian.

At the start of a football season we can guess (or hope) on a team’s chances. With more information, each play, game, and season, we update our idea. Eventually our guess at the start of the year gives way to the information of the year.

It’s hard to do though because we don’t know is this time different. Most of the time it’s not different enough, and base rates work best. But there are three general frameworks which might help us become more Bayesian.

First is to ask a Marc Andreessen like question: is software eating the world? How has the system changed and what does that mean? From FAANG to Testa joining the S&P, it seems like a systemic shift toward technology. Ditto for passing in football.

Second is to ask the Michael Mauboussin question: how much of this was luck? There’s a lot more luck in a single football play than an entire football game. It’s always a mix of skill and luck, but in what ratio?

Third is to consider our identity: am I attached to a position for unacknowledged reasons? This category includes biases like sunk cost and personal influences like ego or status.

The 2021 vaccine rollout is a good instance of practicing Bayesianism. Start with the base rate for vaccines. Watch for evidence. Adjust accordingly.