How much should new information matter? Or, is this time different? Because sometimes it’s not.
“Merlin (Heidemanns) said that essentially the polls gain more weight. It’s not that we construct a model and weight the polls. We don’t take a weighted average of the polls, we estimate latent parameters and the polls are data. That said, you can roughly approximate the estimate as a weighted average.”
Andrew Gelman
According to Gelman, priors like “the economy, stupid” never exit the model. According to Nate Silver, the final poll removes all prior data.
In college sports priors matter more than Gelman allows for politics. Nearly two-thirds of college basketball teams who start a season ranked in the top-25, finish in the top-25. College football is the same.
How much new information should matter is a tricky question, but it’s helpful and why the Wharton Moneyball co-hosts encourage each other to become more Bayesian.
At the start of a football season we can guess (or hope) on a team’s chances. With more information, each play, game, and season, we update our idea. Eventually our guess at the start of the year gives way to the information of the year.
It’s hard to do though because we don’t know is this time different. Most of the time it’s not different enough, and base rates work best. But there are three general frameworks which might help us become more Bayesian.
First is to ask a Marc Andreessen like question: is software eating the world? How has the system changed and what does that mean? From FAANG to Testa joining the S&P, it seems like a systemic shift toward technology. Ditto for passing in football.
Second is to ask the Michael Mauboussin question: how much of this was luck? There’s a lot more luck in a single football play than an entire football game. It’s always a mix of skill and luck, but in what ratio?
Third is to consider our identity: am I attached to a position for unacknowledged reasons? This category includes biases like sunk cost and personal influences like ego or status.
The 2021 vaccine rollout is a good instance of practicing Bayesianism. Start with the base rate for vaccines. Watch for evidence. Adjust accordingly.
The Waiter's Pad 