Here are the 2021 predictions graded. My average Brier Score was 0.227 whereas a coin tosser Brier is 0.25. A perfectly accurate forecaster scores 0 and perfectly inaccurate forecaster is 1. The big misses will be in bold.
My guesses are blue, outcomes are red. The closer the blue is to the red the more accurate I was. This chart makes the predictions looks okay.
But when I bucket them it gets worse.
That looks terrible. Let’s see how.
Hurricanes. NOAA “An average season has 12 named storms, six hurricanes, and three major hurricanes.” Will there be more than 12 named storms? Yes, 90%. Will there be more than 30 named storms (the 2020 record)? Yes, 25% Will there be 3 or more major hurricanes (top winds of 111+mph)? Yes, 60% Will I lose power at my home in Central Florida for more than 3 days? Yes, 10%.
Overall there were 21 named storms and four of which were major and we didn’t lose power at all this summer. Mostly these were safe but relevant predictions. Hurricanes are one of those things I’d like to appreciate correctly. Like an alligator on the river bank, I want to know enough to take a good photo but also to keep my fingers.
Blog.
Will this blog have more than 41,000 views in 2021 (41k is the 2020 number)? Yes, 15% Will this blog have more than 800 posts by year end? Yes, 30%
The blog had 29,000 views and 908 blog posts and a whopping 335,000 lifetime views (!!). What’s surprising was my inability to predict myself. As things go, I got into a great writing streak in the summer and fall of 2021 and the posts reflected that.
Finance
BTC will top 75,000 at any point in the year? Yes, 10% BTC will be under 30,000 at any point in the year (started 2020 at this point)? Yes, 20% ETH will top 5,000 at any point in the year? Yes 5% ETH will be under 130 at any point in the year (started 2020 at this point)? Yes, 20% BRKB will top 275 at any point in the year? Yes 10% BRKB will be under 234 at any point in the year (started 2020 at this point)? Yes, 30%
First, in hindsight this was cheating. ‘At any point in the year‘ is a case of something is always happening. To get better at predicting I need to ask better questions. The biggest paired misses of the year were my guesses about Berkshire Hathaway stock ($BRKB). I thought there was a 10% chance of the stock being under 234 and 30% being over 275 (and that’s with the generous ‘at any point’ language).
I’m not quite sure what to think of this one. Robinhood and retail? The stock didn’t move more than 20-40 points in 2018 or 2019 and as a ‘value’ stock I expected tighter growth. Though mostly correct on crypto, with hindsight the range of outcomes is much wider.
Economic Recovery These will be graded per Bill McBride’s numbers on Calculate Risk. Any single day of the last week of the year will top 2M travelers (2019 was 2.0-2.5M)? Yes, 75% Open Table reservations will be down less than 10% YOY? Yes, 75% Open Table reservations will be positive YOY? Yes, 20% Any movie earns more than 250M on opening weekend during the year (highest grossing movies)? Yes, 5% Hotel occupancy tops 60% (graph)? Yes 80% Hotel occupancy tops 70%? Yes, 60%
Mostly got these correct and directionally too. Predicted 75, 75, 80, and 60% for events that all happened. Missed reservations being positive but grading it I can’t remember what this meant. Also missed on movie opening as Spider Man No Way Home earned 270M, partly from my two daughters and me.
Sports
As of year-end, Tom Brady averages +270 ypg? Yes, 30% The Lakers are NBA champions? Yes, 25%
Tom Brady continues to amaze averaging 313 yards per game. And our Aaron Rodgers saga comes to an end, with Rodgers just falling short by 1.5 touchdowns. This one is another head scratcher. Rodgers had two games with no touchdowns and missed a game due to Covid. Is this the variance that Plus EV noted or was it just luck?
In Average is Over Tyler Cowen predicts that future jobs will reward people who work well with machines and humans. There will be good careers for people who understand people and data.
For instance, a doctor spends years of her life in medical school and residency and attends continuing medical education courses to ‘know a thing’. But she also must convince her patients. The way we do each of these will change with time but these are the two parts any job. Put another way, it doesn’t matter how good the model is if people don’t follow it.
“The (diabetes treatment) model we created beats 95% of primary care physicians, not because they aren’t smart, but they don’t go through the six million (treatment) combinations in their head. For endocrinologists there’s a top quintile who get results as good as the best output of our algorithm. They don’t do it by choosing the best algorithm, they use their humanity to talk to their patients about adhering to the drug regime. They are getting results a different way.” – Len Testa, Causal Inference, October 2021
Good convincing outperforms better medicine! This is why financial education does not work. Action does not follow information like a tail follows a dog.
Testa doesn’t elaborate how the doctors describe the diabetes deterrents, but it’s probably in the listener’s language. “Excise the statistical jargon,” said David Spiegelhalter and communicate better.
In the first post we jumped off with the idea that the S&P500 is an unbalanced collection of stocks. The top five companies, noted Carl Kawaja, generate 22% of the earnings, so why not only ‘bet on’ the best?
This led to thinking about predictability. Chess is easier to predict than my morning pickleball game. Michael Mauboussin wrote a wonderful book about predicability as framed by skill and luck.
Think about events, Mauboussin explained, as a continuum: from the least skill activities (roulette) to the more skill components (stock markets, hockey, football) to the mostly skill components (basketball, chess). An event’s predictability slides as the skill portion of an outcome increases.
Here is how sport predictions look.
On the Wharton Moneyball podcast the hosts often talk about betting the favorites or the field. Bettors in events toward the left of the graph are better off betting favorites and events toward the right taking the field.(1)
With two years of data (sorta) it seems that some sports are more predictable (more skill, less luck) than others. Here’s how Mauboussin ranked sports, starting with most skill based: NBA, EPL, MLB, NFL, NHL.
Contrast that with our (sorta) model: EPL, NBA, MLB, NHL, NFL. That’s a pretty good match!
How to use this: to consider if our predictions are more like the NBA or more like the NFL and it’s “any given Sunday” ethos. In NBA-like situations there are a few big issues (players) that drive the results. In NFL-like situations there are more chances for odd bounces (fumbles), subjective decisions (pass interference), or fortuitous circumstances (turnover-worthy pass attempts).
Nothing in real life will be like these sports, but to have a good analogy is a good decision making tool.
–
(1) “Buying good things can’t be the secret to success in investing,” wrote Howard Marks), “It has to be the price you pay. It’s not what you buy, it’s what you pay. There’s no asset so good it can’t become overpriced.”
What are the odds of more than twelve named Atlantic hurricanes in 2021? That Bitcoin will top 75K in 2021? More than 2M travelers through TSA in a single day?
These, and more, were part of the 2021 predictions. That post built on the ideas of Superforecasting, which offers ideas towards better predictions. Julia Galef adds another.
Here’s one of the prompts: Will I lose power at my home in Central Florida for more than three days? I figure these odds were about 10%, and would wager that, no, we will not lose power for more than three days.
Imagine another prompt. In this bag of twenty balls there is one red and nineteen black, pick the red one and win. Okay, simple enough. There’s a five percent chance to win the red. And here’s Galef’s guise, you can only play one game.
Do I feel more confident about the hurricanes making landfall or the finding of the red ball? The hypothetical ball bag bet can slide up or down: 5%, 10%, 30%, etc.
“You just ask yourself, do I feel more optimistic about taking that bet or [the other]…You can play with the ratio of balls to kind of narrow down the number you put on your confidence in the original question.” – Julia Galef, BBC’s More or Less, August 2021
With my kids we use coin flipping. One day I had an appointment and told them there was a 5% chance they would have to go to after care at the school. “That’s like flipping a coin and landing on heads four times in a row” I told them.
Probabilistic thinking is difficult but it can be helping in making good decisions. Poker’s appeal highlights this idea too.
–
The TSA’s nadir was 87,000 travelers the week of April 13 2020, down 95% from the same week in 2019. In 2021 that number was more than one million. The week of June 11, 2021 there were more than two million travelers. I guessed there was a seventy-five percent chance that would happen.
For each thing that happens there is a field of potential things which could happen. Those potential events fill out a distribution where some events are more likely than others. A daughter’s height for instance, could be between four and eight feet but it’s very very likely that her height will be between her mother’s and father’s heights.
Thinking about these distributions of potential outcomes can be helpful because the areas which are not compact, like daughter’s height, are interesting.
Our annual NFL example (last year was Tom Brady passing yards) is Aaron Rodgers over/under 38.5 touchdowns. Here’s how we visualized it in September 2021:
The thinking then, as now, was that Rodgers would throw between twenty and fifty touchdowns but not with equal odds. The number of touchdowns would be asymmetrical. It was much more likely Rodgers threw half of 38.5 than double it.
Five games into the season offers a chance to be Bayesians and update our forecast. In addition to the preseason line of 38.5, his career average is 33.4, and his current pace is 32. Mix in the chance of injury, and he could also finish the year with the ten touchdowns he’s tossed thus far.
Let’s tack this on to the 2021 predictions:
– +10 TD, 90%
– +20 TD, 85%
– +30 TD, 50%
– +38.5 TD, 10%
I wanted to go lower on the 38.5 percentage, but one lesson from Cade Massey is to be less certain about extreme events. So in the same way that online doesn’t equate to real life and we should adjust for that, I will adjust my percentages as well.
–
Daughter height is top of mind because I have eleven and thirteen year old daughters. 😬
There’s this idea in sports that certain people are “ruining the game”. It’s those baseball people who favor home runs and defensive shifts. It’s the golfers who drive for show and dough.
And we can blame computers.
And us. We’re to blame too.
Computers compress time. I could have mailed this to you as a letter but that would take me buying paper (after a trip to the store of course) writing it…yada yada yada…and you walking to the mailbox. Computers compress all that.
Analytics is a type of compression. Rather than a lot of people and a lot of time to learn about the advantages of home runs or infield shifts in baseball or long drives in golf, a few people with computers thought it might work and ran the data.
This is an issue we will see more of: novel data making interesting predictions.
“We looked on Twitter for anyone who announced they were going to their first AA meeting and we followed what they tweeted after that. Did they stay sober for ninety days or did they go back to drinking? Did they complain about being hungover at work? Did they celebrate their sobriety? Then we took all the data we could model from their Twitter feeds to try to predict if they would be sober. Things like: who do you follow, do they talk about booze, are you over 21, how do you cope with stress? We can predict with 80% accuracy if someone will stay sober or not on the day they decide to go into treatment.” – Jen Golbeck, November 2020
This algorithm, Golbeck notes, is also pessimistic, it tends to say you won’t recover when you will. And it’s confounded by the sample: only certain people announce things on Twitter.
These algorithm approaches will grow in the decision making blend. Part-of-that means understanding the tools. We are time traveling, leaping to the future rather than walking there.
So, this probably should have been written in January. Writing it in March means it should be more accurate. Less time, less variance (see also: Something is always happening)
One of the lessons from thinking like Tyler Cowen was to see the world as it is, not as we’d like it to be. Making accurate predictions is one way to approach that concept. One of the lessons from Phillip Tetlock’s Superforecasting was that improving predictions is possible.
Tetlock gives 10 commandments for better forecasting, one of which is to practice forecasting. Here are the prediction, if you want an overview of Tetlock’s book see the post: Is Bill Simmons a Superforecaster?
Hurricanes. NOAA “An average season has 12 named storms, six hurricanes, and three major hurricanes.”
Will there be more than 12 named storms? Yes, 90%.
Will there be more than 30 named storms (the 2020 record)? Yes, 25%
Will there be 3 or more major hurricanes (top winds of 111+mph)? Yes, 60%
Will I lose power at my home in Central Florida for more than 3 days? Yes, 10%.
Blog.
Will this blog have more than 41,000 views in 2021 (41k is the 2020 number)? Yes, 15%
Will this blog have more than 800 posts by year end? Yes, 30%
Finance
BTC will top 75,000 at any point in the year? Yes, 10%
BTC will be under 30,000 at any point in the year (started 2020 at this point)? Yes, 20%
ETH will top 5,000 at any point in the year? Yes 5%
ETH will be under 130 at any point in the year (started 2020 at this point)? Yes, 20%.
BRKB will top 275 at any point in the year? Yes 10%
BRKB will be under 234 at any point in the year (started 2020 at this point)? Yes, 30%
EconomicRecovery These will be graded per Bill McBride’s numbers on Calculate Risk
Any single day of the last week of the year will top 2M travelers (2019 was 2.0-2.5M)? Yes, 75%
Open Table reservations will be down less than 10% YOY? Yes, 75%
Open Table reservations will be positive YOY? Yes, 20%
Any movie earns more than 250M on opening weekend during the year (Dark Knight in 2019)? Yes, 5%
Predicting like Tyler Cowen, Enrico Fermi, and Nate Silver.
One way to think more like an economist is to think about incentives.
In our piece about Tyler Cowen, the setting is finding food in a strange place. The incentives question is, what to ask a concierge or driver considering their incentives are often avoid blowback rather than emphasize excellence.
The incentives of predictions are boiling in the last week before the US presidential election. Though math is always clear cut (2+2), the selection isn’t (why do 2+2 explain this?). Some polls, Nate Silver noted (on ModelTalk) offer extreme predictions as a form of marketing. Their incentive is attention, not precision.
In his update of The Signal and the Noise, Silver writes that it’s hard to change ones mind once allies, alliances, and reputation are created.
One way to avoid this pitfall is to make guesses to and with others who have a similar incentive structure. In the book, The Last Man Who Knew Everything, David Schwartz writes about Enrico Fermi. It’s December 1938 and Fermi has just fled Italy via Stockholm for his collection of the prize. It seems a good choice as the Italian media and Mussolini-mob wonder why Fermi shakes hands to receive his prize rather than give the fascist salute.
Settling into his professorship at Columbia (New York, but soon headed to Chicago) Fermi joins the “Society of Prophets”, or that’s at least what his wife calls it.
The Society meets monthly, and each member predicts ten yes/no events. A tally and total are kept. When the family moves to Chicago in 1942 Fermi successfully predicted 97% of the events.
“He did this, she (Laura) writes, using the most conservative algorithm imaginable: the next month would look almost exactly like the previous month. He did, however, miss one prediction—the surprise German invasion of the Soviet Union. The game was ideal for someone of Fermi’s temperament, invariably conservative and skeptical of any predictions of quick or revolutionary change.”
Fermi had strong priors. Fermi made boring predictions. Fermi was right. For Fermi and his colleagues the incentives were academic accuracy. For pollsters it’s media recognition.
The book is a bit thick, but inspiring is that while Fermi won the Physics Nobel Prize in 1938, his explanation for the discovery was wrong.
Michael Covel (@Covel) joined Barry Ritholtz (@Ritholtz) to talk about trading, trending, and Thailand. Okay, not just Thailand,but it starts with ‘T’ and I’m a simple writer.
Covel – and Ritholtz – fall into the category of “good talkers” and I listen to at least part of all of their podcasts. Get them together and you get two solid hours of conversation.
This podcast wasn’t as into the weeds I know about but provided lots of superficial anecdotes about things I don’t know like the experiences of working in Japan, what it was like to see economic ideas change, and how capital is like Gap t-shirts.
One final personal note, Covel challenges my ideas on his podcast. Whether it’s something I don’t understand or something I’m not interested in – but I try to listen anyway. One of the ways to use Twitter for good is to bust your biases. That’s something true for podcasts as well.
Ready?
Greats are made, not born.
Covel was influenenced by Richard Dennis(wiki) and the Turtle traders. The story goes, and Covel says he has no reason to doubt this, that Dennis and fellow trader William Eckhardt saw Trading Spaces (Eddie Murphy). Eckhardt said that could never happen. Dennis said it could, and he’d prove it. A bet was made.
Dennis recruited a group of – mostly average – applicants how to trade on a trend-following strategy that he created. This group of ‘Turtles’ turned out to be widely successful.
The application is true with regards to the podcast itself. Covel made himself into a great trader, writer, and podcaster. Ritholtz admits this too, that radio didn’t come naturally to him and that there was a learning curve. He’s gotten pretty good at it.
This is true for so many people, especially comedians.
Comedians are – sometimes sadly – not very good at anything. A lot of the art from people like Phil Rosenthal or Judd Apatow comes from places of pain or darkness.
In his interview with Rithotz, Ken Fisher noted that even though his dad was a great writer, he wasn’t.
Ritholtz compliments Fisher’s work and says, “you’ve obviously inherited your dad’s writing skills.”
“Not true,” replies Fisher, “because if I had inherited them they would have come to me very naturally, but I worked hard to learn how to write.”
Except in rare, physical domains like sports, you can be good (or great) with some work.
Jack Canfield cautioned Tim Ferriss about pegging “being great” on only one thing. He tells the story of a family friend who wanted to be in the NBA. Of course, the guy wasn’t good enough to play professional basketball, so he took another route. He worked to get in the front office. He’s in the NBA, just in a different form than he first imagined.
It’s trend-following, not trend predicting.
As a simpleton (n00b), I didn’t really know what “trend-following” was. Covel set me straight when he said:
“If you’re a trend following trader you don’t have a mindset or prediction where any particular market is going to go. So when it starts to move, you’re just following the crowd.”
Later in the interview he adds, “nothing can be predicted.”
I was thankful to hear this because predication is hard. Like, really, really, really hard. It seems to me that Covel (trend-following) champions non-prediction. This is paramount for complex systems (trading, ecosystems, a college classroom). There will be some outcome, but not any particular one. Mathematicians explain it regarding birthdays, so I will too.
Imagine you’re in a room with 25 other people. Mostly strangers, none who you know well. One of them is me, and I approach you with a bet.
“Would you wager $100 that two people in this room share a birthday?” I ask. Note, this is much easier on the internet because my poker face is terrible (and so is this bet).
“Hmm,” you think to yourself, “this seems like a good bet.” There are only 25 people in the room, and there are 365 days in a year. Chances are definitely against it. Right? “Sure,” you say, “I’ll take the bet.”
So you and I go around the room (with a paper plate) and write down birthdays until we get a match, and chances are that we will.
This birthdays bet (explained very well by John Allen Paulos in Innumeracy, and Clay Shirky in Here Comes Everybody) demonstrates the something will happen ethos that trend-following subscribes to. We can’t predict what birthday will match, only that one probably will. The math, very briefly, looks like this.
Trend-following lives in this same cul-de-sac of knowledge. There will be something that happens, and you can profit by acting when it does. (And now you have a bet for the next in-law family reunion)
Would you rather have a chance of 80% of yes or 20% of no?
Covel and Ritholtz touch a bit on Kahneman’s work on framing and loss aversion.
Look at the question above, it seems like they would be the same but Kahneman found that the responses people give are quite different.
If people are given a 40% chance on winning $100 or just to take $20, they’ll often favor the latter. The saying could go, $20 in hand is better than any psychology mumbo jumbo and rewards in the bush.
That’s well and good and unremarkable, except that people switch their choices around when the outcome is reversed. If people are given the chance to a 40% chance on losing $100 or just to lose $20, they’ll often favor the former. People are risk seeking to avoid losses. This matters in trading because if you’re trying to get back to zero, psychology suggests you’ll take more risks.
Related to this is loss aversion, which Richard Thaler explains beatutifully with a pyramid analogy. Also in that post is how Josh Brown creates win-win situations from down markets.
The prostate test test.
Covel says that he and Ritholtz are at the age for getting a prostate test, but should think twice about it. The side effects may be higher than advertised and the payoffs lower. (Note above we just mentioned our tendency to seek risks – have the test and side effects – to avoid a loss).
Covel says that reading Gerd Gigerenzer has changed his view on this. Gigerenzer has been on this blog before. When Scott Galloway spoke with Rithotz he outlined his five aspects of great companies. His model is simple and we noted that that’s good because we don’t want models that fit too well. In Gut Feelings Gigerenzer writes, “In an uncertain world, a complex strategy can fail exactly because it explains too much in hindsight.” That could be part of Cove’s apprehension of prostate exams. The other part is bad math, which Covel also got from Gigerenzer.
Let’s explain.
Pretend that 14% of a population has a disease, and a screening tool exists that is 98% effective. If we have 1,000 people, how many will be correctly diagnosed? Much like our birthday bet, things are not as they first appear.
Let’s do the easy math first. If 14% of 1,000 people are sick, then we have a chart that looks like this:
Has disease
Does not have disease
Total people: 1,000
140
860
Not so hard, and if we had a test with perfect prediction, then that would be the end of it. But our test is only 98% accurate.
Of our 140 sick people, we will correctly identify 98% of them, 137. Our chart gets updated.
Has disease (140)
Does not have disease (860)
Tested positive
137 (98% of 140)
Tested negative
3
Then we do the same to the ‘does not have disease’ group.
Of our 860 not sick people, we will correctly identify 98% of them, 843.
Has disease (140)
Does not have disease (860)
Tested positive
137 (98% of 140)
17
Tested negative
3
843 (98% of 860)
We see then that 20 (3+17) of our original 1,000 people will get the wrong diagnosis – 5% of the general population.
Now if you thought this was a bit murky math, you aren’t alone. Doctors miss this too.
Successful people have systems, and follow the rules.
“This is not a day to day guessing game,” Covel says. There’s not single point in the interview where I noted the emphasis on systems, but by the end it was clear. Covel is a systems advocate. When Dennis taught people, he taught them a system.
When Covel speaks about trend-following he explains it as a system with rules:
1- What’s the portfolio?
2- What will force you in?
3- How much will you be in for?
4- When will you exit for a loss?
5- When will you exit for a win?
Those questions are easy in hindsight, but hard in application. In part because it takes time and temperament.
Systems don’t have to be complicated either. Ritholtz says “never ask a room full of people what they want for dinner.” That’s a system too. Like a basic computer code or logic statement; if more than 8 people, then do not ask what people want for dinner.
Covel’s rule #4 (When will you exit for a loss?) is easy to state, but hard to act on. Ritholtz says that when he was a trader it was “okay to be wrong, but not okay to stay wrong.” You can’t reach and try to get back to even because you’ll take more risks getting there.
Some problems are too hard for any system. In his book The Hard Thing About Hard Things, Ben Horowitz notes that for hard problems there’s no prescription.
“The problem with these books (about business) is that they attempt to provide a recipe for challenges that have no recipes. There’s no recipe for really complicated, dynamic situations.”
The best systems are ones where the big mistakes are eliminated (as best they can be, but never absolutely) and wiggle room is given for the small choices.
, and Clay Shirky in 
) demonstrates the something will happen ethos that trend-following subscribes to. We can’t predict what birthday will match, only that one probably will. The math, very briefly, looks like this. 
, Covel suggests 
, 
, 
, 
, and 
The Waiter's Pad 
