Survivor Bias in WW2 Airplanes, NBA players, & Mutual Funds

My project on Survivor Bias is an attempt to answer the question: “What can we learn from dead startups?”

Looking at dead startups is  form of  thinking called inversion that Tren Griffin and Charlie Munger both use. Rather than asking:

– What do successful startups do?

We can ask:

– What did unsuccessful startups do?

Avoiding the latter can be just as helpful as pursuing the former.

Look at Blockbuster and Netflix, two companies that had a lot in common. In many ways Blockbuster was the better company. They had more offerings of each new release movie, and you could get the movie from them more quickly. Depending on how fast you turned around each DVD, the cost savings wasn’t huge. Netflix relied on the notoriously error prone United State Postal Service, Blockbuster on prime retail locations.

Why study Netflix but not Blockbuster? Because Netflix survived, but that dismisses everything Blockbuster did right – or wrong. Much of that information is now lost and we can’t learn from Blockbuster’s mistakes. Luckily the same can’t be said for technology startups.

Thanks to blogging – and especially Medium.com – the number of “my tech startup failed and here’s what happened” articles are plentiful. Even better is that they’re consistent. In combing through the reflections on 60+ defunct startups a few patterns emerged.

Those findings will come soon enough (you can follow the project at Medium). First we’ll explain what Survivorship Bias is with three characters: World War 2 airplanes, basketball players, and mutual funds.

WW2 airplanes.

In World War 2 the allied nations had a problem, they were losing too many aircraft. At the most dangerous times of the war the odds of coming back were a coin flip. 

The Navy decided that the best solution would be to add armor to the planes, but could only add so much, and didn’t know where to put it. They took this question (how much armor to add and where) to Abraham Wald, a mathematician working on the war effort. Wald applied his models to places where the plane hadn’t been shot.

This took everyone aback. Why armor where the planes hadn’t been hit?

Wald reasoned that if some planes were able to return after being shot in certain places, other planes had not been able to return after being shot in the inverse of those places. Wald found the survivorship bias in looking at only the planes that made it back.

Basketball players.

Earl “The Goat” Manigault is one of the most famous basketball players in New York City. Born in 1944, Kareem Abdul-Jabbar once called him “one of the best basketball players in New York City,” Manigault is a small blip on the radar of American basketball, and maybe not even that.

His story is still worth telling. Part of the value of inversion comes from looking at what not to do. For planes the inversion led to the conclusion that damage to certain areas was very bad. For basketball players we can follow the same reasoning.

Manigault failed to become a successful basketball player for two reasons; his game and his addiction.

Besides praising his game, Abdul-Jabbar also noted it had flaws. “Earl couldn’t shoot the ball from beyond eight feet. He could leap out of the gym, but he couldn’t shoot the ball beyond eight feet, and he wasn’t interested in passing it.” Outside shooting and passing are two key basketball skills and Early didn’t have either one.   

The second fatal flaw for Manigault was his additions. He was kicked out of high school for smoking marijuana. The same year Abdul-Jabbar entered the NBA, Manigault entered prison for possession of drugs.

We see in this story of two New York City basketball players that there are things to learn from both. Study Abdul-Jabbar’s path and you’ll see that he took dance lessons and worked hard. You’ll also see external forces, Abdul-Jabbar had the good fortune to be at UCLA and with the Los Angeles Lakers. Those things are good to know, but not that helpful for aspiring basketball players. 

Study Manigault and you’ll see equally instructive things. If you don’t have the core skills for your pursuit, whether that means passing in the NBA or coding a website, you won’t survive there. Ditto if you become addicted or lack a supportive structure around you. The most helpful advice might come from Manigault.

Mutual funds.

When I first looked at a mutual fund statement I wondered how people could be so stupid. “You just pick the one that’s had the higher returns,” I told my father. I was obviously a know it all who knew nothing.

Time passed and I noticed the fine print, past performance may not be indicative of future results. A few years after that I learned that survival bias is present in mutual funds as well.

Mutual funds are collections of investments where investors pool their money to buy securities, usually with a theme. An example might be when all the teachers in a state – through their pension – buy a “long term growth fund.” This fund, along with all other “long term growth funds,” will be lumped in the “long term growth fund category.”   

At the end of each year, one news source or another will report on how well “long term growth funds” did that year. The next year the same thing will happen and so on. Now here’s where our bias enters.

This reporting won’t include all the funds. Some of the funds will go out of business because of poor returns. Those poor performers will be omitted from the next year’s report because they won’t be around anymore. That is, the report will only include the survivors.

We see that CNN or whoever will fail to report about Growth Fund B, which due to poor performance has been closed. This boosts the returns at all levels and happens in a fractal manner with nearly all investments.

Why should founders care?

Founders need to be warned about specific causes of death and survivorship bias because other founders wish they had been. This is not my opinion. A consistent thread in the postmortems was the influence of Survivor Bias.

One founder wrote, “all that you can really glean by examining the crème de la crème is perhaps an understanding that putting in just enough hard work will (probably) generate just enough sheer luck for you to accomplish what you set out to do.”

Another wrote: “The startup press glorify hardship. They glorify the Airbnbs who sold breakfast cereal to survive, and then turned their idea into a multi-billion dollar business. You rarely hear the raw stories of startups that persevered but ultimately failed.”

A third: “For every entrepreneurial story where the success is attributed to the founders being steadfast through hard times, there are probably five failures that were due to founders being stubborn or scared about making a hard decision. This is a prime example of survivorship bias, where we attribute success to traits that survivors exhibit, even if there was no causation by those traits.”

That failed founders made time to share their experiences is a gift. How often does someone account for why their marriage fell apart or their attempt to flip houses ended?

We don’t hear those stories because stories of failure are difficult to tell. Except in the industry of technology. The mantra of failure is not seen as a scarlet letter as much as a badge of honor. It’s more like a “I ran a marathon and all I got was this lousy t-shirt.”

–

Thanks for reading, I’m @mikedariano on Twitter. If you want to follow along, the bestplace is at Medium; https://medium.com/survivorship-bias.