Measure What Matters (book review)

Measure what matters book review

There are two aspects – contents and context – to John Doerr’s 2018 book, Measure What Matters, a book about OKRs (Objectives and Key Results).


Objectives are “what is to be achieved, no more and no less.” Grow the blog, lose weight, or strengthen important relationships.

Key Results are ways to “benchmark and monitor how we get the objective.” List ways to grow the blog, lose weight or build relationships. “It’s not a key result,” Marissa Mayer would say, “unless it has a number.”

Straightforward enough. Is this a book that could have been a blog post?

Maybe, but Doerr offers a trio of cautions.

Warning 1: OKRs are not a way to show activity, they are to focus attention and weigh the opportunity cost. Organizational achievements, not ego appeasements.

Warning 2: Sometimes incentives hijack the Key Results (Goodhart’s Law). An antidote is paired counterparts. In the Wells Fargo cross-selling scandal the Key Result of open accounts could have been paired with monthly active accounts.

Warning 3: OKRs are a tool to use not a dogma to follow. If objectives change then OKRs change too.


Context is a Bob Moesta word encompassing who, what, when, where, why, and how? Steak and hot dogs are ‘good for dinner’ within the right context. The same goes for OKRs.

Doerr is a venture capitalist at Kleiner Perkins. OKR adopters include Intel – where Doerr learned from Andy Grove – and Google among other technology firms.

The OKR system, Doerr wrote, “was a great impedance match, a seamless gene transcription into Google’s messenger RNA. OKRs were an elastic, data-driven apparatus for a freewheeling, data-worshipping enterprise.”

Google was a perfect match. But your business may not be.

OKRs, as Doerr presents them, requires a certain culture. Part of their effect is to argue well. Andy Grove set the Intel culture for OKRs because Andy Grove was at Intel. Once he left the culture changed from bettering to bullying.

Doerr has many examples, one of which is Zume Pizza, but they’ve gone out of business. What’s the right lesson in that? What’s the context?

OKRs are lightweight, malleable tools. But their usefulness varies. Will OKRs be OK for you?

ESPN’s innovation dilemma

One pant leg on is a local maximum. One problem is solved but the larger set is not.

Clayton Christensen’s series on disruption and innovation is about local maximums.

Money machine go brrr is a strong incentive to keep printing. Maximizing a profitable business makes sense, which is the dilemma! Organizations find themselves looking good in one pant leg.

The solution to local maximums is exploration. But this is costly – money, status (uh oh), time, reputation. Plus the stakeholder’s opinions.

The solution, Clayton Christensen writes, is separation. Different groups with different strategies, finances, and when possible physical locations.

Solutions via exploration are important because customer and consumer preferences – their JTBD – change.

“We are all under the Disney umbrella,” Brian Burke said, “ is a huge enterprise with an army of people and is a revenue generator in so many ways. It’s difficult to change course. FiveThirtyEight is agile, nimble, and experimental so (publishing there) was a great opportunity”. go brrr.

Which is the dilemma, and Disney/ESPN uses FiveThirtyEight as the exploration solution. Who knows if Burke’s writing approach is better, but the publishing strategy is a solution to the innovator’s dilemma.

“The next” will be different. Whatever is next will have a different business model than the current Great Firms (Christensen’s subtitle). Whatever is next will have a different maximum. It will be a short vertical video or the degradation of the sport monoculture or something we can’t predict today.

Or even an analytic forward analysis from Brian Burke.

Zone Two cardio

When not listening to podcasts, I can often hold a conversation while running, and conversing is a proxy for Zone 2 cardio. It’s a Goldilocks exercise. Is this a better way to run?

Between Instagram stories, Andrew Huberman emails, and runner scuttlebutt, Zone 2 is in the zeitgeist. Without getting into physiology, can we figure out if it’s a fad or worth our focus?

Metrics. We count what’s easy. Miles and minutes are easier to count than training zone sessions. Hurricanes are graded by their wind speed but it’s the water volume that does the damage. Speed is easier to measure and report. Daily users were standard but it was people who used the product that gave Pinterest engineers a better signal. Easy to count, collect, and combine metrics are overrated.

+1 for Zone 2.

Incentives. ‘Why am I seeing this?’ is a good question. Who is selling me something? Vegetarianism is underrated because the people who sell it don’t have that much to gain. Status points, sure, but if something’s not being sold it’s more believable.

+1 for Zone 2.

Barbell approach. Sometimes average measures are worse. Exercise may be like that. Every workout at 145 beats per minute is not as good as some workouts at 170 bpm and others at 125bpm, though the average is the same. Zone 2 is easier than the 145 average which goes against the ‘more is better’ mindset.

+1 for Zone 2.

Swept up. It’s easier to get swept up in something. Housing bubbles. Crypto. Or just the feeling of a local sports team playing deep into the playoffs. The internet feed makes it even easier. Instagram knows my age, interests, and engagements. Oh, a forty-year-old running dude, the magic math concludes, here’s a bunch of running videos promoting zone two cardio. Is this an echo chamber?

-1 for Zone 2.

Conclusion: I’ll do more Zone 2 training. Happy New Year.

Crazy Russian incentives

Around 1992 Russia privatized state companies. The government gave each citizen one voucher they could bring to an exchange for a share of that day’s company. A simple plan – until humans get involved.

Not all Russians wanted to own shares. Local markets emerged. A small fish bought all the vouchers in one neighborhood, a medium fish bought all the neighborhoods in a town, a large fish bought all the towns in a region. Eventually sacks of vouchers made it to the national exchanges.

Though unintended, these mini-markets worked. Free economies FTW. So far so good.

Each exchange had a schedule. A modern Monday might be 1,000 shares of Apple at nine, 200 of IBM at ten, 500 of Ford at eleven and so on. If only one person showed up Monday at nine they would get all the shares for their vouchers. It was the market mechanism at work. It’s cheaper (more valuable) to not bid against someone in an auction. When one companies shares went up they shut down the airport the day before their voucher offering. Another company ignited a tire fire on train tracks leading in and out of town.

Insiders were insistent on owning their companies because the valuations were way off. By one estimate, the voucher privatization program valued the entire Russian economy at ten billion dollars, or one sixth the market cap of Walmart. If you could buy a legitimate twenty dollar Amazon gift card for one dollar would you? Rather, how many? This economic transition was called a katastroika. A combination of the catastrophe and perestroika – Gorbachev’s politics.

George H. W. Bush has his last year as president, Achy Breaky Heart finishes the year as the fifteenth most played song, and there’s money to be made in Russia.

“I went to someone in the investment management division,” Bill Browder writes in Red Notice, “expecting him to hug me since I was sharing the most joyous jaw dropping investment opportunity he would ever see. Instead he looked at me as if I was suggesting the firm should invest in Mars.”
Russian privatization was a huge opportunity. Everyone at Salomon Brothers missed it. Why? Incentives.

On Browder’s first day, his first manager explained the system: generate five times your salary or you’re done.

“Nobody at Salomon Brothers could divorce themselves from their own narrow mindset. Perhaps if I had been more subtle and clever I could have pierced their myopia, but I wasn’t, I had no political skills. I presented my idea for weeks and weeks hoping that through repetition I would get through to someone.”

Incentives and culture form what people do when they’re not told what to do.

At the London office the formula – which worked wonderfully – was fees through consulting.

Eventually Browder’s repetition got through and he got a call from Bobby Ludwig in New York. Two days after a phone call with Ludwig, Browder pitched the idea. An hour later Ludwig delivered twenty-five million dollars and marching orders. At the New York office the formula for Bobby Ludwig was to make money.

When Browder returned to London he had to switch departments but couldn’t find a desk. “Bill, why are you bothering me with this?” Ludwig asked when Browder appealed to him, “If they won’t give you a desk just work from home, I don’t care where you work. This is about investing in Russia, not desks.”

There’s this idea that to understand what’s going on in the world someone has to know the history or stay on top of things. But sometimes we can come back to first principles. We’re all humans with incentives. Also, the Red Notice audiobook performance is amazing.

Survivor Games

December 13 is the Survivor finale and a chance to highlight the different ideas of the blog.

Sampling. These people are not “representative samples“. Survivor hosts a casting call for people with good stories. Like Bob Iger’s big lesson, entertainment isn’t about reporting so much as stories.

Incentives. Like the many games of Jeopardy, Survivor has layers of games. When there are layers of games it’s difficult to judge actions. Is someone trying to win the game or claim later fame?

Business models. Thanks to MTV in 1981 and then Real World in 1992, one entertainment business model is to create value by editing rather than crafting (unlike Seinfeld). Put regular people with backstories (hence sampling) in interesting situations and things will happen. Edit a month of island living to a few dozen hours and viola.

Customer acquisition costs. Sequels – it’s season 43 for Survivor! – have lower CACs. Consumers don’t need to be educated.


Prices are set by the amount supplied and the amount demanded. When supply is mostly fixed, like top home-run-hitters, prices rise. This is the market mechanism.

One way to shimmy around this feature is to find things nearly as valuable, but less demand. This is Moneyball. It’s also investors who “fish in smaller ponds”. It’s also art. Collectors pine for Picasso but many fewer for real estate. Discretionary income + housing budget is a lot of money. Find a different attribute to compete on can be good advice.

Sometimes. We can overcorrect. Kristen Berman noted that one experiment which shifted the incentives from monthly to daily saw sales reps “focus on selling large numbers of cheaper items rather than more expensive items that have higher margins. A focus on short term returns can undermine pursuits of higher impact goals.” It was a case where 100 monthly sales did not equate to 4 daily sales.

This is Goodhart’s Law, when a measure becomes a target it ceases to be a good measure. A textbook example is higher education ranking hackings. Some schools counted “a postcard expressing interest” as an applicant. More applicants meant more rejections and a selectivity shine that was only a veneer.

But wait. Goodhart’s Law is a human quirk and quirks can be hacked. Airbnb grew because like eBay or Amazon, stars replaced brands. But while a four star hotel is mostly the same four star homes were not. So the company added subcategories.“We picked the subcategories based on what guests want,” said Jiaona Zhang, “but we also picked subcategories based on what we wanted our hosts to do.” 

Airbnb used Goodhart’s Law to direct their host’s attention. Once a category was counted hosts worked toward it.

Measures are a tool. They can be like Moneyball and show cheap things. They can be like Goodhart observed and show unintended consequences, but also tweaked for tidy Airbnb hosts. Measures only seem static but really reveal a lot.

Alchemy in the office then at the pump

“The tricky part of incentive design is that there are noisy signals about what could work and what may not work. For example, when given a hypothetical choice between cash and non-cash incentives, people overwhelming choose the cash incentives but studies consistently show that giving additional compensation in the form of non-cash rewards can be more motivating than cash.”

Kristen Berman, The Science of Change podcast

The basis of Alchemy is to create a lot of of value for a small cost. Alchemy happens for donations, for interest payments, and even for paying for a Peloton. Non-cash incentives are another form of Alchemy: the value of the item is greater than the cost to produce it.

On March 9, 2022 gas in Florida cost $4.19 per gallon. A friend noticed this and stumbled into a bit of Alchemy. His company is creating a mileage reimbursement program for the employees.

A “mileage reimbursement program” moves past a basic cash benefit and into the realm of alchemy because it uses our availability tendency: what’s top of mind is most important. Gas prices are often top of mind thanks to their frequency (and SIZE), but with Russia’s invasion of Ukraine the availability is even greater. And that’s not all! Thanks to our mental accounting we perceive pain at the pump.

A “mileage reimbursement program” is a great idea. It delivers more value than it costs. Here’s a few more:

  • Mother’s Day. There’s no holiday that deserves status elevation more. What if an organization had flowers for moms (or husbands to take home to moms) or bulk ordered a delivery. Hey everyone, at lunch today there’s a signup form if you want to order flowers for your mom. This is kinda tricky because if gifts are too easy they lose some impact.
  • Annual bonus. There’s alchemy calling it a “Christmas” rather than “end of year” bonus.
  • Four day workweeks. Businesses have some quantity of work that’s divided into some number of days. Divide those two for a rate. Reducing the number of days means the rate must rise, but many employees find this a fair trade as evidenced by the work-from-home success.

The original advertising alchemist, Rory Sutherland, noted that organizations often optimize the wrong metrics. Don’t make the trains run faster, which is very costly, Sutherland said, if you can make them more enjoyable for much less. A place to sit, a plug to use, and a coffee to drink go a long way for a small cost. That’s alchemy. Find something to provide that has an outsized impact.

Energy for change

This post is part of our thinking about ease and its counterpart design.

Football in Florida in February

John List begins his book, The Voltage Effect, with the example of Nancy Reagan’s D.A.R.E. program. That program, failed, List notes because it was based on a false positive. “It was a pretty large scale study in Honolulu,” List said, “the problem was it was only one study, it was never replicated, and it was never the truth.” Part of the reason D.A.R.E. is a case of something is always happening is the social incentive. It felt good to have a solution. It felt good to align with Nancy Reagan or local law enforcement or your child’s school.

A modern parallel is Michelle Obama’s Let’s Move! program. My year of AmeriCorp was in this heyday and it felt good to align with a political party or athletes or your child’s school.

Contrast Obama’s and Reagan’s initiatives with what I witnessed the first week of February: Florida football. This park was swarming with kids. Every NFL team was represented (and what a rollercoaster the kids on the Bengals team went through) and the parents were into it.

There’s no lack of football in Florida. If the current First Lady wanted a win, she should create a program that further promoted football in Florida. And the reason why is the ease. There’s not friction for Florida football. The weather is good to great (though dangerously hot) all year long. The culture welcomes football. There’s lots of people already doing it, so doing more of it wouldn’t be too much. Contrast Florida football with ‘Just Say No!’ or ‘Let’s Move!’. All the things in favor of Florida football are missing for the former First Ladies.

How to vaccinate the world: Hire the smartest, most attractive, and persuasive medical students (doctors and nurses) to go door-to-door across the country. Or along the football theme, get the best recruiters. Have them sit on the couch, look the person in the eye, and sell them on vaccination. That would work. But like ‘Just Say No!’ and ‘Let’s Move!’ it takes too much energy. But football in Florida? No energy needed.

There’s a gap between things I would like this person to do and things this person does. Energy closes the gap. Wordle wonderfully demonstrates energy. It’s easy to learn, easy to share, easy to play, easy to habituate. But Wordle will fade because it struck kindling. Unlike football in Florida there’s not a lot of factors working in its favor like with Facebook or automotive culture or take-out-pizza.

Energy, ease, friction, design – they’re all ways to address the same idea, how to change.

How to board an airplane?

Everyone knows how to board an airplane, back to front. It’s logical. Back to front means that if someone is taking their time in seat 21C the person in 20A can still seat themselves.

Physicist Jason Steffen built a computer model to see how much faster back to front was relative to front to back. He was shocked. The difference was minimal. Hmm, Steffen scowled at his code, is there a better way?

What if instead of 30 to 1 or 1 to 30, a plane boarded everyone in row 30, then rows 1, 2, 3…. That might work right? The folks in row 30 would have time to stow and seat without holding anyone up.

That kinda works. Steffen’s code continued and compared the 30, 1, 2…29 option to the 1, 2,…30 option. The code noted the faster sequence, and switched around two more numbers. Again it kept the faster option and computed another switcheroo. The fastest boarding process turned out to be boarding every other row.

“It turns the boarding process from a serial process, where one person gets to their place, puts their luggage away, and sits down into a parallel process where you send in fifteen people, say in all the even rows, and they all put their luggage away and sit down at the same time. And then you send in the next group of people.” – Jason Steffen, August 2021

Steffen’s code was a Markov Chain Monte Carlo, a way to solve problems through computer code and exploration. But like wet bias or wait times, ideal solutions may not be the best.

One problem with Steffen’s method is when people travel in groups, especially families. Another obstacle is the culture of air travel, there’s some established norms. Further confounding the case is an airline’s incentives. Faster turns do matter, but relative to upgrades how much does saving time save the company?

What to think while waiting

David Henderson recalls his first class with Armen Alchian in 1972, who started class noting that if someone understands the incentives they can understand a lot of ‘mysterious’ behavior.

“On the first day of class he said when we went to the bookstore we’d be in a long line and we’d be angry about that. But we shouldn’t be. We should be happy because it means economics works. The people working are on a standard wage and they don’t have a strong incentive. There’s no ownership, no one gets a big benefit if they do better.” – David Henderson, September 2021

We should be happy to wait? Kinda.

One theme around here is the value of curiosity. Mostly that means to wonder how and why and when things happen – or even when they don’t happen. And incentives are ripe for thought.

We’ve looked at prediction incentives, Marine incentives, and the incentives of salting roads. There’s music incentives, be careful what you count. And football recruit incentives, more careful counting.

Incentives come up so much because one way to be curious is to ask: why does that happen? and part-of-the-reason that happens are the incentives.

This kind of curiosity takes work. Like exercise takes calories, crypto (or cars) take gas, or relationships take conversations – curiosity takes work. To be curious is to think, wonder, and ask questions. To be curious is to move past the simple story and see what else is there. Sometimes there’s not more, sometimes there’s incentives, and sometimes there’s something unexpected.

One of the highlights of my interneting has been writing a post about curiousity for Ribbon Farm.