Not just OK OKRs

Sarah Tavel told Share Parrish:

“At Pinterest our growth team decided their OKR was monthly active users, a lowest common denominator thing. But if you choose the wrong metric you end up optimizing for the wrong thing, you’ll build the wrong features. Startups are incredibly resource constrained and you waste a lot when you focus on the wrong things. When the team realized this and changed the OKR to Weekly-Active-Pinners the entire roadmapped changed and we were able to serve the users much more successfully.” – @SarahTavel The Knowledge Project.

Tavel’s quote could be about 2000s baseball as well. The early days of baseball Moneyball were an era of what Tavel calls vanity metrics. At one point in the Michael Lewis bestseller, protagonist Billy Beane yells: We aren’t selling jeans! His point was that classic metrics like hits, home runs, or even body-type weren’t the optimizations he was looking for.

The problem that Tavel’s and Beane’s teams faced was a data collection problem. These metrics were mostly right and easy to collect.

“I have an allergy for vanity metrics. I can see a vanity metric a mile away. It comes down to intellectual rigor and being honest with yourself: what are you measuring and is it the right long term thing?” Sarah Tavel

Really wrong metrics push behavior in absurd directions. For instance, records used to earn certifications (Platinum, Gold, etc.) based on shipments not sales. Sgt. Pepper’s Lonely Hearts Club Band soundtrack (1978) was a Platinum album but was a sales bust. That’s what happens with an OKR based on shipments, not sales.

To their credit, the RIAA changed the rules for certifications in 1979. That’s what Beane did too. Tavel too. It’s a good reminder to ask: am I using this information because it is helpful or easy?

–
Moneyball might be the best way to win in sports but sports is a story and stories need narrative. I loved the Tim Duncan Spurs but the media didn’t. It’s why there’s only one honest sport.

Weekly active pinners? Hold my beer.