Tag: hidden metrics

Alice and Bob own soccer teams…

MIKELeave a comment

Alice runs her team conservatively and finishes with 17 wins, 17 draws, and 4 losses. 

Bob runs his team with more variance and finishes with 19 wins, 11, draws, and 8 losses. 

Which is better? 

Let’s reframe, like the ball bet. Is it better to exchange 2 wins for 6 draws and 4 fewer losses? 

Haralabos ‘Bob’ Voulgaris bought a soccer team because he knows these answers because he’s seen these questions. 

After Moneyball but before Morey-ball, Haralabos discovered and gambled on basketball inefficiencies. The best known now is the three-point shot. Voulgaris thinks that soccer is similar. Teams earn three points for a win, one for a draw, and zero for a loss. Rather than three or two points in basketball, it’s three or one points in soccer standings.

Soccer’s business model is like the music business model. Artists lose money recording an album, break even touring, and profit from the merchandise. This had to be Pixar’s business too. Division three soccer teams lose money, division two teams break even, and La Liga or Premier League teams “print money”. 

Soccer teams can move up (promotion) or move down (relegation). Bob’s team, CD Castellón is in the third division and they need about sixty-eight points for a chance at promotion. 

Both Alice (17/17/4) and Bob (19/11/8) earned sixty-eight points – but they don’t seem equal. This is Bob’s point – it’s worth risking more for wins than less for draws.

The big question is: What are the right metrics for this system? 

  • Hurricane wind speeds are probably the wrong metric. Though easy to measure they don’t convey the potential storm damage which comes from the rain, surge, and flooding. Moneyball and Morey-ball are both descriptions of systems where the important metrics shifted.
  • ‘Draws’ is a wolf in sheep’s clothing. It seems fine – splitting the difference between a win and a loss – but the unique point system shifts the weight. 
  • Risking more – Bob’s approach – focuses on what matters. It’s the points stupid.

Humans are loss averse but the soccer standing scoring rewards bucking this trend. Alice and Bob own soccer teams, let’s see what happens.

Travel budgets

MIKE4 Comments

Actions are the children of mindset and environment.

When running his document storage company, AJ Wasserstein created a travel budget. Budgets are good. Budgets are a design tool, and we are all designers.

Wasserstein’s budget wasn’t denominated in dollars, it was in days away from home

“One thing I did while working at Archives One was give myself a travel budget. I gave myself permission to travel a certain number of days a month. It wasn’t a financial budget, rather a nights-away-from-home budget. If I started to exceed that consistently, my role at the company needed to be cleaved and I had to hire someone to do part of what I was doing.” – AJ Wasserstein, Circle of Competence, June 2021

Wasserstein asked a different question. Rather than ask what was financially costly he asked what was socially costly and optimized for that. A lot of times we assume that the important is easily measured. Dollars? Yes. But other things too.

‘Good’ numbers

MIKE

This summer my kids were not going to watch too much YouTube. But, things changed. My eleven-year-old got into Moriah Elizabeth, a YouTuber into decorating and painting. Her channel is good. It’s interesting and entertaining. It, for me, avoids the overreactions and clickbait present on YouTube. She’s super positive and if not teaching kids how to be creative at least she shows them that it’s okay to mess up, laugh it off, and try again.

She wrote a book, Create this Book where each page is a prompt to draw only with polka dots, or draw a structure, or draw something without lifting your pencil from the page. We bought it. It’s fun. We do a page a day and laugh at or admire our drawings after.

This is to say that not all screen time is equal. But it’s easy to count and present equally. Apple offers a Sunday notification that your screen time was higher/lower than last week. That’s not really helpful. It would be like if a refrigerator displayed the calories consumed but not what exactly someone ate.

It also happens, says Betsey Stevenson, at the macro level during each jobs report. There’s the unemployment number and the initial response is that more workers are better. However it kinda depends on the timescale.

“When we see the ‘quits’ numbers really high that seems bad. In the short run we’re going to see fewer jobs. But it’s actually an optimistic time.” – @BetseyStevenson The Ezra Klein Show

People tend to quit their jobs when times are good and the next job is immediate. As people move about in the economy it follows that wherever they land will probably be a better fit, a win-win for everyone. But that’s hard to quantify.

One way to flip this problem is to restructure the counts. Basketball coach Todd Golden will redraw the lines on a basketball court. If a player shoots from inside the arc it’s worth one point. Shots outside the arch are worth four. That’s clever counting. Restructuring the way a player perceives the points is a way to find the ‘good’ numbers.

Not just OK OKRs

MIKE2 Comments

Sarah Tavel told Share Parrish:

“At Pinterest our growth team decided their OKR was monthly active users, a lowest common denominator thing. But if you choose the wrong metric you end up optimizing for the wrong thing, you’ll build the wrong features. Startups are incredibly resource constrained and you waste a lot when you focus on the wrong things. When the team realized this and changed the OKR to Weekly-Active-Pinners the entire roadmapped changed and we were able to serve the users much more successfully.” – @SarahTavel The Knowledge Project.

Tavel’s quote could be about 2000s baseball as well. The early days of baseball Moneyball were an era of what Tavel calls vanity metrics. At one point in the Michael Lewis bestseller, protagonist Billy Beane yells: We aren’t selling jeans! His point was that classic metrics like hits, home runs, or even body-type weren’t the optimizations he was looking for.

The problem that Tavel’s and Beane’s teams faced was a data collection problem. These metrics were mostly right and easy to collect.

“I have an allergy for vanity metrics. I can see a vanity metric a mile away. It comes down to intellectual rigor and being honest with yourself: what are you measuring and is it the right long term thing?” Sarah Tavel

Really wrong metrics push behavior in absurd directions. For instance, records used to earn certifications (Platinum, Gold, etc.) based on shipments not sales. Sgt. Pepper’s Lonely Hearts Club Band soundtrack (1978) was a Platinum album but was a sales bust. That’s what happens with an OKR based on shipments, not sales.

To their credit, the RIAA changed the rules for certifications in 1979. That’s what Beane did too. Tavel too. It’s a good reminder to ask: am I using this information because it is helpful or easy?

–
Moneyball might be the best way to win in sports but sports is a story and stories need narrative. I loved the Tim Duncan Spurs but the media didn’t. It’s why there’s only one honest sport.

Weekly active pinners? Hold my beer.

Danny Meyer is an Alchemist (and you can be too)

MIKE1 Comment

“A question that stymied me for years and years, a question I got almost anywhere I went, it seemed like every organization asked, ‘How do you manage to hire so many awesome people?’.

“I said to look for fifty-one percenters, people who are emotionally wired to be happier themselves when they delivery happiness to you.”

Danny Meyer, ILTB

The central idea to Alchemy is to optimize important but overlooked things, with especially large returns from inexpensive yet important finds. How to find these things? Numbers provide a good clue.

When things are easy to measure, they are numerate. These numerate items are easy to discuss, to compare, to enter into spreadsheets then sum, average, and compare again.

Danny Meyer has succeeded (in part) because he competes in new areas. In the beginning of the podcast he tells O’Shaughnessy about competing on food and wine and ambiance and all that, but that’s what everyone does. It’s hard to have THE BEST food when everyone is trying to have that.

THE BEST food has convention. It has history. There are norms. There is price. Having THE BEST food in New York City is like being the best investor in New York. Good luck.

However, being the best at something slightly different is quite a bit easier. There’s a lot more area and a lot less competition if you do things off the beaten path.

Meyer found this in hospitality. Listening, we don’t get the impression that the tag wags the dog, but it’s got to be part of the reason Meyer is around, and talking on the Invest Like the Best Podcast.

Often Alchemy is using (free) psychology rather than (costly) structure. Better service rather than better linens.

One story Meyer tells in the book is about ‘the medicine cabinet’. One establishment was having the normal rumble of friction getting its legs under it and when patrons had a bad time, Meyer and staff offered a glass of dessert wine to soothe their pain.

Not only was the wine complementary, but it was special. At the time, dessert wines were novel so it was a special treat. The kicker was that they were the cheapest wines Meyer stocked.

Alchemy is like improving weaknesses, there is a lot of return for the initial effort, often much-more than optimizing factor f for the tenth time.

Danny Meyer is an alchemist. From the people he hires to the businesses he starts.

You are an alchemist. Find something that’s important but not measured, and deliver that.

Cowen’s Chow Choices

MIKE3 Comments

One local topic during COVID has been motor homes. Some fellow dog walkers want one, some don’t. The obstacle, as often the case, is cost.

A few friends have them and universally they mention the deal they got. It was either a family friend, a distressed seller or a trade-up-buy-out-sale. For us, the math doesn’t work. Thirty-thousand dollars is a lot of nights at a Hampton Inn.

“Buying good things can’t be the secret to success in investing. It has to be the price you pay. It’s not what you buy, it’s what you pay. There’s no asset so good it can’t become overpriced.” – Howard Marks

Great rewards come where value diverges from price. This is the moneyball insight. This is the JTBD insight. This is Tyler Cowen’s Dining Guide insight too.

Where are the wrong metrics being used?

Consider the name of a restaurant suggests Cowen. Would you eat at an Ethiopian restaurant called EYO Sports Bar? Cowen commented: “When I heard that name I thought, this place must be great. When Americans want to eat Ethiopian food what kind of name are they looking for? The Red Sea? Queen of Sheba? Fine. But when it’s EYO Sports bar you know it’s really for Ethiopians.”

In general, better food will be at places with bad names.

Also avoid places on the beaten path, full of beautiful people, and with famous chefs. These are all metrics some people use to choose a restaurant but that don’t necessarily contribute to the quality of the food. It might be good food, but won’t be a good deal.

Instead, use the economic Cowen espouses. Like the name Rus-Uz, a place that serves Russian and Uzbekistan food (and caters!) in Arlington Virginia. Ask, “‘What is the appeal to the masses?’ In relative terms it’s the Russians, so of course that means the Uzbek dishes are better.”

One way to think about metrics is to consider anything that has been quantified, counted, or numbered. It’s easy to count units but hard to count quality.

Part of the reason personal productivity has been an internet subject for so long is that it’s hard to measure. How does someone measure their work? Ask anyone who creates content online and they’ll tell you that it’s the oddest posts that get shared the most. The best productivity advice might just be: don’t give up.

Like this idea? Read more here.

Appropriate Proxies

MIKE5 Comments

 

roryh
Is this art?

One of the best way to find deals in life is to find things where people have attached the wrong proxy metric.

In any systems where we count things (scores, hits, years, etc.) we optimize. People buy more fuel efficient cars rather than consider riding a bike, carpooling to work, or calling an Uber. If it’s counted, it’s prioritized. “Personally, I feel it has gone too far in that direction, and economics has overinvested in one very particular kind of intelligence,” laments Cowen.

How do people find new metrics?

Creativity. It takes a certain worldview to see that there might be a better way. Like the vertical-horizon illusion but for our worldview.

New counting techniques. Sports changed, and is changing, when data collection  (e.g. cameras) changed. Math ability is easy to quantify, so we do.

Career capital. Jason Blum outfitted a van as a mobile office because he spent so much time in Los Angeles traffic. From there he made calls to directors who had recently flopped and asked if they wanted to make movies for his company, Blumhouse Productions. In an industry where ego rules too many decisions, Blum only had to answer to himself.

Ties to history. Since 1987, the NFL combine has been held in Indianapolis. With activities like the 40-yard dash, 20-yard shuffle, and 3-cone drill it’s an annual tradition for fans and front-offices. But those tests have less predictive power than collegiate performance. Yet the show goes on.

First principles. Of the many flaws highlighted in Bad Blood, was a lack of scientific understanding from the Theranos investors. “Since he didn’t have the expertise to vet her scientific claims,” John Carreyrou wrote, “Parloff interviewed the prominent members of her board of directors and effectively relied on them as character witnesses.”

Front lines. Proxies exist to make things easy, but business doesn’t exist on a spreadsheet. Alain Bertaud said ” By living in the cities, and confronted very early, I learned the difference between the theory about a city which you read books and what it means to live in a city.”

Perhaps Rory Sutherland, of course, put it best. Too many people, Rory reasoned, think of art as something you hang on a wall. It’s something famed and famous. It’s by a so-and-so from a certain era. It’s recognizable. That. Is. Art.

Unless it’s not. Architecture is also art, and because people don’t think of homes that way, they may be undervalued. Sutherland thinks so:

“Human decision making is also pretty path-dependent. In one case in my life I’ve been able to profit from this. I live in a house, which in the UK is something called Grade 1 listed. It’s by the great eighteenth century Robert Adam, and the grounds are by Capability Brown. I’m in a four bedroom flat on the roof of a house built for the doctor of George III in about 1785. For a time it was the home of Napoleon III.”

Want more? Check out this pay-what-you-want placebo prescription pdf.

The Right Proxies

MIKE1 Comment

Wharton Moneyball Super Bowl Show.

“I was blessed to know Bill (Belichick) back in college. We worked together for seventeen years. Bill can make complicated game plans but his general principles aren’t very difficult. He had three rules: be on time, pay attention, and work hard. Those seem like simple things but when you’re deaing with players who are entitled, who do things on their own, they have to buy into that system and fall in line. Bill didn’t care how many earings, how many tattoos, how long your hair was. That had nothing to do with discipline.

Scott Pioli

Using proxies can be helpful or not. It all depends how accurately they map to what matters. When Roger Paloff from Fortune Magazine looked into Theranos, he didn’t understand the science and talked to the board members instead. If these smart, accomplished, wealthy people think this makes sense, it must make sense the thinking went.

Other times, proxies are toxic. Often times, it’s for easy-to-measure things. People love the authority of numbers, regardless of how well they map to reality. Another proxy-tally-folly is mistaking action for effectiveness. Regarding productivity, Cal Newport writes, “busyness is not a proxy for productivity.”

It’s impossible to predict the future so we rely on things that, looking back, were present in good outcomes. Sabermetrics using numbers, not if someone looks good in jeans. Belichick avoids appearances too. Those things come to mind easily, but may not be good proxies. 

Ticketing Analytics in 2020

MIKELeave a comment

At SSAC20, Rob Sine, Adam Grove, Kristin Bernert, Patrick Ryan and Shira Springer spoke about ticketing in professional sports, among other areas. A few highlights.

Do we measure what matters? When asked what the opportunities are in the industry, Sine said, “going from season ticket units to revenue which keeps the lights on.” We can imagine a time when season ticket sales were a good proxy for revenue but with the secondary markets, public spaces, and better televisions at home people go to games less. Plus, people are busy. The successful teams will head back to ‘first principles’ and re-focus on revenue.

Who is your customer? “When you look at the data, an account holder for a full season goes to thirty percent of the games, the half season buyer goes to about sixty percent, and the quarter buyer goes about eighty percent. So you’re kinda servicing the same person,” explained Bernert. It’s a case of JTBD. It’s a question of, what are they hiring me to do?

Are there latent needs? When asked if the season ticket is dead, one panelists wonders if they were ever alive. “Teams and venues have always had to recreate what the fans are looking for,” said Sine. Patrick Ryan suggested teams talk to the ushers to hear what the fans are saying—not necessarily what they are asking for.

What are the intangibles? “There’s a lot of pride in being a season ticket holder.” “There’s a great benefit to saying, ‘I was there.'” “A lot of the L.A. Dodgers season ticket holders said the biggest benefit was the person checking them in at the premium station knowing their name, and how that impressed the clients they were with.” The best returns on an investment are the ones with the smallest cost, intangibles are often just that.

Are there latent needs, part 2. One growing request from customers is something Rory Sutherland calls this the airport lounge problem. What some customers want is not one visit on each trip to the airport, but one visit on some trips and a family pass twice a year. Teams like the Orlando Magic are offering this, buying back unused tickets for full face value and allowing that money to be used in the gift shop, concessions, upgrading future tickets, or special events.

One thing that SSAC offers is the chance to hear from people on the ground who may not often speak about their experiences. This was certainly one of those panels. Thanks again to Jessica and Daryl.

Your random fact of the day: Only two college bowl games sold out last year (2019/2020) (45:45).

The Disney Stroller Couple

MIKE3 Comments

It was going to be a long day.

One of the great advantages of living an hour from the Disney World parks is day-trips. Typically we won’t have access to the best attractions, but we’ll have access to many good ones. Plus, seven day-trips over seven weekends is different from a seven-day trip during the week. That’s just basic Alchemy.

Before we moved to Florida, we lived in Ohio and scheduled annual visits. On those trips we had to ‘get our money’s worth’. Tickets might be one-hundred dollars per day per person. Hotel rooms can cost a few hundred dollars too. We bought two ice-cream sandwiches during our last trip and those were twelve dollars.

Flying to Florida and doing Disney costs a lot of money, requires a lot of planning, and introduces some unexpected stress. Keep in mind, this is all a vacation. On our last trip we heard this exchange as a man tried to fold a stroller so he could board a tram.

Him: “This thing is so hard to fold because of all the stuff under here.”

Her: “Welcome to being a dad.”

I felt the frustration like the Florida sun.

…

Disney’s Animal Kingdom has a few fantastic rides, some nice animal viewing areas, and a few of our favorite shows: A Great Bird Adventure, Finding Nemo, and Festival of the Lion King.

At the big finish of Festival of the Lion King, performers choose children from the audience to take a lap around the stage. One girl got to shake hands with Timon, the singing meerkat. If someone can smile with their entire body she did. It was magical.

Many people believe that Disney’s advantage is their IP—but it’s not their only one. Disney magic is an advantage too. These moments for guests don’t happen because of a character but because of a person.

When I walk around Galaxy’s Edge it’s neat—but when you watch Storm Troopers hassel guests it’s great.

If the stroller couple had a magical moment, they’ll be back. Guaranteed. They’ll look back on the trip, and it won’t have been a long day, but a great one.