We share some tactical strategies to tackling estimation questions :)
You’ve probably heard some of the crazy questions Google (and other tech companies) famously asked in interviews:
What does that have to do with search engines? How would I even begin to figure these things out? Doesn’t Google exist to help you look up this kind of thing if you need to?
You get three typical responses to a question like this:
Throw up your hands and say “that’s impossible”
Try to look up the answer somewhere
Ask for more information
These questions were perceived as so tough and unfair, that the company eventually banned them. However it’s still worth learning how to solve this type of problem — and not just for interviews — as many real world problems can be solved with this method.
The dirty little secret is that Google (and everyone else) still asks these questions, they just make them sound less whacky by relating them to actual business problems, like “estimate the cost to Google to provide free storage to all Pixel phone users”.
So what’s the response the interviewer is looking for? The fourth option — start making educated guesses. Wait, isn’t it wrong to guess without asking clarifying questions? Actually no — typically the interviewer won’t provide any clarifying information so asking for more information is seen as a distraction or stalling technique. So by asking questions you’re basically admitting you aren’t in on the game, or don’t ‘think the right way’ about solving problems.
If you recognize these questions as Fermi problems, you’re already halfway there. That’s why asking these questions can be a little unfair — most people have never heard of Fermi, and have been trained by the education system to try and get every answer exactly right. But that’s not how this game is played.
Enrico Fermi was a physicist famous for being able to make accurate measurements with little or no actual data — like the time he estimated the strength of a atomic bomb test (a highly classified secret at the time) by dropping a few scraps of paper and making a few rough calculations in his head.
How did he do it? Well by observing how far the scraps of paper traveled at the time of the blast, making a few assumptions, and understanding how to calculate the change in volume, he guessed right within 10 kilotonnes.
This method has been used in all sorts of domains since, including helping us calculate the number of potential alien civilizations in the galaxy. Believe it or not, it’s used every day by Product Managers, Engineers, Marketers and Business people to model out the potential impact of decisions before they're made.
When you’ve been trained your whole life to take careful accurate measurements for your calculations, it can be hard to believe these sloppy guesses can be useful in any way. But in real life, we rarely need to calculate a number precisely — and there is a cost to more precision.
For example, take one of the most famous Fermi questions — “How many Piano Tuners are there in New York City”. If you were building software for Piano Tuners to book appointments, would it really matter if the exact answer is 27 and you guessed 28? Not at all!
But if you could be reasonably confident that there were less than 50 or closer to 500, you’d know whether there was a big enough market without incurring the cost of building your product. Sure you could turn to Yelp and see there are about 55, but what about the ones not listed? Do we count the ones that can tune a piano but only do it for friends and family? Have some of these listings gone out of business? What about businesses that have more than one Piano Tuner on staff?
To get a truly accurate number we’d have to do an inordinate amount of research, investigating each listing one by one and tracking down anyone who can tune a piano — it’d be prohibitively expensive and maybe even be impossible to get the true number. Thankfully we don’t need it — we can make a series of educated guesses and get to a ‘good enough’ number right now.
And do you know what, weirdly Fermi estimates actually do end up pretty accurate. Maybe it’s the brain’s remarkable ability to make rapid estimates, or potentially with natural variance our over-estimations end up cancelling out our under estimations. But when you do enough of these, be prepared to be quietly impressed with how accurate your numbers get.
Let’s tackle the Piano Tuner question together, so you see how it works. The key is to break the problem into smaller steps. For example to arrive at an estimate we need to guess the population of New York city, how many people own Pianos, how many appointments a piano tuner can keep in a day, amongst other things. As you break these steps down, fill in your best guesses for each stage.
According to Google the real number is 8.3, but assuming you’re on the spot you can go with 8. What matters is not the 0.3 difference, but knowing it’s more than 5 million and less than 10.
To get to this assumption, I remembered that there are 2.2 kids on average per couple (family size = 4.4), and then assumed that roughly half of people had no kids (2 + 4.2 / 2 = 3.1). I wasn’t far off — the United States has an average household size of 2.6.
This is just a wild guess, based on personal experience, so this was the estimate I was least confident on. I arrived here by thinking about which of my friends' families owned pianos — I could only think of 1 in 20, so I guessed 5%. I struggled to find good data on this, other than only 30,000 pianos are made every year, down from 100,000 in 2005. So it’s definitely not going to be north of 10% with a population of 300 million, even if we assume pianos last a long time.
How many appointments can they get to in a day? I guessed 4 here, thinking it’d take about an hour to tune the piano, and then accounted for travel time. The true figure is about 4 to 6 — and it’s ok for us to be on the low side as some piano tuners won’t be doing this full time.
Now we have all the main assumptions we need to make to figure this out. You can hopefully see we’re allowing ourselves to be relaxed about our assumptions — we can of course Google each one and plug in the real numbers if we prefer, but we actually got pretty close with our guesses in this case.
If you plug in these assumptions to an Excel or Google sheet, this is what you get:
We guessed 167 piano tuners in New York City — Yelp says there are 55, but it’s reasonable to assume that some of these listings employ multiple tuners, and when you account for anyone that might be unlisted, we’re not far off!
The important takeaway here is that in a domain that we know nothing about, with some reasonable guesses and assumptions, and simple math, we can arrive at a realistic estimate of the number within an order of magnitude of the real number. If our new business venture or software product needed more than 1,000 piano tuner customers to break even, we now know that’s wishful thinking.
Let’s run through another example, this time covering a real life Fermi question Google continues to ask in their interview, that would be useful after you got the job too — “How much would it cost Google to store all photos taken on Android phones this year?”
To answer this question, you run through exactly the same process:
Break the problem into smaller steps
Make the key assumptions needed
Calculate the final number
The first key assumption you’d need to make here are the number of people who have an Android phone (work backwards from Android market share, smartphone penetration and World population). You’d also need an estimate for how many photos people take per day, and the size of those photos, as well as some assumptions on Google’s cost to store a gigabyte. Here’s how it could all look when you’re done.
Now we can’t really check the actual value (probably this isn’t even disclosed to most Google employees), but I guarantee you — give an answer like this in an interview, explain your reasoning as you go, and the hiring manager can’t help but see value.
The great thing about making a Fermi estimation and being wrong, is that you learn something very precise. Because the method forced you to break the problem into smaller steps, and put your assumptions down on paper, you now know which exact assumption was wrong, and by how much. This is a rare moment where you can see clearly what you thought before, and how far you were off from reality — this is a fantastic learning opportunity and can really help inform your future strategy.
For example, if you were launching a new mobile app game, you could estimate the number of users you would get in year one. You’re confident you’re making a game so good it’ll go viral — the first initial test users will tell their friends, who will tell their friends, and so on. You can actually model this out, by making a series of assumptions on number of invites per user, and conversion rate from invite.
Now before you launch your app you know what needs to be true to hit your numbers. Compare your assumptions to popular benchmarks, and see if it’s even going to be realistic to achieve your goals. Commonly you find out that even in the rosiest scenarios you can’t possibly make enough money from this project to be worth it — you just saved yourself a lot of heartbreak.
After launch, if your initial data shows users are only inviting 5 people each, or the conversion rate on the app store page is lower, you have an early warning you’ll be way off. It also helps you concentrate on fixing the right problem.
If all your other assumptions prove true, except your app store conversion rate is 20% instead of 40%, you know you need to focus on it — rewrite the copy, add compelling new screenshots of the app, or add testimonials, awards and quotes from the press.
Hopefully by now you can see the power of these back-of-the-napkin calculations. This isn’t just a hack or useful method, but a whole new way of thinking. By being willing to be imprecise, you’re free to get to a ‘good enough’ answer and move on.
One big warning — this method works for most problems, but does tend to break down where virality is involved. You can see in the model below how small changes can make a huge difference due to exponentiality. If your job is placing bets on exponential growth, you should err on the side of generosity, lest you pass on the next Facebook, like Andrew Chen famously did.
Want more practice with Fermi questions? Check out this spreadsheet where you can reverse-engineer all the problems mentioned in this post (and many more!).
Find out how much you’re worth and how to ask for more — the right way.