Imagine this scenario. You are writing code for a new insurance web app that you’re building and suddenly come across a stumbling block. You realize that using Google Sheets as your back-end database is a clever hack and can be implemented within just a couple of hours. While this won’t cause any performance issues now, Google Sheets will likely not be a great solution once you exceed 10 million rows. The other alternative is to set up a new MongoDB database. This is faster in terms of performance and can easily scale to billions of entries. However, due to the constraints of your runtime environment setting it up and configuring it might take a couple of days. Which option do you choose?
Most teams at large corporates will likely go for the latter option. Given the security and infosec considerations, they will likely also have a 1 month project to debate both architectural choices and get the final choice approved by various committees. However, most startups will choose the former option (Google Sheets) - no questions asked.
Both choices have clear merits and demerits so there is not necessarily some right answer. However, I believe that Google Sheets should be the default choice for anyone. Most projects don’t get to 10 million rows so that is a luxury that few get to enjoy. Moreover, by the time most projects get to that stage, the project often evolves so much that if there is technical debt, it likely isn’t just in your database choice.
Computer science has actually a term for this whole scenario: premature optimization. Simply put, it refers the idea that developers get tempted to spend a lot of time doing things that they might actually not need. The concept is a powerful one and has implications beyond computer science to the business world. Public companies engage in premature optimization of all sorts all the time. A few examples of things that might qualify as premature optimization in a business context are:
Setting up a board and reporting structure before you have an MVP for your product
Doing a 5-year business plan for something that isn’t launched yet
Spending >$10k/month on paid marketing before you have achieved product-market fit
Getting a patent for your brand-name when the product is being launched
Many more examples come to mind but I assume you get the point. Startups generally tend to engage in a lot less premature optimization as capital, time and resources are all generally constrained. As a result, their speed-to-market and ability to iterate is much quicker. Most public corporations often almost engage in tonnes of premature optimization before launching newer products and innovations. That is actually one main reason corporates tend to be less innovative.
There are a few organizations that come to mind that seem to have avoided this trap. Amazon is first among them. However, an idea from the Japanese Amazon, Rakuten, is perhaps the best way for organizations to think about scaling and how too approach such isuses. It is called ‘The Rule of 3 and 10’. Simply put, it is the idea that your organization changes as it grows to 3x and 10x its current size. So while it is important to plan with these milestones in mind, your organizational structure and processes will invariably change as it expands. Operating like an organization of 1000 when you’re a team of 10 is illogical and premature. As is operating with the same reporting lines that you had when you were 10 people (if you’ve grown to 200+ people). i.e. Do not engage in premature optimization.
Here is the full piece from Philip Libin’s interview with Sequoia which explains the concept much more eloquently:
One of the best pieces of advice I ever received came from Hiroshi Mikitani, the CEO of Rakuten, the Japanese retailer. He said that everything changes at roughly every third and tenth steps.
When you go from one person to three people it’s different. When it’s just you, you know what you are doing and then you have three people and you have to rethink how you are doing everything. But when there are 10 people it’s all going to change again. And when there are 30 people it will change again. Same when you reach 100 people.
At every one of those steps everything kind of breaks. Everything. Your communication systems and your payroll and your accounting and customer support. Everything that you put into place needs to change when you put in those three and 10 steps.
When you get into trouble is when you skip one of those steps without really thinking about it. So we’re at a little over 300 people now. There are still processes and procedures that we put in place when we were 30 people. The way that we onboard people hasn’t really changed--we still bring people on board and just throw them into the deep end of the pool. And that works great when you are 30 people, but it really doesn’t at 300.
By the time you get to 1,000 people you are likely going to stay there for years. When you get to 10,000 you might stay there forever. But as a startup when you are growing from three, to 10 in a short amount of time it is very easy to skip a few steps and then end up with all sorts of broken stuff.
There’s nothing magic about three and 10, but it’s been true for us. The structures we are putting in now should get us to be 1000 people.
To receive more posts like this, sign up for our daily newsletter!