The Lean Startup

In order to write this review, I read this book for a third time and combed through it making notes, and I still think it's pretty poo. And before anyone tries to defend it, I ask two things: 1. Where did you get your participants from for experimenting and measuring? 2. How did you ensure statistical power? The book preaches a scientific approach to business, but doesn't focus on the fundamental parts of science, such as experimental design and statistics. Where are you getting your samples from?

Build, measure, learn. It should be called building confidence around false positives. The book has chapter headings, but in many ways it's very unstructured. It's essentially a handful of cherry-picked stories that cite subjective accounts of why some businesses were successful. But it's pitched as a repeatable scientific process that raises a business's chance of success.

It's also so long-winded. Eric Ries gives some advice about process and then tells a story that doesn't really make his point. Which is a tell in itself. The subtext is this: perhaps there is no repeatable process. The world is messy. Business success is complex, and luck will often go further than skill. However, skill means you're less likely to mess up any luck you get.

The book attempts to apply a scientific method to developing a business. But the chapters on measurement make anyone with a basic statistical background cringe. Either the book encourages false positives, or Eric always had a massive budget to drive enough traffic to prove a hypothesis. Or he's in favour of extremely high risks and only comparing quick, drastically different splits, where the difference is large enough that he's able to get the participant rate down and achieve a minimum detectable effect on a smaller budget.

And, to be honest, I'd be fine with this. BUT, instead of giving an example like: "In order to reduce costs and raise the chance of finding a significant difference between two marketing funnels, you must make sure your independent variables are drastically different," he gives you a 15-minute account of a company that did A/B testing and never says what they actually measured, lol!

It's so annoying. Every time I think we might get somewhere, he wraps up and says something broad like, "Company X was able to find a product that worked by using A/B testing." All fluff, no substance.

He touts stories of companies like Facebook as examples, but they didn't do what he's saying. So it's like an afterthought: here's a story that fits my idea. And then he doesn't account for the biggest factors, like right place, right time. There's no mention of this. He's trying to make out that people have control over macro events and viral growth. Try starting your own Facebook now, lol.

Moreover, it only loosely talks about participant/customer recruitment. In the book, participants often seem to appear out of nowhere.

Finding early adopters, taking risks, spending money, and putting yourself out there warrants a book in itself and, to my mind, would be much more valuable. So while the book has some nice ideas, it's not practical or critically minded. It's loose where it needs to be clinical, and detailed where it needs to be concise.

It focuses on multiple case studies as examples of start-ups and new divisions within companies that have been successful. However, they are really poor examples of any kind of repeatable, measurable process. He has a chapter on innovation accounting that is farcical.

However, if you can push through the laborious nature of stories that never go anywhere, you will find a few diamonds in the rough.

If you don't want to read the book, here's what I think it gets right, and I'll save you quite a few hours:

• Don't try to perfect your offering. Launch now with an MVP, or even a smoke test, to lower risk and start pivoting as early as needed.
• Focus on metrics like sales and customer retention/repeat buys. Forget clicks, views, likes, and registered users (who cares?). These are not customers. Unless your business model is advertising, these metrics are almost useless if they don't have a clear path to sales.
• Double down on what's working.
• Pivot if something is not working; persevere if something is working.
• Don't batch; it's too much upfront. Stay agile, start with one thing at a time, and iterate.
• Speak to customers and work with customers to improve.
• Growth happens when the rate of new customer acquisition outpaces the churn rate.
• Word of mouth or network effects can mean growth happens automatically as a side effect of users simply using the product.
• Growth happens when the Lifetime Value (LTV) of a customer is significantly higher than the Customer Acquisition Cost (CAC), allowing profits to be reinvested into ads.
• Overcome issues by continuously asking why, to discover the root cause of any issue.

Something else it mentions that I believe in:

The Toyota way: Genchi Genbutsu - go and see for yourself firsthand. You cannot be sure you understand unless you go and see for yourself firsthand. It is unreliable to take anything for granted or rely on the reports of others.