Don't be the slowest gazelle

Every morning in Africa, a gazelle wakes up. It knows it must outrun the fastest lion or it will be killed. Every morning in Africa, a lion wakes up. It knows it must run faster than the slowest gazelle, or it will starve. It doesn't matter whether you're a lion or a gazelle -- when the sun comes up, you'd better be running.
-- proverb attributed to Roger Bannister (first man to break the 4 minute mile)

At a startup, there are inevitably moments when you question whether or not you're moving too fast. From defects, regressions, and poor performance to missed requirements, and reimplemented requirements. Now you have more customers, you're investing more in marketing, you can't afford the mistakes you used to be able to tolerate. Your startup team wants to begin establishing slower, more measured process for greater predictability and fewer mistakes. You might think about hiring a more "experienced" program manager or project manager, maybe you think about moving to a waterfall process. Although it seems counter-intuitive, you must not slow down... you need to speed up. In his post, Speed up or slow down? lean startup evangelist Eric Ries refers to this as the speed-up-or-slow-down-moment. Eric advocates for speeding up -- "The day-to-day process that startups build should also attempt to maximize speed of learning." Not only do I strongly agree with the need to speed up, I believe you need to speed up regardless of your company's current size, age or market cap. You need to focus on staying lean and agile -- or suffer the fate of the slowest gazelle...

I'd like to use Eric's broadened definition of a startup, with minor modification:

"A startup is a human institution designed to create a new product or service under conditions of extreme uncertainty."
-- Eric Ries

I've struck out new. While this may be a useful modifier for the traditional view of a startup, I strongly believe it is not a precondition for applying many of the lean startup principles. All businesses, regardless of product or service (old or new), are operating under conditions of extreme uncertainty. Over half the companies in the Dow Jones Industrial Average, still a key benchmark for performance of the US Equities Market, were added in just the past 20 years. Even if you think you've moved beyond startup classification, there are lions stalking you -- competitors, regulators, patent trolls, your customers' whims. You need to be faster than all of them.

In 2007, at my previous company Vonage, we were found to be infringing on Verizon's patents. Verizon sought and won an injunction from our use of those patents -- which included technology at the core of Vonage's call processing infrastructure. I will abstain for now from editorializing on the sanity of these rulings but, I wrote about my general sentiments on a similar patent issue here. Vonage managed to appeal and receive a stay for the injunction. This stay set the timeline for our designing patent workarounds. A patent workaround is basically a new way of getting something to work that can not be read on the patent. A judge then needs to approve the workaround. If you think involving business owners in designing software is hard, try working with lawyers... Then, drop a judge in as your QA team. Only if the judge doesn't approve your workarounds, it is basically lights out. Goodbye 2mil customers; farewell $50mil monthly revenues; goodnight future potential for pivots.

Prior to beginning the patent workarounds, it took months to deploy substantial changes to our call processing system. The planned deployment cycle was 6-8 weeks; 3-4 weeks for QA plus another 3-4 weeks to deploy. Inevitably, between week 6 and week 8, a critical defect would be found (memory leak, call loop, etc), the code would be rolled back and the cycle would begin again.

Gaining agreement to move faster with code deploys with QA and Operations was initially met with antagonism - "well, if your coders didn't put any bugs in the code, it would get deployed faster." Once our survival as a company was inextricably tied to our ability to modify and deploy our core processing system, that agreement became substantially less complicated. We had already begun to develop code quality testing and automated integration tests. The manager of the team ran to Best Buy for a switch to build a network that the development team could run automated deployments on. We worked with the QA team to define a < 1 work day set of tests to be ran for the most frequently used features. Then, we (substantially) automated the deployment and production integration testing of the call processing system.

In around 2 months, we went from taking months to deploy the system, to at the height of the workaround madness, deploying the entire system of ~100 machines in less than 24 hours -- for a system supporting 2 million customers making calls next door, across continents, to the other side of the world and most importantly to 911. All systems have bugs and defects, but, when you can fix those bugs and redeploy on a daily basis the impact of any one bug is substantially reduced. Most importantly, we learned what we needed to do to release quickly and often, with lower stress and higher quality thanks to fewer regressions, better automated testing and faster identification of bugs caused by rare or edge cases. By hitting those edge cases early on in the process, we knew we had a stable base to build from if there was a ruling that required any new change.

Being able to "run faster" helped ensure Vonage's future in a very uncertain time. No matter how big your company is, there is no way of knowing where the "lion" is going to come from. Rest assured they're out there; will you be ready to outrun them when they pounce? For a technology company, choosing to speed up improves the likelihood you'll be able to outrun your competitors, react to regulators, work around predatory patent litigation and most importantly react to your customers' needs. Don't be the slowest gazelle.

* As a footnote, Vonage did end up settling with Verizon before the workarounds came to a court decision. Even though we were confident we could quickly modify our code to whatever specs were needed to qualify for a workaround, in an appeals situation you're not guaranteed the judge will accept your workaround. A single day without using our call processing technology would have essentially put Vonage out of business.