Do you how much quality costs? Have you ever calculated it? If you do, you’ll learn that you spend thousands of dollars and hours of productivity on every bug. From the cost of labor, to the cost of deployment, to the cost of finding the bugs in production, no bug fix is free.
Many companies and academics, from IBM to Microsoft to large banks have performed studies to quantify the cost of finding bugs. It’s been well known for many years that the costs go up the closer to production you go, but the actual numbers are a bit staggering.Their findings show the dramatic financial difference between discovering a bug in the wild, and finding it during development.
- Fixing a bug in production – Very expensive. These take time to fix, and while repairs take place, your business experiences pain. Your teams are working around the clock to fix something that also requires your developer to rework something they wrote months ago and forgot about.
- Fixing a bug in QA – Somewhat expensive. The QA person has to report the bug. A senior manager has to determine who to be assigned to this. The developer has to step out of his daily tasks, sometimes impacting the current sprint, attend to the bug, reproduce it, and return to edit some code from the last sprint, which the developer has likely already begun to forget about.
- Fixing a bug in development – Least expensive. takes a few minutes for the developer to fix. If you have an immediate feedback loop, your developer will get feedback minutes after pushing code. If a test fails, it is still within the current sprint and the developer should recognize the code and its implications as everything is still fresh in their mind. The bug didn’t escalate and the likelihood of on time release is much higher.
Shift left is the term coined for a software development cycle that includes a constant and immediate feedback loop. You may have heard stories about how Facebook has put a billion dollars worth of engineering into creating this feedback loop. The company even wrote a PHP-to-C compiler called HipHop to speed up the loading of its test environments.
If Facebook is willing to spend a billion dollars to get that feedback loop, you can spend a few weeks ensuring your own loop is as fast and reliable as possible. This means fast builds, fast test batteries, and short amounts of time between developer compiling, and developer testing.
This requires certain infrastructure that we’ll get to in a future post but organizations that have deployed it, have seen decrease in cost of quality and increase in on-time delivery. You may even have to disassemble your build pipeline to replace a few slower moving items.
Your goal, here, should be one minute or less between build and test. This may seem impossible, or it may already be the way things work for your team. This can require a lot of work, or a little depending on the existing systems. No matter the environment, there is one fundamental truth for all software development: the faster the developer can see their results in a running binary, the faster they can fix problems and the lower the cost of overall quality.
Want to know how to shift left? Stay tuned.
Sources:
