Practical steps to shift left
Last month we published “Do you know how much your quality costs?” where we discussed the importance of shifting left and shortening the feedback loop as it relates to the cost of quality. According to IBM System Science Institute Relative Cost of Fixing Defects, discovering a bug and fixing a bug in QA would cost about 15X the cost of finding it in the design phase. This is exacerbated even more if that bug is found in production / maintenance to 100X the cost of finding it in the design phase.
Four key ingredients to achieving shift left:
- Test automation – Traditional QA processes would result in months of testing before you get feedback. A productive tester can get about 20 tests done in a single day. Assuming your test suite includes dozens of tests there are two ways to get the feedback loop down to minutes: Hire hundreds of testers or automate your tests. Automated tests don’t get tired, take days off, go out for lunch etc.
- High concurrency – Shifting left means giving near real time feedback i.e minutes. Running dozens of tests, each taking roughly 30-60 seconds, with the expectations of getting the results back in, say 5 min., can only be done if you are running your tests in parallel. The higher your level of parallelism the shorter the feedback loop is. Running your tests in parallel required a grid, a cloud based infrastructure that includes virtual environments allowing you to spin in and out different browsers.
- Build-test-deploy automation – Shifting left means you are optimizing the handover across the different stages of the development process. The development cycle includes dozens of handovers and human intervention can always delay that process hence you want to automate the handover using Continuous Integration (CI) servers. CI servers monitor events through your development cycle, such as a new build, and can be programed to trigger a set of actions once a certain event occurred, such as triggering your automated tests.
- Suites – It is best practice to optimize your quality assurance processes by segmenting your tests into suites and running different suites at different stages of your development cycle. For example you full regression might include thousands of tests. Running all those tests every time a develop push code would be time and resource consuming. If 2-3 key scenarios failed, there is no point in waiting for the entire suite to be over. You might want to create a small subset of tests, composed of your high priority flows, and run that “sanity” suite often, while running the more extensive suites once if sanity suite ran successfully.
