Testing has been in the spotlight of software development, as organizations continually seek to optimize development cycles. Continuous Delivery and its promises of frequent releases, even as often as hourly, is a big factor driving executives to find ways to shave time off of any eligible technical processes.
As enterprises embark on their DevOps transformational journeys, delivering at lightning speed without compromising on quality takes more than cultural changes and process improvements. Test automation is key in helping project teams write and manage the smallest number of test conditions, but get the maximum coverage.
The case for automation: If the scenario is highly quantitative, technology is superior to humans. That’s clearly the case in the current technology landscape, where according to the 2015 OpenSignal report, massive fragmentation has resulted in over 24,000 distinct Android devices that all run on different versions of the Android OS, plus the several variations of iOS devices currently in use. Web is no different. It’s impossible to test every scenario manually, so automation must be leveraged. But therein lies the main point, that automated testing is a tool to be leveraged as needed instead of relied upon to dictate an entire testing strategy.Although logic would deem that if automating a few test scenarios yield fast results, that automating any and every eligible scenario will shorten the test cycle even more; but alas, that’s usually not how it goes.
Automation endeavors take a good deal of planning and forethought in order to add value at the anticipated level. More often than not the overhead of maintenance, especially in CI/CD environments where tests are automatically triggered and one needs to analyze the reports and fix locators, a task that , could take hours. This won’t work for organizations who are truly DevOps. If full stack developers are going to be fully responsible to take the code to production, then they will need tools to automate the process through infrastructure, testing, deployment and monitoring. This continuous framework enables weekly , daily and hourly releases. Leading DevOps organizations like Netflix and Amazon are said to deploy hundreds to thousands times per day.
What’s more, studies reveal that a high percentage of projects utilizing automated testing fall short of the anticipated ROI or fail altogether. These transformations fall short due to their duration, ramp up time, skill-set gap and maintenance overhead. If the benefits of automated testing aren’t significant enough to mitigate risk then speedy releases could become more of a liability than an asset.
There are varying levels of automation just as there are companies of all different shapes and sizes. Instead of one or the other, it’s more fitting that automated and manual testing are looked at as complementary. Agile and DevOps have created new testing challenges for QA professionals who must meet the requirements for rapid delivery cycles and automated delivery pipelines.
DEMANDS OF CONTINUOUS TESTING
Testing used to have the luxury of having its own phase or at least a set timeframe to stabilize new code prior to pushing to production. No longer. In the era of DevOps, QA must adopt a truly agile mindset and continually be prepared to shift focus to testing new features and functionality as they become available – that could be weekly, daily, or hourly.
From web, mobile and connected devices to IoT, a quick inventory of current technology will reveal a multitude of different devices and literally thousands of potential ways that these technologies can be combined. Across networks, apps, operating systems, browsers and API libraries, the number of combinations with each requiring its own verification and testing support. As the push towards Devops continues, the upward trend towards continuous testing is inevitable. As a result, testers and/or developers will need to deliver in brief and rapid iterations.
Beyond automation, what does this look like? QA, Engineering, or whomever is responsible for testing will need to shift left and engage in rounding out user story acceptance criteria to ensure accuracy and completion. In addition, active participation in sprint plannings will help to ensure that user stories are truly decoupled and independent. Or if dependencies are necessary, confirming that story precursors and relationships are in-place. Partnership with Product Owners to confirm that the scope for a given sprint or iteration is realistic.
And finally in support of shifting left, collaboration among Dev and Ops to ensure the necessary support in the event that a code check-in causes a bunch of automated tests to fail. The culprit for these tests will need to be checked manually so it’s important that QA is prepared so as not to pose a blocker for the release. All of these activities is largely what comprises testing outside of automation. It’s a broader role than the traditional QA Specialist that requires initiative and the willingness to embrace an always-on mentality that’s necessary to support DevOps efforts.
SUMMARY
Software development practices have changed and the change brings with it new challenges for the testing arena. The growing need for test automation in addition to the profusion of new testing scenarios will constantly evolve the role of testing and QA. Yes, the world is changing for testers—in a good way. The days of exhaustive manual testing are certainly numbered, and automated testing is an essential part of the delivery pipeline. But testing has never been more important.
SOURCES
