If you have been in QA for any length of time, you can probably recall a project where the test suite failed every time a new build came out. You’d spend hours trying to read through the test execution report and determine why half of the test cases that ran successfully last week failed the night before. It can be frustrating and stressful to go back and forth with a developer about why the test is suddenly failing and what has changed. A more challenging outcome is that is can create a lack of confidence in the test suite. A perception develops that the test suite is unstable, which can lead to blaming the testing team as the problem.
I recall a large customer in Florida who had a fairly robust suite of automated smoke and functional tests and complained of super-long test cycle times. We reviewed the testing logs only to realize that the test suites were never being run. When we spoke with the customer’s Agile teams, they indicated that they had no confidence in the test suites because half of them failed whenever they were run. The other problem was that no one was responsible for maintaining the automation. Even though the organization had a robust test suite, the lack of confidence rendered it useless.
Had this customer leveraged self-healing automation that could adapt to the application changes, the organization’s confidence level would have been much higher. Using a tool like Tricentis Testim that leverages AI to identify changes in the application and automatically adjust the test case can save the team valuable time and keep test engineers focused on developing new scripts. Incorporating AI and self-healing into automation tools makes automated test case development faster and more efficient, and reduces the amount of time the engineer has to spend doing maintenance. This results in improvement in the engineer experience and job satisfaction.
Conway’s Law
Conway’s Law is a theory created by computer scientist Melvin Conway in the late 1960s. This Law states that the structure of an organization is driven by the structure of the systems it creates. In other words, companies tend to align their organizational structure to the architectural structure of the systems they create. Traditional organizational structures were organized based on platforms or technologies, and were often command and control structures. With the advent of Agile techniques, the focus moved from hierarchical structures toward cross-functional structures, where the skills necessary to deliver systems were aligned in a single team. In addition, Agile brought a concept that Jeff Bezos called the “two-pizza rule,” which reduced the optimal size of a team down to the number of people that could eat two pizzas (8-10 people). We now see smaller teams with a cross-functional structure focused on delivering complete pieces of work. These teams contain the primary skills necessary to build software, including defining, building, and testing skills.
Embracing Agile techniques in the marketplace has given traditional QA resources a seat at the table as an equal participant in the development process. QA is no longer outside of the development value stream, sitting in judgment of quality, but is now an active participant within the value stream. QA teams now have accountability in the feature team for championing quality in the value stream, and they are an advocate for built-in quality.
Traditional QA organizations trying to retain their organizational structure should understand that embracing Agile means a change in focus from quality control to built-in quality. The job of a centralized quality function becomes focused on enablement and overall quality outcomes. Organizations should first focus on building frameworks and toolsets that enable the embedded testing resources to work more efficiently and with higher levels of automation. This requires robust tools that don’t require a narrow set of skills to use and that leverage AI and self-healing to increase resiliency and the robustness.
The Scaled Agile Framework (SAFe) includes the concept of a systems team that’s focused on value enablement. This team focuses on continuous flow and includes the DevOps team, infrastructure as code (IaC), security, end-to-end and integration testing, and the development of the pipelines and frameworks that enable that flow. Even if your organization isn’t doing a form of Agile at scale or Agile at all there are some great lessons to be taken from aligning the testing organization closer to the development value stream, or the sequence of activities necessary to deliver a feature, product, or capability to market. I am not advocating for handing the testers over to a development manager, but for cross-functional teams that deliver end-user value at the speed of business.
Small teams in big companies
As we discussed earlier, with the advent of Agile development we have seen that teams typically have 8-10 cross-functional members. That means that an organization of one hundred would create ten different teams, each aligned to a development value stream. Usually, one or more teams are aligned to a product that a company develops.
Large organizations may align multiple teams to a value stream in a SAFe structure called an Agile Release Train (ART). Substantial companies may have multiple ARTs aligned to large value streams, and each ART typically has a systems team focused on enabling the train to run smoothly. The systems team can support up to ten feature teams with DevOps and testing capabilities. It is critically important to ensure that the foundation for continuous flow is resilient and as robust as possible, so the feature teams can focus on developing value rather than dealing with intermittent test automation and test case maintenance. Leveraging AI to build in self-healing keeps the feature teams from getting distracted by unstable test suites and keeps them pushing forward at the speed of business.
The processes for small teams are the same
Whether you have one feature team with eight members or 10 feature teams, the need for a consistent and stable continuous flow foundation is critical. Inefficiencies and instability reduce the effectiveness of the feature team and steal the what is the most precious commodity for most teams: time. A sound DevOps foundation with built-in quality that leverages AI to optimize robustness will ensure that your quality outcomes are improved. It doesn’t matter whether your organization is small or large: quality matters, and delivering quality at the speed of business is a game-changer.
Leveraging AI for better business outcomes
Artificial intelligence in testing helps organizations move away from tested-in-quality toward a built-in-quality approach. Adding test automation into the pipeline removes waste, wait times, and friction. It enables the automation engineers to focus on developing new scripts rather than constantly updating them. If your organization spends time after every build trying to do root cause analysis on all of the failures, then it is time for you to explore how continuous testing leveraging AI can enable your testing function.
