What is Test Architecture?
Test architecture can be thought of at two levels. At the broader, more strategic level, test architecture is thinking holistically about the best way to test an application. It can help answer questions, such as, how do we want to use unit, integration, and end-to-end tests to gain coverage? What tests should be automated versus manual? What tooling or frameworks are we going to use?
Test architecture can also be thought of at a more technical, detailed design level. Here it can help address questions such as, what are the priority areas of the application we need to verify? How do we construct our tests to gain maximum coverage with minimal duplication? What data are we going to use?
As the complexity of modern applications grows, teams need to test more efficiently. They need a test strategy that rapidly delivers feedback at different stages of development, covers the highest risks of the application, and doesn’t require an army of testers to keep it running.
Although QA teams can automate much of software testing, it isn’t an entirely autonomous activity. Instead, it’s a predefined step-by-step process that you need to follow to ensure that the particular product you’re about to launch works as expected.
This post aims to highlight some aspects of a test architecture that can help teams build quality from the bottom up and scale their testing more efficiently and effectively.
The Testing Pyramid
The testing pyramid is a framework often used as a guide for test architecture. It’s a simple yet powerful model. First, use more unit tests (the bottom of the pyramid) because they provide the quickest feedback. Next, use fewer integration tests (the middle layer) to validate that components of the application work together. Finally, the top of the pyramid includes UI and end-to-end tests that help validate the user experience and are used more sparingly. They take more effort to write and maintain and deliver the slowest feedback. For more on the testing pyramid, read this.
Expand Your Test Coverage
Applying the testing pyramid framework requires a solid understanding of the application under test. For example, do you really need to test a component five times because it’s used on five pages? If it’s been tested at the unit and integration test level, perhaps one UI / E2E scenario through that component is sufficient.
Of course, you should also consider other factors in your application of the pyramid to your test architecture, such as the skills available for testing or the risks of different parts of the application. For instance, if your application has a part that changes frequently, you will want to apply more layers of testing than if it is relatively static. If your UI testers aren’t knowledgeable of the application or don’t understand how components are reused in the application, then you might need to rely more heavily on the unit and integration tests written by developers.
Test Automation
Automation should be a critical ingredient of your test architecture to keep costs down and prevent regression defects from escaping to production.
Web applications have grown significantly in recent years—both in terms of business importance and also as measured by features or lines of code. With any software product, making changes can have unintended impacts (i.e., regressions). To guard against regressions, teams need to test areas of the application that might not seem closely related but could nonetheless fail due to new code changes. This is extremely time-consuming, and when performed manually, it can add days to release cycles.
Automated testing allows you to achieve a level of detail you could never achieve with manual testing. With automation, you can execute more tests much faster as compared to manual testing. Also, it eliminates repetitive tasks that could lead humans to make mistakes. Automated test cases are reusable, and you can repeat the steps of a test case without variation. You can identify more problems, receive feedback sooner, and bring cycle times down compared to manual testing. Find more test automation benefits here.
Test Design
Test design is a bigger topic, but you can find some tips in this blog. For now, I’ll spend I’ll mention some areas to think about that can align with test architecture goals.
First, automated tests, whether written in code or authored in a low code tool, should follow coding best practices. For instance, writing smaller tests even at the UI level can provide you with faster, more definitive feedback. They can also make maintenance easier if they are written to align with frequently updated features. For instance, an e-commerce app might offer in-app promotions that change often. Including the promotions in broader test cases can increase maintenance. Another way to minimize the test updates of frequently changing features is to put them in reusable groups that can be updated once for all dependent tests.
Another area of test design is to have simple and more extensive test cases for mandatory functions. For example, most applications shave a log-in function. You will want some tests that check whether a valid user can access the system and others that attempt to log in with invalid credentials. You can do this in a single test with a few data variables. On the other hand, if you must first log in to the application under test before testing any of its functions, you might create a second simple login test that only uses approved test credentials as a shared group across other tests.
What About Scaling Your QA Team?
For many teams, scaling test automation is an afterthought. Instead, they worry about scaling their automation project when it starts to get out of control. Too often, the fix is to add people or time. Unfortunately, the business can’t wait, and you have to adapt your testing strategy to keep up with constant product changes while maintaining your team’s effectiveness as it grows.
As you grow automated tests, increase team size, or add software releases, scaling becomes more difficult. You’ll need tools, processes, and controls to help you maintain efficiencies and effectiveness.
Test Operations or Simply TestOps
TestOps is “the discipline of managing and scaling test automation people, processes, and tests to maximize efficiency, delivery speed, and application quality.” TestOps aims to improve the way testing is perceived and implemented. TestOps consists of four main components: planning, control, management, and insights.
Planning
Planning takes place early, all stakeholders take part, and it’s an ongoing process. With early and thorough planning, teams understand better what to test, who should run the tests, and the expected outcomes.
Control
Tests are like code and should have controls in place to ensure that authorized tests are. Control includes standardizing processes for changes to test cases, clearly defining what different team members can do, and motivating everyone to follow best practices.
Management
Tests need to be extremely well-organized so team members can stay productive. It would help if you also had tools to distribute the workload to know where to focus, and management can hold them accountable. Management can also extend to the test’s lifecycle—is the test in an evaluating stage, active in the CI, or perhaps quarantined until it’s fixed. Treating the test differently based on its status helps build trust that test failures represent issues in the application and not just flaky tests.
Insights
The final component is insights. Knowing that your test passed or failed is a minimum requirement for any test. To scale efficiently, you must also know the quality of your release, how your quality is trending, and how you can improve. You’ll need insights to help identify failure trends, duplication levels, and team members who may need more training. In general, you can better understand how you perform, why you’re performing well in some areas, and what areas need improvement.
You can learn more about how Testim helps teams scale smarter here.
Conclusion
Although software testing has evolved through the years, it still struggles to keep up with the pace of software development. Even large companies often deploy software before having final test results. A solid test architecture can help you determine the right tests to perform and at what time. In addition, it will set your quality program to scale more efficiently as your application grows.
