The software testing world sometimes includes too much confusing jargon. Many expressions sound similar but refer to different concepts. The “test strategy vs. test plan” dilemma is a good example.
What is the meaning of each one of those terms? Why do you need them? How do they differ? And what are their similarities? These are the kinds of questions this post will answer.
We’ll begin by covering the difference between a test strategy and a test plan at a high level. Then, we’ll cover each term in turn, detailing what the concept means and why it’s useful. After that, we’ll present a comparison between the two concepts, followed by a discussion of the interplay between the two terms and test automation.
Before parting ways with some final considerations, we’ll give you a summarized comparison between a test strategy and a test plan in the form of a table.
Test Strategy vs. Test Plan: a High-Level View
Understanding the difference between a test strategy and a test plan lies in understanding the difference between strategy and tactics.
A strategy is, by definition, very high level. It defines a vision—usually a long-term one—to achieve one or more goals. A lot of effort goes into the creation of a strategy, which can include extensive research. On the other hand, tactics are the actual things you’ll do to achieve the desired objectives. They originate from your strategy.
This article shows a nice example of strategy and tactics, featuring a bicycle manufacturer:
Strategy: Differentiation
We will succeed by creating bikes with innovative features which our competitors cannot match, and which allow us to charge a price premium to customers.
Objective: Be recognized by the industry as the leading innovator in the space of on-bike health tracking technology.
Tactic 1: Launch a new Bluetooth-enabled module that attaches to the spokes of the bike and sends performance data to a linked smartphone app.
Tactic 2: Send out free bikes to popular tech journalists inviting them to write a review of the bike.
As you can see, a single strategy gave origin to two distinct tactics that can be put into practice to achieve the desired goal. Going back to the software testing domain, it becomes clear that while a test strategy is obviously a strategy, test plans are tactics. What does that look like in practice, though?
Test Strategy: Definition and Application
A test strategy is a strategy for software testing adopted in an organization. Think of it as a document outlining the overall approach the organization will adopt and defining high-level guidelines that will direct all decision-making related to testing and QA.
Why does your organization need a test strategy? A strategy helps you address key risks while minimizing effort or costs. With a well-thought-out test strategy, an organization can answer questions like the following:
- How do we provide fast feedback on quality to developers?
- What is the balance between the different types of tests we want to utilize, including manual vs. automated?
- How often should we execute our tests?
- How do we know if we are succeeding in our quality objectives?
Expand Your Test Coverage
What Is a Test Plan? Why Do You Need One?
Your test strategy is your “north star,” giving you high-level guidance regarding your test-related decision-making. Eventually, though, software tests will actually need to be performed. A test strategy doesn’t tell you the specifics of how you’ll be testing your applications. And that wouldn’t even be possible anyway.
Remember, a test strategy is high-level and general. Each application, however, might have different testing needs (think of a brand new application vs. an existing application, for instance.)
That’s where test plans come in handy. A test plan is a detailed itinerary of how software testing will be carried out to a given application. It defines, in more detail, how the test activities will be performed. That might include the tools that will be used, specific instructions on reproducing an error, or details on obtaining adequate test data.
The test plan might contain one or more test suites, which, in turn, will contain several test cases.
Time for a Comparison
So how do a test strategy and a test plan differ?
The first main difference is that a test strategy is usually defined once for the whole organization. As such, the test strategy serves as the foundation upon which specific test plans are created. So a test strategy doesn’t usually change very frequently.
On the other hand, test plans can change from release to release for a variety of reasons. For instance, if the new release only made some cosmetic changes, you might be more concerned with UI, visual, or exploratory testing. However, if the application was refactored on a new database or used a new algorithm, you likely want to provide full regression testing at multiple layers.
Since it resides at the organizational level, a test strategy affects all projects carried out by the organization. On the other hand, test plans are usually tied to a single project due to the different testing needs of each application.
Plans and Strategies in the Era of Test Automation
Test automation is essential for organizations that want to deliver high-quality software at a fast pace. So it’s no surprise to see organizations automating more and more activities in software testing.
Does this trend affect how test strategies and plans work? It certainly does. For starters, the portion of manual testing in most software strategies is bound to decline. Sure, testing types such as exploratory testing still play an important role in certain scenarios.
For many organizations, it still makes sense to try and find the optimal amount of manual testing to have. However, as test automation tools advance, the territory of what can’t be automated continues to diminish.
Also, some naming confusion is inevitable. There are automated testing tools—unit test frameworks, for instance—that employ terms like test plans and test suites with slightly different meanings than the ones you’ve learned about today.
Test automation affects test strategies in yet another important way. Namely, in the choice of which types of automated testing to use, and how much effort you should resort to each one. As you’re probably aware, test automation comes in many different types, and it isn’t always clear which ones to choose.
The good news is that there are helpful guides out there, such as the concept of the testing pyramid.
Last but not least: let’s talk money. Test automation is an investment, and you should treat it as such. That includes reasoning about its ROI (return on investment.)
When assessing the effort you should put on adding test automation to a previously untested codebase, understanding the ROI of test automation will help you decide, for instance, between going for full coverage or opting for a risk-based approach.
Test Strategies, Test Plans, And Who’s Responsible For What
In another post, we argued that in a modern software organization, testing is everyone’s responsibility. And we went further, arguing that’s a good thing. What are the implications of this line of thinking when it comes to test strategies and test plans?
First of all, we’re not saying you should fire your QA staff or anything like that. What we are saying is that modern organizations should develop test strategies that acknowledge the fact that everyone in the organization should own testing. Developers shouldn’t fear QA, and neither should non-technical people refrain from creating tests using codeless test automation tools.
The software testing life cycle in a modern organization offers plenty of opportunity for everyone to get involved with testing:
- Developers write and run unit tests as part of their daily workflows—they either use or don’t use TDD (test-driven development)
- QA analysists and testers write integration and end-to-end tests and testers might perform manual exploratory tests.
- Business analysts might write acceptance tests, specially when the organization uses BDD (behavior-driven development) tests
All of the actors above will probably be involved in the creation of test plans at some point. The specifics of how those plans are expressed depend on the type of testing, the knowledge level of the person and the project to be tested.
On the other hand, strategy is usually concentrated on the hands of the leadership.
Test Strategy vs. Test Plan: a Summarized Comparison
Here’s the summary of the comparison we made:
| Test Strategy | Test Plan |
| Defines the high-level objectives and guidelines of software testing. | Defines the specifics of how testing will happen. |
| Usually resides at the organizational level. | Usually, a test plan per project. |
| Doesn’t change often. | It can change more often due to various reasons. |
| It affects many projects since it’s valid for the whole organization. | Affects only a single project. |
Scale up Your Test Strategy With TestOps!
A test strategy is simply essential for any organization that cares about quality. With a strategy in place, you’ll be able to create test plans that simultaneously reflect the company’s general vision while meeting the specific needs of each application.
Despite understanding the importance of having a test strategy, many organizations still struggle to do it. Some of them perform testing with no reason nor rhyme. They don’t know the return on investment of their testing approaches, which are aimless and don’t have defined goals.
Other organizations do have a test strategy in place but struggle when trying to scale it. Growth is usually a good problem, but that doesn’t make it less painful. Lucky for those companies, the growing capabilities of TestOps might offer the answers they need about how to scale quality efficiently.
