In this day and age, no serious manager or tech leader would question the importance of testing. However, it’s also necessary to decide how to carry out the tests in practice. This takes us to today’s topic: test design.
Test design, in short, is the process of defining how test activities will be done. Here are some of the topics we’ll tackle in the post:
- What does test design mean? Why do it?
- When is test design done, and whose responsibility is it?
- What are the techniques people use for test design?
Also, we know how hard it is to keep up with the gigantic—and always growing—lexicon of the software testing world. That’s why we’ll use this post as an opportunity to kill two birds with one stone; we’ll cover several concepts and terms that are crucial to know, not only in the context of test design but of testing as a whole.
Test Design Fundamentals
Let’s open the post with some fundamentals on test design.
What Is Test Design?
Test design means determining how tests will work. When you do test design, you’re essentially defining the details of your test cases. What will their steps be? How will they be structured?
Among other things, designing a test might involve:
- Thinking about the data the test will use, including its realism, amount, and boundaries
- Expressing detailed test scenarios in a visual—e.g. a diagram—or a written way—e.g. a table.
- Trying to predict possible errors and edge cases based on one’s previous experience and knowledge of the application.
What Is the Purpose of Test Design?
Software testing—both automated and manual—isn’t free. You should treat it as an investment, which includes tracking its ROI. Thus, you want to make sure your software testing strategy is as efficient as possible.
That’s where test design comes in handy. With a proper test design process in place, your team will create tests that generate value for the organization, ensuring you can ship high-quality code as fast as the market demands it.
How Is Test Design Done? Who Does It? When?
You already know what test design at a high level is and why it matters. Let’s talk about how to get it done.
Who Performs Test Design?
Who actually does test design? It depends on the type of organization. In more traditional companies with a more stark divide between roles, testers and QA staff would be responsible for test design—and probably for all other activities regarding tests.
However, “the times, they are a-changing.” Due to the rise of automation, agile, and DevOps, the lines between roles inside tech organizations are becoming thinner and thinner. Today, we dare to say that everyone performs testing. This, by extension, means everyone does test design.
Here’s what this might look like in practice:
- Engineers write unit tests, which involve designing the necessary assertions, doing setups and teardowns for classes, and coming up with test data.
- Testers with no coding skills can use record-and-playback tools to record UI or end-to-end tests, which they or other QA staff designed.
- If the organization adopts BDD (Behavior-driven development), they might have the customer—or someone acting as their proxy—collaborating in the creation of automated specifications.
The list above isn’t exhaustive. But you can see how test design can be the work of many different actors.
When Is Test Design Done?
When does test design take place? Like the previous question, the answer to this one depends on how things are done in your organization.
In organizations that still follow a more traditional, phase-based development methodology, you’ll probably also find a more formal software testing life cycle in which test design is a phase with a well-defined place.
However, nowadays most organizations are following an agile-based approach to software development. Such approaches favor development in short, iterative cycles. In such scenarios, testing is no longer a phase but an activity you carry out as early in the process as possible. If that’s the case, test design is done several times in the context of a single iteration.
Test Design Techniques
Having covered the “what,” “why,” “who,” and “when” of test design, the only major question left for us to tackle is the “how.” Now we’ll cover three techniques for test design.
Equivalence Class Testing
One of the hardest things when designing test cases is knowing when it’s enough. How many examples do you need? How comprehensive do your test data need to be?
Equivalence class testing—aka equivalence class partitioning—is a solution to this dilemma. This technique tells us to partition our test data into big “buckets” we can consider equal. Then, only a few values inside each group—and at their boundaries—are enough to ensure our test is comprehensive.
For instance, let’s say you need to test a feature that asks for the user age and does something different according to these age groups:
- 0 to 12 years
- 13 to 18 years
- 19 to 40
- 41 to 65
- more than 65
For each of these “buckets,” you need to have a test case:
- at each boundary
- immediately below and above each boundary
- for one value inside the group
So, for the “19 to 40” group, the values 18, 19, 20, 39, 40, 41, and 23 should be enough as test cases.
State Transitioning
Most commercial or enterprise software features can be described as finite-state machines. I know that computer science people are fond of scary-sounding names, but this concept is quite simple. It refers to how a given system goes from one state to another after some action happens.
When designing a test case, you can use a diagram or a table to reference expected state changes in your application. FitNesse, a popular tool for automated acceptance testing, does precisely that: it allows developers, QA, and other stakeholders to express the behavior of their systems through tables that depict state transitions.
Exploratory Testing
Exploratory testing is essentially testing that doesn’t follow any pre-defined script. When performing this kind of testing, testers will use their creativity, domain knowledge, and previous experiences to come up with novel and unexpected ways of interacting with the system under test. In doing so, they often uncover edge cases and unexpected problems.
But if exploratory testing is non-scripted by nature, how can it be a technique for test design?
Easy; with the help of a test automation tool that doesn’t require code skills, testers can convert their manual exploratory testing sessions into automated test cases that they can execute later. That way, they preserve the insights found out during their creative testing sessions, converting them to a more perennial form.
Test Design: The Top Concepts You Should Be Familiar With
Before parting ways, we’ll cover some essential concepts that will be valuable in conversations about test design and software testing as a whole.
The Test Automation Pyramid
You’re probably familiar with the concept of test automation, its benefits, and its relationship with manual testing in a well-balanced quality strategy.
However, test automation comes in many forms. How do you tell them all apart and decide which ones are the best ones for your organization?
The test automation pyramid is an answer. This is a simple framework that teaches you about the main types of automated testing and how they differ in complexity, cost, and the kind of feedback they provide.
Test Coverage and Code Coverage
People often confuse test coverage and code coverage, but the two metrics refer to different things.
Code coverage refers to the percentage of a codebase covered by unit tests. Test coverage, on the other hand, refers to how much of an app’s features are exercised by at least one test case of any type. It’s less about code, and it’s not relegated to a single type of testing.
Both metrics are important—test coverage more so—and knowing how they differ is essential when thinking about test design.
Test Suites and Test Cases
Test suites and test cases are yet another pair of terms people often mix up. While we have an entire post about this, here goes the TL;DR: a test suite is a collection of tests—often run together—while test cases are the individual tests.
When doing test design, you’re typically designing the test cases that belong to a test suite.
Test Design: Since You’re Doing It Anyway, Do It Right
In this post, we’ve offered you a brief but complete guide to test design. You’ve learned the what-why-how-when of test design, as well as some crucial concepts/vocabulary related to it. What should your next steps be?
How about learning more about software testing? Here are some more articles you might enjoy:
- What Is the Software Testing Life Cycle? A Complete Guide
- The power of Session-Based Exploratory Testing
- No Code / Codeless / Low Code testing: What’s the difference?
A good second step would be to look at available tooling that can help you improve your testing approach, such as Testim TestOps.
Before parting ways, a word of advice: There’s no such a thing as “having no test design.” If you have tests, someone designed them, period. The problem is if you’ve been doing poor test design. Since you’re doing test design anyway, you might as well invest some time and effort and do it properly. Your clients—and your pocket—will thank you.
What to check out next
Why Test Automation Fails: Test Design and Implementation Tips
