We’re here to demystify yet another testing-related term: white box testing. No, this isn’t yet another new type of test you must add to your ever-growing list. So, what is it?
You can think of white box testing as a category of testing. So, you can classify all types of testing as either white box testing or black box testing, with a little bit of a grey to spice things up.
What are the test types that fall under the white box category? Why is this distinction useful in the first place? These questions—and more—will be answered in today’s post. Let’s begin.
What Is White Box Software Testing?
We start with fundamentals, defining white box testing, and explaining how it fits into the bigger picture. Instead of outright saying what “white box” is, let’s start by explaining what it isn’t.
So, the first thing to keep in mind about this term is that it’s not another type of testing, like unit testing or UI testing. That means that questions like “I already have unit and integration tests; should I have white box testing as well?” don’t make sense: it’s an apples and oranges comparison.
White box testing likewise doesn’t refer to a testing best practice, like shift-left testing (that is, the practice of starting testing practices as early in the development lifecycle as possible.)
So, what is white box testing?
White box testing—and its counterpart, black box testing—are ways in which to categorize the different types of software tests, when it comes to a specific factor. And this factor is the extent to which the test knows about the internal implementation of the system under test.
So, here comes our definition.
White box testing is a category of testing in which the internal structures of the system under test is known to the test.
Even though white box is the most popular name for this kind of testing, this category is also known as clear box testing, glass box testing, transparent box testing, and structure testing, among other names.
The opposite of white box testing is black box testing, and it refers to kinds of software tests that are oblivious to the internal structure of the software.
Examples
Having learned the definition of white box testing, you’re now ready to know some of the most important types of white box testing.
As you’ve just seen, white box isn’t yet another type of testing, like unit testing or integration testing. We’ve defined it as a category. You can also think of it as a method or approach that you can apply to different types of testing.
So, what are the types of testing to which white box testing is applicable? And how does that contrast with black-box testing?
In a general way, we could say that black-box testing is more applicable to higher-levels forms of testing, which don’t usually require coding knowledge. Take end-to-end (E2E) testing, for instance. This is a type of software testing where you’re generally not concerned with the internal implementation of the application. Instead, what matters for end-to-end testing is that the application behaves as intended and the features are implemented according to the users’ requirements and needs.
On the other hand, performing something like unit testing or integration testing does require both programming skills and a strong understanding of the application’s internal structure. The need for coding skills is due to these types of tests being literally programs that test other programs.
Advantages and Disadvantages of White Box Testing
Every testing approach has its pros and cons. White box testing is no exception, so let’s examine the good and the bad when it comes to this approach.
Advantages
White box testing techniques are supposed to thoroughly cover all possible execution paths in an application. If you manage to achieve such high coverage, you increase the chances that you detect bugs before they reach production. And I imagine you’ll agree that when it comes to detecting and fixing problems, the sooner, the better.
UI testing obviously requires a UI. End-to-end requires…well, that your application has the two ends completed. That’s to say, these forms of testing require a complete application.
When it comes to testing techniques that fall under the white box umbrella—such as unit testing or integration testing—you can and should use them as early as possible. This will enable you to catch more defects, to catch them sooner, and have testable code from the get-go.
In other words, white box testing and the types of tests associated with it enable shift-left testing.
Summarizing, these are the main advantages of white box testing:
- You have a better chance of catching bugs due to most execution paths being covered.
- Testing can start super early in the development process, without the need for a UI.
- You have testable code from the start.
Disadvantages
It’s not all a bed of roses when it comes to white box testing: it has its cons as well as pros. The first disadvantage of the white box testing techniques that comes to mind is that tests in this category will usually be reasonably complex and require programming skills. That means that it’s mostly programmers who write them, which decreases the number of potential people that could write them.
Also, that introduces an opportunity cost type of problem: highly paid software developers who are writing tests are not doing other potentially more valuable activities.
Another disadvantage of white box testing has to do with test maintenance. These tests are more tightly coupled to the underlying implementation of the application under test. That could give birth to tests that often fail due to the slightest changes in the codebase.
A brittle test suite impacts the team in mainly two ways. First, it creates more work in the form of test suite maintenance. Second, it brings team morale down and makes people lose trust in the automated testing effort.
Summing up, the cons of white box testing are the following:
- It requires coding knowledge, which decreases the number of people who can work on them.
- Having developers writing tests might be considered an opportunity cost problem for the team.
- White box testing is tightly coupled to the app’s internal structure, which can cause brittle test suites. That, in turn, leads to higher test maintenance and people losing trust in the testing effort.
White Box Testing: Does This Term Still Matter?
In today’s post, you’ve learned the definition of white box testing and its pros and cons. You’ve also seen some examples of both white and black box types of testing. Before we part ways, there’s an important question we need to answer: is this categorization still relevant?
The line between white box and black box can often become blurred. For instance, even though people consider unit testing a classic example of white box testing, there’s still some controversy on the matter. An article by Kent Beck argues that TDD (test-driven development) radically alters the game and makes test dichotomies irrelevant. Its conclusion is that unit tests can be both white or black box testing. It depends on how you actually write the tests. So, what’s our verdict?
Well, the white box/black-box terms are exactly that: terms. Vocabulary. They allow conversations to take place. To determine how useful they are, we have to think about the kinds of conversations we want to have.
Take “test maintenance,” for instance. When it comes to this issue, it’s important to have conversations about the different types of software testing available and the test suites they generate, in terms of how brittle or robust they are. Are the “white box/black box” terms useful for this conversation? They certainly are.
However, as you’ve seen, the line between the two types might sometimes be thin. Nowadays, the software testing scenario, and the software world as a whole, is way more complex than they were back then when these terms appeared.
So, instead of thinking of white box/black box as two extremes, try to think of the situation as a gamut—less black and white and more shades of grey.
This post was written by Carlos Schults. Carlos is a .NET software developer with experience in both desktop and web development, and he’s now trying his hand at mobile. He has a passion for writing clean and concise code, and he’s interested in practices that help you improve app health, such as code review, automated testing, and continuous build.
