It’s frustrating, isn’t it? Customers want more features, and they want them yesterday. At the same time, everyone wants fewer bugs.
You know what they say. Speed, quality, and money: pick any two, but not all three.
At least your developers and testers appreciate what you’re dealing with. They understand the need to test what they create on more than a single platform. They just don’t have the time or the resources to do it. The combinatorial explosion of operating systems, browsers, and devices—not to mention different versions of them—makes them, and you, shudder.
There is a solution to this. Grid testing.
So what’s grid testing? This post explains what it is, what it does, and how it helps you create better software and deliver it faster. Also, it looks at the pros and cons, what you need to get started, and a few of the tools you can use, such as Selenium Grid and others.
Let’s dig in.
What Is Grid Testing?
As you know, there are many aspects to front end testing. Ensuring that your web application works across a large variety of environments is one of them. This is known as cross-browser testing. Cross-browser testing means running all your tests not just once, but once for every environment in which your application is likely to be used.
Even when you’re already using automated UI testing software, running the full suite of tests in a multitude of environments is time-consuming. That’s where grid testing comes in. It allows you to run your tests simultaneously on a grid: a collection of machines, operating systems (OSs), and browsers.
The simplest grid is a single machine with multiple browsers. For example, a desktop machine running Windows 10 and, let’s say, Chrome, Firefox, Opera, Brave, Internet Explorer, and Edge. This grid supports six environments: six unique combinations of hardware, an OS, and browsers.
Many Machines and Devices
When you want to spread the test load so the entire test suite finishes faster, you’ll need more than a single machine.
Different versions of the same browser on one machine simply don’t play nice. So, if you need to support multiple versions of OSs and browsers, or even just multiple versions of a single browser, you’ll also need more than a single machine.
It quickly adds up to a lot of hardware.
Testing Traffic Agent
Your developers and testers won’t always run the full suite of tests. They’ll focus on the tests covering the part of the application they’re working on.
Finding the machine or machines in the grid with the right environments to run a specific test is a task no one wants. It’s best left to the software that manages the grid. You can think of this grid software as a traffic agent for tests.
Your staff and your integration and build server only need to talk to this software. They can stay completely unaware of the complexity of the grid behind it.
For example, a developer has just fixed a bug. They now want to make sure that the fix hasn’t caused any problems in other parts of the application. They send a set of tests to the traffic agent, along with what environments they want to execute them in. The traffic agent matches a test with one or more machines in the grid, gets these to execute the test, and reports back the results.
How Does Grid Testing Affect the Bottom Line?
Well, just imagine the number of people and the effort in organization and coordination you’d need to start and run your full test suite against all environments you need to support. Even when you already use automated UI testing, so you don’t need an army to manually execute steps in test scripts, being able to hit a button and have everything taken care of saves a lot of time and hassle.
The reduction in manual effort allows you to deliver features faster, making your customers and salespeople happy.
And that’s not all.
Like it or not, even browsers compliant with W3C standards still differ in their implementation of those standards. So when you test in multiple environments, you’ll get fewer bugs reported when you go live because you’ll have already caught them.
And that means three things. First, you’ll have greater confidence and less to worry about when you release a new version to your users. Second, you’ll have greater customer satisfaction, and third, you’ll be able to save in the support department.
Finally, your developers and testers will be pleased that they can focus on the functions and features to develop and only worry about browser and device peculiarities when the grid testing tells them there’s a problem.
What Are the Costs of Grid Testing?
You’ll face initial and ongoing costs with grid testing. Here are a few of them:
- Devices and the licenses of the software you run on them
- The grid testing software to run the grid and execute tests on the grid
- Training of your staff to use the grid testing software effectively
- Adapting your integration and build server to integrate with the grid testing software
You can find an in-depth discussion of ROI for automated testing here.
The 3 Parts to Taking Advantage of Grid Testing
Apart from a good approach to design and implement automated tests, you’ll need the following:
- Automated UI testing software
- Grid testing software
- Physical or virtual hardware grid
Grid testing software hardly ever comes as a stand-alone solution. It’s either bundled with an automated UI testing tool or comes as part of the service of a grid provider.
You have three options for creating your grid: full DIY (building your own grid), signing up with a computing provider, and signing up with a grid testing provider.
The Pros and Cons of DIY Versus Using a Provider
With a full DIY solution, you get to control every aspect of your testing grid. And that’s also the drawback. You’ll have to do everything yourself. Not just at the start, but for as long as you have it. Still, it’s a good option when the number of environments you need to support is limited.
Computing providers such as AWS, Microsoft Azure, and Google Cloud or device providers such as AWS Device Farm, Xamarin Test Cloud, and BrowserStack don’t change what you need to do to keep your grid running. Their advantage lies in taking the pain out of acquiring and managing the hardware yourself. They also offer a lot more choice in devices than you would contemplate if you had to house the hardware yourself.
Grid testing providers include everything you need for grid testing: the hardware accessible through the cloud, the grid testing software, and the automated UI testing software.
Selenium Grid
Selenium Grid is the most popular grid testing software.
It’s part of Selenium, one of the most-used automated UI testing solutions for web applications. Apart from Selenium Grid, Selenium consists of the Selenium IDE. This is what you’d use to define the tests for your application. Then, there’s Selenium WebDriver, which drives the browsers according to the test definition.
A Selenium grid consists of a single hub and one or more nodes. The hub is the traffic agent, and the nodes are the machines and devices that actually execute the tests. Each node is registered with the hub, specifying the characteristics of the machine, the operating system, and the browsers it has installed.
Grid Testing Tools
You’ll find a good discussion of the best software testing tools in Guru99’s recently updated list of top 50 software testing tools.
The top grid testing tools it mentions are LambdaTest, CrossBrowser Testing, and SauceLabs.
All three are cloud providers offering automated test execution on a Selenium grid with an extremely wide variety of desktop and mobile browsers. SauceLabs also offers JavaScript unit testing on its grid. And CrossBrowser Testing can use different browser drivers such as WebdriverIO and Nightwatch, which are said to be the fastest.
Testim
Guru99 lists Testim as an automated testing tool, heralding its features for authoring, maintaining, and executing tests.
Of course, Testim is a grid testing tool as well, and Guru99 does mention that Testim can execute tests on multiple web and mobile platforms. The reason it’s not listed as such is probably because most cross-browser testing tools and services rely on Selenium, Appium, and other tools for test definition.
While Testim also uses Selenium for test execution on its grid, it’s unique among grid testing tools in taking the pain out of creating Selenium tests and … maintaining them.
Writing a Selenium test is a lot of coding work—and like any other coding, prone to errors. The Selenium IDE (a browser extension) does allow you to record and play back user actions, speeding up test creation and running it in different browsers, but testing then remains a manual process. You’ll have to transform the recorded scripts to code to be able to run them automatically.
Testim.io allows you to record tests and run them automatically, meaning you get the best of both worlds: faster test creation and no manual labor running scripts.
With testim.io you can also opt to code tests manually (currently in beta testing) using an API that feels a lot more natural than the bare metal code “raw” Selenium tests require.
Selenium Box
When security requires you to keep everything within your own network, you’ll find one option that gets you the best of both worlds. Selenium Box is an on-premise managed Selenium Grid solution. In other words, it’s a testing grid within your own network without the hassle of DIY.
Now It’s Your Turn
Now that you have the full Monty on grid testing, you can stop shaking and help your developers and testers set up an environment that’ll help them create software to be proud of. Software you can release with confidence and without the need for heroic late nights.
Your first step is to get your developers and testers together and work out what you most need grid testing to achieve for you. Then, figure out which of the tools would best suit what you already have and most closely match how you want to work.
This post was written by Marjan Venema. Marjan is a Smart Blogger certified content marketer, working from the Netherlands with over 30 years of experience in software requirements, analysis, development, and support for the Business Planning and Analytics, Financial and Manufacturing industries. Her specialty is writing comprehensive engaging content that makes complicated and complex topics easy to understand and consume. She writes straightforward language in a conversational style and illustrates abstract topics with concrete examples.
