Automated UI testing is a critical and often neglected part of the software delivery pipeline. I can’t count the number of projects I’ve worked on with terrific, comprehensive unit tests for server code and absolutely nothing for the UI. Part of the reason for this is because UI testing is a real challenge to set up. If you’re a back-end or full-stack engineer, you probably intuitively understand how your server unit testing library works. If you started a new project today, you’d know just how to wire up unit tests.
The same isn’t true for UI testing. While you can do good work with unit testing functions in your UI, those tests provide limited utility. To effectively test a UI, you need to render it. It turns out that’s a lot more complicated than just writing a few unit tests and picking a testing library to execute them.
The last 15 years have seen a variety of browser testing libraries designed to simplify that process. One of the oldest is Selenium, which has been widely used since the mid-2000s. In this post, we’ll make an in-depth comparison between Selenium and Puppeteer, a newer browser testing library from Google.
Puppeteer vs. Selenium: Where Do They Come From?
To understand why these two libraries work the way they do, we need to understand where they come from. Let’s dive a little into their history, which will help inform how they might help you in the future.
Selenium is an open-source project started by a company called Thoughtworks in 2004. It sprang up from a need to test an internal application’s integration all across the stack. Thoughtworks had long understood the benefits of automated testing in shipping quality code. But they recognized that they needed to test their entire application with all the pieces working in sync and talking to each other. From these humble beginnings, one of the titans of the automated browser testing world was born. Since then, the open-source nature of Selenium has served it very well, leading to a breadth of new products. Today’s Selenium features not just Selenium WebDriver, the testing platform, but also an IDE designed for writing Selenium tests and a testing grid designed to leverage many machines to run tests in parallel more quickly.
Expand Your Test Coverage
Puppeteer takes a bit of a different direction. For starters, it’s not specifically a testing library, though that’s what it’s most often used to do. Instead, Puppeteer is a library for controlling Chrome and Chromium browsers. It’s written and maintained by a team at Google, who have unmatched access to the internals of the Chrome browser. This means that Puppeteer can do things that browser testing libraries like Selenium can’t. For our purposes today, we’re looking at these two libraries as testing libraries. But it’s worth remembering that Puppeteer isn’t just a testing library. It’s a tool that gives you unprecedented control over the browser.
Puppeteer vs. Selenium: Support Structures
As a testing library, Selenium seeks to support a wide variety of browsers. Selenium WebDriver supports not only Chrome and Chromium but also Firefox, Safari, Opera, and the now-deprecated Internet Explorer. Puppeteer only supports Chrome and Chromium. This plays back to their different goals. Selenium is a web-UI testing library, while Puppeteer is a remote control library for Chrome. There is an experimental library adapting Puppeteer support for Firefox, but it’s explicitly not ready for production use.
With that said, Chrome handles about two-thirds of the overall web traffic in the world. If you’re worried about browser-specific issues troubling your site, it’s good to know what percentage of your users use which browsers. If you’re comparing Selenium versus Puppeteer, it’s good to understand how the libraries’ support will impact your users’ experience. It’s very likely that a significant majority of your users visit your site on Chrome, and that those who don’t, won’t experience any ill effects for using a different browser.
Programming Language Support
This is another place where Selenium and Puppeteer diverge. As a testing library, Selenium is focused on providing broad coverage for several different programming languages. Selenium officially lists support for Ruby, C#, Java, Python, and JavaScript. Those languages are all supported in the first-party Selenium IDE, a tool that simplifies writing Selenium tests. If you’re working with a team of developers to build automated web application testing, the list above is basically the list of the most popular web programming languages. It’s likely that most of your team already knows more than one of those languages.
Puppeteer, on the other hand, is not focused on providing a broad experience for a range of developers. Instead, as a browser control tool, it’s focused on providing a particular set of control structures in a focused way. For that reason, the only language that Puppeteer supports is JavaScript. You’ll need to run and install Puppeteer using npm or Yarn. The good news here is that if you’re building a web application today, it’s likely that you already have an entire JavaScript ecosystem set up. It’s also quite possible that a significant percentage of your team already writes JavaScript or one of its variants every day. Puppeteer’s focus on one programming language probably won’t be a deal-breaker.
Puppeteer vs. Selenium: Ease of Use
This is one of the places that UI testing libraries often fall down. Writing tests for application UIs is often a tedious task made worse by verbose APIs. Unfortunately, this is one of those places where Selenium has contributed to the negative perception of UI testing. Selenium is a testing library, which means that it involves painstakingly setting up specific situations, then verifying that your software did what you expected. Speaking from experience, this often involves a lot of code. And because Selenium provides broad support for so many browsers, you’ll likely need to end up writing this code by hand, as opposed to quickly generating automatic control code.
On the other hand, this is a place where Puppeteer absolutely shines. Because it is narrowly focused with a single API, it’s much easier to automate generating Puppeteer code. In fact, Testim did just that recently, releasing a tool that lets you record test workflows right in your personal (Chrome only) browser. Using that test recorder for Puppeteer will save teams about 60–70% of their setup time for automated browser tests. Since setting up and writing tests is where most UI testing initiatives fail, shaving 60% off the setup time makes your project a lot more likely to succeed. Even better, the tool is completely free.
Which Is a Better Fit for You?
The truth of the matter is, if you’ve made it this far, you’ve probably figured out which testing library is the right choice for you. We don’t need a scorecard or a point-by-point comparison. Puppeteer and Selenium are two different tools serving slightly different needs. If your needs align with Puppeteer, you’ll likely find it’s the better tool for your job. It’s also likely that the extra power and focus provided by Puppeteer will speed your team’s work in setting up tests. What’s more, it’ll let you control a variety of other browser functions at the same time.
Conversely, if you need to support non-Chrome browsers and you need language bindings that aren’t in JavaScript, you already know that Puppeteer won’t work for what you want. If not, that’s OK. Selenium is a time-tested tool used by developers worldwide.
Whichever project you pick, you’ll find a terrific library there to support you in your journey toward using automated tests to ship better, safer, more secure code. That’s a journey worth taking, no matter which tool helps you along the way.
