Selenium bills itself as a tool to automate browsers. That kind of tool is critical for today’s continuous integration web development workflow. When you’re shipping code hours after a developer writes it, you need to be sure it works. The best way to do that is to automate tests that run against your application on every deployment. You don’t want to ship application-breaking bugs to your customers, no matter how much you might believe that you should “move fast and break things.” Not even Facebook believes in that motto anymore, and neither should your company. For teams looking to add browser automation to their testing stack, Selenium might be a great fit. But if you’re a developer recommending Selenium for your organization, you’re going to run up against an age-old question: “How much does Selenium cost?”
The Short Answer
It’s free! You can download it from their website without paying a penny. See? Wasn’t that an easy answer?
Not so fast, my friend. While it might not cost anything to install Selenium, that’s not really answering the question. “How much does it cost?” isn’t asking about an upfront fee. It’s asking about what adopting Selenium will cost the organization as a whole. That’s a significantly more complicated question. It bears some diving into before we attempt to answer it completely. If that shiny download link up above didn’t distract you too much, let’s dive into what adopting Selenium means for your organization and whether an alternative might suit you better.
The Long Answer
Like with any open-source tool, the upfront cost isn’t the only variable to consider during adoption. Many organizations look to adopt a variant of Linux as a desktop operating system for their end users, only to discover that they need to work around piles of hidden administrative costs. That’s not to say those companies made the wrong choice, merely that a major decision like a testing program requires careful consideration. By looking at Selenium from every direction, you’ll do your company service before you decide whether it’s the right tool for the job. In this post, we’ll look at some of the costs your team will have to pay if you choose Selenium and how other tools might be a better fit.
The Learning Curve
If you did click over to that download link for Selenium, you might have noticed something. The first thing you need to decide when downloading Selenium is which language bindings to use. That’s right; Selenium is a programming library. It most definitely is not a tool that you can plug into your existing ecosystem. Instead, you’ll need developer time to get Selenium going. While Selenium’s language bindings do come in various popular languages, that’s still a learning curve that you’ll need to climb. And the learning curve for Selenium can be quite steep at times.
If you’re like most software companies, developer time is one of your most precious assets. It’s worth remembering, too, that you don’t just pay this cost once. You’ll need to pay this cost for every developer you onboard. The goal is for your developers to write functional tests for all the code that they ship. You’re trying to avoid your QA teams acting as a bottleneck for the software you ship; that means you need developers writing effective tests.
Slow authoring
Even after your developers are skilled-up on Selenium, it still takes a while to write stable tests. A developer or automation engineer can spend up to a day writing an end-to-end test that consistently passes. This is a huge cost, consuming some of your most valuable resources. Even using Selenium IDE, the record/playback tool for Selenium, you can still spend hours per test written. When you are trying to release new features to the market and running short, two-week sprints, this is still too much time spent creating the tests. More modern tools enable test authoring in a fraction of the time.
Brittle Tests
Another critical issue with Selenium is that testing is often brittle. Selenium’s WebDriver works as a browser that renders your web page’s content and then takes actions based on that rendered content. This has one enormous advantage: it means your tests run in the exact environment your users see. As a developer, that’s a huge help. When you ship a change, you know that the automated tests will give you the same experience your users have. It eliminates the “works on my machine” disconnect when some bugs are difficult to reproduce. If the tests break, the experience will break for your users.
Well, that’s the idea, anyway. Reality isn’t always quite so simple. Because Selenium requires scripting every test you run, tests can wind up inadvertently prone to breakage. For instance, imagine a test that first required logging in, then took some action on the user’s profile page. As a developer, you might script that test to click on a button with a CSS class of login_button to perform the login action. Now, let’s say that the marketing team does a CSS redesign, and the class for the button changes to loginButton. Your test is broken, even though the functionality didn’t change one whit. The test is brittle.
Because Selenium tests can be brittle, developers often need to spend time fixing broken tests that broke through no fault of their changes. This is another hidden cost of Selenium. Time spent maintaining tests is the time your developers aren’t working on new features.
Troubleshooting Tests
Troubleshooting a test in Selenium can also be a pain. It often involves looking at error codes like “element is not clickable at point” and combing through logs or .har files to figure out what went wrong. If you are running your tests in the CI and one fails, you need to quickly diagnose the error, fix the test or the code, and rerun the test. This is time spent by costly engineers who could be building innovative features.
A Better Alternative
Fortunately, Testim provides a simple system for recording tests that aren’t reliant on programming. Instead of requiring that you program every interaction to specification, Testim enables you to record user interaction and configure it in an intuitive visual editor. You can add custom JavaScript actions to run code inside or outside the browser to fit your application. Because Testim is easy to use, it reduces switching costs when developers move from programming to testing.
Testim uses AI-based Smart Locators to understand the entire DOM and lock-in elements to improve stability. This means that when a code change updates CSS or XPath selectors, tests verifying the page’s functionality aren’t going to break.
Testim also provides tools to simplify troubleshooting when tests fail. The information to diagnose is at your fingertips, from step failure indicators to highlighted, comparison screenshots to parsed console logs, and network information. This means someone who didn’t write a test can easily see what caused the failure, whether a test problem or application bug, saving time, and eliminating dependencies on particular resources.
Free Doesn’t Mean No Costs
Even though there’s no fee to download and install Selenium, it’s not free of cost. Selenium has high hidden costs that often add up. Consider the total cost of ownership that a tool requires. Often, that’s a cost you’ll be paying for years after adding that tool to your build process. You might find that the total cost of ownership is reasonable for your organization. You might also find that it makes sense to investigate other tools. Those tools often require a little more upfront cost but save lots of money and time in the long run. And in the case of tools like Testim, you can also try the tool without paying any money upfront.
Trying out Testim is a risk-free proposition. Check it out today, and find out that sometimes you don’t need to spend anything to save your team’s time and money.
