Updated: Nov 16, 2020
Background:
Organizations are increasingly software-driven, differentiating their products and services with software. Yet end-to-end (E2E) testing remains one of the biggest bottlenecks to faster software releases. Functional UI and E2E tests are slow to create and break with minor code changes. As teams add functionality or update their UI to maintain a modern look, they break the tests and delay releases
Some companies and open source projects have sought to address the slow authoring problem using record and playback technology to speed authoring. While many of these tools make authoring faster, they used the same underlying technology that made tests flaky. Besides, many recorded tests were inflexible, either failing to capture the user scenarios accurately or requiring frequent re-recording to update test cases to reflect minor changes.
This is why we created Testim, to address the two biggest challenges with UI functional testing—slow authoring and flaky tests. Along the way, we have focused on core principles that guide our product development and make Testim Automate flexible to handle diverse applications and use cases.
We believe functional UI testing automation tools should:
- Enable fast authoring and configuration of understandable tests.
- Grant flexibility—through code insertion to adapt to nearly any application.
- Minimize test maintenance—through self-healing and other mechanisms.
- Integrate into your tools and DevOps workflows.
- Simplify test results, troubleshooting, and root cause analysis.
We recently came across Angie Jones’s article, “10 features every codeless test automation tool should offer.” She does a great job of discussing the problems with some test automation tools and lays out what’s needed in the next generation of codeless automation platforms.
This post will use Angie’s framework and show how we address each of her 10 feature categories. We’ve even added a couple of bonus categories that weren’t in her article.
1. Smart element locators
Testim’s founder (Oren Rubin) coined the term “Smart element locators” in 2015 when he gave Testim’s first demo. He showed how AI could be used to improve element locators. Using hundreds of attributes to identify an element instead of a single static selector, Testim can adapt and identify elements when they’ve changed and use that information to improve over time.
With static locators (e.g., CSS-Selector/XPath), only one attribute of an element is used to identify it on a page. If the attribute changes, the test breaks, and the QA team spends a considerable amount of time troubleshooting the problem and fixing it. Based on our research, with traditional test automation platforms, about 30% of the QA team’s time is spent maintaining tests.
With AI-based testing, we can overcome this problem by using dynamic locators. Testim’s Smart Locators use multiple attributes to identify an element and lock it in. This way, even if multiple attributes change, the element can still be successfully found using other attributes extracted from the DOM.
2. Conditional waiting
Testim supports conditional waits, easily configured with a click of a button. We provide built-in wait functionalities based on element visibility, text (or regex), code-based (custom) waits using JavaScript, waits based on a downloaded file, and of course, a hardcoded “sleep.”
3. Control structures
Testim supports “if” statements and “loops.” Looping can be applied on the test level by parameterizing your tests with different datasets (aka Data-Driven) or on a specific subset of actions, including a group of test steps. Testim can even do this in context, understanding the type of element, and repeating a set of test steps for every similar element. These conditions (when to stop the loops) can either be simple, predefined (such as element or text being visible), or can be more complex with custom code. This capability is unique to Testim and stems from how Testim inspects the application’s DOM under test and fully understands the relationships between elements.
4. Easy assertions (validations)
Assertions are one of the most widely performed actions with test automation. You want to validate an element based on different conditions. For example, if a particular element needs to appear on your website’s specific page, we need to add an assertion to validate the element’s presence. With Testim, we made it easy for users to add assertions with a single mouse click and built all of them within the tool itself.
Users have various validation options that include:
- Validate element visible
- Validate element not visible
- Validate element text
- Validate via API call
- Validate file download (any file)
- Validate HTML attribute
- Validate CSS property
- Validation via custom JS code running in the browser (great for custom UI)
- Validation via custom JS code running in node.js (great for pdf and DB validations)
- Visual validation – via Applitools integration*.
Testim integrates seamlessly with Applitools, a Visual Assertion platform, enabling you to validate appearances like font and color.
5. Modification without redo
Testim not only supports easy modification of steps, but the platform also supports version control systems, including creating branches and auto-synchronization with GitHub. Branches give you the confidence to try new testing techniques without the fear of messing up your tests. Testim is the only vendor to include conflict resolution on branch merges, providing flexibility to select only those changes you want to merge.
In Testim, you can add a step at any point in your test, either through a recording or by inserting a manual step.
You can also easily delete or modify any step in your test.
6. Reusable steps
Testim supports reusability allowing you to share any step or group of steps across tests. It’s easy to create a group—select the steps, and choose the group icon. You can reuse the group in other tests, including passing different parameters.
Testim takes the reusability seriously and is the only vendor to facilitate auto grouping. Testim identifies duplicate test steps across the test project that can be automatically turned into shared groups.
The Testim Automate platform simplifies iterating through similar objects on a page through a feature called Group Context. Imagine multiple components (e.g., gallery of images) within a page or across several pages, and you need to perform the same action on every component in the group. This makes it extremely easy to iterate a test through similar objects on the page. Testim is the only codeless platform that can currently support this action.
7. Cross-browser support
Testim uses Selenium underneath the hood and supports test execution across different browsers, including Chrome, Safari, Edge, Firefox, and IE11 and mobile web browsers.
8. Reporting
It is vital to get quick feedback on your test runs to see whether they passed or failed. It’s also critical to get the details you need to diagnose the root cause quickly.
Reports should be easy to read and need to have relevant information on the state of the test. In Testim, there are different reporting levels to help users know what exactly happened within each test run. Some of the features worth mentioning here include.
- Screenshots
While each test is recorded, the platform takes screenshots of all successful and failed steps. The actioned element is highlighted to see differences between the baseline and current test run easily.
- Feedback on each step
The user gets feedback on all the passed or failed steps in a test by showing a “green” or “red icon” on the top-left corner of the step, as shown below.
- Test Logs
Logs are a rich source of information on what happened underneath the AUT. Testim provides test logs when the user runs the tests on the grids. The option can be found in the top section of the editor.
- Suite and Test Runs
We have suite and test runs views that enable the user to get granular details on each test run in terms of when the test ran, the result, the duration of the run, the level of concurrency, and what browser the test ran on, and much more. We also have filters to drill down based on different options.
- Failure reports
Testim also aggregates the errors by type. This allows you to focus on the most common error types. For instance, in the below image, you can see that four tests had the same error type of “Element not found.” You can drill down into each test and see the step-by-step results of the test.
- Reports
We make it easy to get a high-level health check of your tests using the reports feature. There is no other testing platform that provides this level of detail about the tests, and this is done all in one single dashboard. Details include the percentage of test runs passed, the number of active tests, the average duration of tests, how many new tests were written, how many tests were updated, how many steps have changed, the flakiest tests in your test suite, and more. All of these details can be filtered based on the current day, 7 days or a 30 day period.
9. Ability to Insert Code
Testim is the most flexible “codeless” automation platform enabling organizations to extend the functionalities of our platform using JavaScript. Custom code can be executed in the browser, where Testim can help by finding elements (aka dependency injection), or code can run on node.js, allowing loading many common libraries.
The options are nearly endless. You could use JS to verify that a downloaded MS Excel file has a valid value in a certain cell, to create a custom method to handle a popup message (with before or after hooks), or even parse HTML to use in another step. 
10. CI/CD Integration
Testim easily integrates with ALL CI servers (e.g., Jenkins, Circleci, VSTS) by easily copying the automatically generated CLI command and pasting it in your build servers. We integrate with ALL 3rd party testing grids supporting Selenium and Appium (Amazon/SauceLabs/BrowserStack…). We also support locally hosted grids and also provide our own Testim grids.
Apart from all the above-mentioned features to help build stable automated tests, we also have a notable feature to make tester’s life a lot easier by saving time to report bugs.
Bonus categories
Not all of Testim’s features fit nicely into the above categories. We thought that they deserved a separate mention here.
Network or service virtualization (aka mocking)
Sometimes your application under test isn’t fully ready to be tested, or perhaps every time your application makes an API call to a service it incurs a fee. You still want to test your application as early and often as possible. Mocking the service response can enable you to test earlier without incurring fees for invoking the service. When you run tests against a mocked response, they run faster and aren’t impacted by server-side errors, allowing you to isolate testing to the frontend.
Testim makes service mocking really easy as you can record the a HAR file, edit it, or create your own mapping file and run the test against the test.
Coded tests
I know that this blog is about codeless test automation, but Testim takes it one step further. Testim enables you to export your tests as code so that you can edit in your IDE and run the test on Testim. Why have codeless when you can have codeless or coded?
Bug capture
A time-consuming aspect of testing can be bug reporting. After finding a bug, we need to report it to the developer with relevant information to speed up the troubleshooting and fixing of issues.
With Testim, you can do this much easier with our bug capture feature and integration to key devtools like Jira, Trello, GitHub, and Slack. All the details related to the bug are automatically generated for you in a matter of seconds in including the steps to reproduce it, executed through a Testim test.
In summary, we wanted to build a tool that could help in the authoring, maintenance, and troubleshooting of tests. We built a platform that is simple and flexible and enables the team to collaborate on test automation.
Get started using Testim for FREE. You can also get a free consultation on Test Design and Test Automation on your existing testing frameworks and practices.
What to read next
Codeless or Coded End-to-End Tests—We’ve Got You Covered with the New Testim Development Kit
