We’ve already undergone quite a journey to evolve from “classical testing” and test automation to continuous testing. Nevertheless, when we look into the future, it’s clear that even continuous testing won’t be sufficient.
Certainly, digital transformation happens in test automation, as it does in every field. We’ve come from manual testing to a robust automated testing suite that integrates nicely with continuous integration and continuous delivery tools. The Testim blog has more information about the history of software testing.
The rise of AI has created high hopes, even in the field of test automation. To meet the quality needs of a future driven by robotics, IoT, and quantum computing, we need more sophisticated testing approaches and tools.
Can artificial intelligence (AI) help us get there?
Learning the Basics of Artificial Intelligence
Before we dig into artificial intelligence (AI) assisted testing, the question arises: What is AI?
According to a study by Shane Legg (cofounder of DeepMind) and Marcus Hutter (professor at the Australian National University), it’s not clear among experts what AI is. They gathered over 70 definitions of AI from psychologists, researchers, and experts and concluded that no consensus existed.
That’s why they came up with the following simple definition for artificial intelligence: “Intelligence measures an agent’s ability to achieve goals in a wide range of environments.”
If you look closely at this definition, you’ll notice that the current state of AI is good at one specific task only. For instance, that task might be playing a game, recognizing images, or using computer vision. Most work that’s being labeled as AI isn’t AI, according to this definition.
What term other than AI makes sense to describe these capabilities? It’s more accurate to label this type of work as machine learning (ML). Legg and Hutter describe machine learning as an algorithmic field that uses statistics, computer science, and many other fields to make predictions and decisions—and that’s the foundation of most of today’s AI-labeled systems.
However, test automation can benefit from the AI hype. AI definitely has the potential to support test automation and make the testing process even more efficient. Let’s explore how to use AI in test automation.
How to Implement AI in Test Automation
A more explicit interpretation of artificial intelligence would be: “The ability of a computer program to think (reason for itself) and learn (collect data and modify future behavior in a beneficial way).”
That’s exactly how we can use AI in automated testing. However, artificial intelligence in testing is still in its infancy and is much less evolved than areas such as self-driving systems or voice-assisted control. Researchers put the focus of AI in automated testing on making the software development life cycle easier and more efficient.
Next, let’s look at a couple of examples of AI in software testing.
Examples of Artificial Intelligence in Testing
Many projects have started research on integrating AI in software testing products. Let’s find out what the possibilities are.
AI for Unit Test Generation
One of the most simple use cases for applying AI is generating unit tests. AI can help with static analysis of the software to determine which areas of the code unit tests don’t cover.
Next, the tool can use this information to generate unit tests for uncovered code and even provide input for these unit tests. It’s much easier for an AI-assisted tool to generate unit tests, as these type of tests don’t require any setup or interactions with other components.
In the future, AI will be able to update unit tests whenever the source code changes, so artificial intelligence will be capable of maintaining unit tests. This will be great news for developers because they’ll spend less time on test maintenance.
However, the AI needs to be trained to know what pieces of code require stubbing or mocking in order to isolate them. In order to excel at this skill, the AI first has to learn to determine dependencies in order to stub them correctly. Unfortunately, this training process requires a serious amount of time to teach the AI how to build up unit tests. Once the AI has reached a point where it can create mocks and stubs by itself, it will be one of the most time-saving parts of unit test creation for developers.
AI for API Test Generation
Application programming interface test generation goes hand in hand with the user interface that lies on top of it. The AI tries to understand the patterns and relationships in the different API calls made while exercising the UI. From that analysis, it can create a series of API calls and provide parameters to test.
Furthermore, by analyzing the behavior of users, AI can generate more advanced patterns and inputs for testing the API. The generated tests cover more edge cases and ensure a higher level of quality for your API.
Testim’s Machine Learning Testing Tool
Testim has a tool that uses machine learning for the authoring, execution, and maintenance of automated tests. It emphasizes functional, end-to-end testing and user interface testing. The tool becomes smarter with more runs and increases the stability of test suites. Testers can use JavaScript and HTML to write complex programming logic.
Now let’s explore how AI-based testing can benefit you.
Benefits of AI-Based Testing
Clear benefits exist for using artificial intelligence in software testing.
- Developers spend less time writing tests, including unit tests.
- Your testing engineers spend less time running the actual tests.
- As your team spends less time on testing, you speed up your development cycle.
- Code coverage increases by using artificial intelligence, as it can determine uncovered pieces of code.
- AI can generate more advanced test cases that improve the quality of your application.
- It’s likely that your company will save money. AI reduces the testing cycle from days to a couple of hours.
In short, developers and testing engineers can benefit from a mixture of test automation and artificial intelligence.
If AI-based testing is so great, then why doesn’t everybody use it?
Challenges for AI-Based Test Automation
The major challenge lies in the type of testing. AI optimization can’t be applied to every type of testing. Functional and unit tests are easy to generate using AI. However, testing types like integration tests are much harder to generate, as they require a more advanced setup and complex flow.
Another well-known issue is the “Oracle Problem.” A research paper by Earl T. Barr et al. describes why we need more research on the challenge of distinguishing the corresponding desired, correct behavior from potentially incorrect behavior.
The authors explain: “Test oracle automation is important to remove a current bottleneck that inhibits greater overall test automation. Without test oracle automation, the human has to determine whether [the] observed behavior is correct.”
As a result, there’s still some work to be done to bring automated testing to the next level.
The Future of Automated Testing and AI
The current state of AI in software testing is immature. Many project leaders are just beginning their research into how they can use their testing tools with AI. On the other hand, machine learning is showing some positive signs, including test generation and dependency discovery.
Although other areas, such as self-driving cars, have evolved much faster, we’ll definitely see AI-based software automation tools that will start a new phase for test automation. There’s definitely a need for more optimized testing, as testing is still a slow and difficult topic. However, we first have to tackle important problems like the Oracle Problem.You can find out more about test automation at Testim.
This post was written by Michiel Mulders. Michiel is a passionate blockchain developer who loves writing technical content. Besides that, he loves learning about marketing, UX psychology, and entrepreneurship. When he’s not writing, he’s probably enjoying a Belgian beer!
