It’s 2024 and AI is now a normal part of our lives, and especially of our work as testers. And the estimations are that AI usage in software testing will increase over time. Apart from AI testing tools, like Testim, which generate and maintain test cases, we can also use tools that are not designed specifically for testing or test automation but are still helpful. Such tools are chatbots and virtual assistants.
We’ll start by understanding what each of these are, continue with the differences between them, and then discuss how each of them can help us with our day-to-day tasks.
Differences between Chatbots and Virtual Assistants
Before we see how to use them in test automation, let’s understand first what chatbots and virtual assistants are.
What are Chatbots?
Chatbots are tools that are developed to understand texts or spoken audio and create a conversational response relevant to the original question. The most well-known chatbot at the moment is OpenAI’s ChatGPT. The original purpose of these bots is to help with various support tasks. A lot of websites use them for customer service, but we can now use them for pretty much any type of question.
Here is ChatGPT’s definition of a Chatbot:
What are Virtual Assistants?
Virtual assistants are more complex than chatbots and can deal with nuances in human language. They are usually integrated into devices or apps (think Alexa or Siri) and can manage a variety of tasks such as setting reminders, sending messages, controlling smart home devices, searching the web, and providing recommendations. In test automation and programming in general, virtual assistants can be integrated into IDEs. Some notable examples of virtual assistants are GitHub Copilot or Gemini for Android Studio.
Main differences
- Chatbots offer scripted responses to user questions, while virtual assistants perform tasks and help users with requests.
- Virtual assistants are more advanced than Chatbots and can learn and adapt.
- Chatbots have limited personalization, depending on the domain; Virtual assistants can be customed based on user preferences and history.
- Interactions with chatbots are task-specific; Virtual assistants work best for long-term, personalized interactions.
Chatbots in Test Automation
There are several test automation tasks where chatbots come in handy. But before you start completely relying on them, please double-check the information they provide, because they are not error-free. At least in my experience, I found it best to review and adapt the responses I received (at least from ChatGPT). Also, it’s very important to use the right prompts. Below are the most useful tasks for chatbots.
Test Case Design
One of the first tasks chatbots can help with is designing test cases. You can prompt them with the acceptance criteria of a user story, bug details, or a description of the functionality or screen being tested. With more recent chatbot apps, you can also upload a screenshot of a page/functionality, and ask them to provide test cases.
Code Generation
Once test cases are identified, you can also ask the chatbot to create the automated scripts for them. For instance, if you describe a scenario to be tested, chatbots can write the corresponding test code using frameworks like Selenium, pytest, JUnit, or others.
Debugging and Refactoring
This is my personal favorite task. If I get errors or exceptions I don’t know instantly how to fix, I like to ask ChatGPT. It can review and suggest improvements to existing test code, fixing possible issues. It doesn’t work *all* the time, but it does help a lot. The same for refactoring – you can prompt chatbots to review and improve a code to make it more readable and reusable. Small caveat here – make sure you are not violating any IP rules in your organization by sharing the code with a third party.
Test Data Generation
Another common task where you can use chatbots is test data generation. Based on the test cases provided, the chatbot can identify boundaries, and equivalence classes, and create sets of data for the scenarios that can be used in the automated scripts. This is helpful, especially for more complex scenarios that require a lot of data. For example, you can ask it to create realistic data for usernames, first and last names, addresses, and phone number combinations.
Virtual Assistants in Test Automation
Virtual assistants play a different role in test automation when compared to chatbots. This is because the interaction with them is different. It can be almost like a pair programming collaboration. Virtual Assistants will have access to the entire automation project, so the prompts don’t need to include a lot of background information, because they already have access to it. Again, check with your company that third-party tools are allowed to access your code. Let’s see some of the possible ways they can help us.
Code Suggestions
A code assistant makes coding suggestions as you code. Sometimes the assistant may propose a way to finish the current line of code, and other times it will recommend a completely new block of code. You can accept any part of the suggestion, or dismiss it and continue typing. An AI coding helper saves you time by eliminating the need to create a boilerplate or basic code.
Repetitive Tasks Automation
Delegate repetitive work to a Virtual Assistant, thus freeing up your time for more creative and gratifying tasks. You can use virtual assistants for code refactoring, documentation generation, test writing, and VCS commit change summaries.
Code Review and Refactor
You can improve code quality using an AI Virtual Assistant, which can diagnose errors and exceptions, and provide full explanations and suggestions on how to resolve them. You can use AI-driven refactoring suggestions to optimize and clean up your code, making it more efficient and maintainable.
Code Generation and Pair Programming
Some tools allow you to give instructions in your natural language (maybe not any language, but natural English will work). Because assistants are integrated with the IDE, they have access to the entire codebase and can make more “educated” suggestions than a chatbot, where you need to provide a lot of context to reach the right answer.
Here is a demo of Cursor parallelizing some API tests:
Conclusions
AI is transforming daily tasks, especially in software testing. Chatbots and virtual assistants are AI tools that understand texts and create conversational responses. While they do share similarities, there are some differences between the two. Chatbots offer scripted responses, while virtual assistants learn and adapt. Chatbots assist with test case design, code generation, debugging, and data generation. Virtual assistants can automate various repetitive development tasks, help with refactoring and code reviews, and can generate code based on the existing work. They can be integrated with the IDE. Each of them can play an important role if used wisely.
