Devada builds the communities where developers and tech professionals learn, share, and collaborate to connect the world’s digital innovators. Devada has two primary products, DZone, an online community for developers with over 40 million users, and AnswerHub, a knowledge and collaboration center for organizations. 

Meet Alyssa Capps, a Quality Assurance Engineer at Devada. Alyssa joined Devada shortly after they decided to start using Testim. She helped lead the transformation from a manual end-to-end testing process to fully automated UI testing.

DevOps Modernization

Devada, like many software companies, understands the need for rapid innovation. They wanted to utilize modern development practices like short sprints to iterate quickly, daily standups to improve communication, and automation to accelerate manual tasks. They also wanted to make testing an integral part of the development process rather than a job performed at the end by a separate QA team.

They reorganized their standalone QA team consisting mainly of manual testers into a small but flexible DevOps team that included QA, Site Reliability Engineers, and deployment specialists. But there’s overlap in roles so that team members can step in and fill gaps when needed.

Devada shifted-left the testing responsibility. Engineers are now responsible for creating test plans with each new feature or bug fix. Quality Assurance at Devada now functions as a monitoring and advisory service, helping the engineering team identify and create the right tests to validate new features. The QA service helps ensure that the test strategy for individual features through full application regression is consistent and complete. 

Challenges

Devada enables a lot of end-user customization of their platform. Customers can choose different rich text editors such as Froala and WMD to create and publish an article. However, the various options aren’t always compatible, requiring separate test environments to ensure they meet the prerequisites before running the test suites. 

Users have different permissions, allowing them to access additional features. They needed a way to automate and group tests into suites based on the user type.

Devada considered Selenium, but the time involved in learning and maintaining was deemed too expensive. They were concerned that authoring and maintenance time would slow release cycles and make it difficult to scale automation as product capabilities grew.

Solution

They selected Testim to automate their UI functional and end-to-end tests for DZone and AnswerHub. Alyssa stated they chose Testim “because of its ease-of-use and ability to adapt to meet Devada’s numerous configurations and options.”

Software engineers consult with QA Engineers to create a test plan to validate new features while leveraging existing tests to minimize rework. Once approved, the engineers are responsible for writing and submitting Testim UI tests with their feature pull requests. 

Over the past two years, they have created hundreds of tests in Testim. They organize tests with suites and labels to handle different user types, applications, and environments. 

They’ve integrated their tests with Jenkins continuous integration server and deploy new builds to the testing environment every day. They run UI tests nightly in this clean environment to minimize environment interference. 

Before Testim, they spent a full week manual testing each release. Since automating their UI tests, they now spend about 10 hours of QAE time, mainly reviewing results. 

Alyssa affirmed,

by eliminating the time we were spending on manual testing, it shortened the testing phase of our release cycles by a couple of days. It also freed up more QA time for other activities like cleaning up our pen testing—something we would have never had time for before.

Lessons and best practices:

We’ll take a little detour from the typical success story format to share some useful lessons that Devada learned in their transformation. We found these helpful for any test automation project and wanted to share.

• Naming and labeling are critical – whether it’s your tests or your shared steps and groups, ensure the team quickly understands their purpose by using descriptive and unique names and labels.

• Developers and testers need to collaborate – the developers have the best insight into what they are building and how it will impact other parts of the application. The QAEs know the test automation system, where they already have test coverage and the reusable components that can streamline test design and creation. Together, the collaboration helps keep the tests lean and focused on incremental changes.

• Document testing decisions – Devada uses forms to document the testing decisions resulting from Developer and QAE discussions. The documentation helps keep the team focused on testing needs and minimizes unnecessary test duplication.

Using these best practices along with Testim’s AI-based test automation platform, Devada increased its testing speed, improved trust, and is releasing higher quality code.

Benefits realized using Testim:

• Reduced their bug escape rate by ~ 70%

• Shortened testing in release cycles by approximately three days

• Reduced labor involved in UI testing time by ~ 85%

• Lowered monitoring and maintenance of UI tests to 16 hours every two weeks

• Minimized training time for new users