Aaron Greenwald, a software engineer at Microsoft takes us through some of the test automation theory and decision making processes that led to using Testim. He explains why slowing down and creating test coverage can actually improve quality and confidence so that the team can move faster.
On this journey from zero tests to a quality-focused team that requires tests on every pull request, Aaron discusses some of the lessons they learned. From test architecture to selling value to management, this talk provides tips for everyone.
Some key takeaways from the webinar:
Microsoft made the decision that test automation was not only important but it was something that should be part of the development process, and not pushed to another quality team. This is why developers are responsible for writing their unit, integration, and end-to-end tests.
Where to get started on testing?
Microsoft focused on three key attributes to identify the highest risk areas to approach first:
- Fragile code. Not all code is equally likely to fail. The team had a pretty good idea of where their code was more prone to bugs, so they decided to start there.
- Frequent changes. Areas in the application that change more often are more prone to errors.
- Complex or confusing code. Some code is more complex or more confusing than others, perhaps because it’s more novel, requires certain expertise, or was written by people who aren’t actively involved.
How to write tests?
Tests are code too, so it’s important to follow good coding practices.
- Use short, readable names that make sense.
- Each test should have three parts: setup, when, then
- Use groups to capture repeatable test steps
- Keep the tests short by using groups to hide the “mess”
- Use descriptive failures so that messages in the CI are clear and lead to an action
When to run tests?
Integrate into your pipeline so that you can run automatically.
- Run shorter tests on each pull request. These tests run on a mock server, are fast, lightweight tests and the data is predictable. Only merge the PR if all tests pass. This ensures that the code you are adding to the main branch is solid.
- Run nightly tests – on a real server, live data, completely E2E. These tests help check for regressions, and broader, real-life use.
