The practice of applying automated testing methodologies and technologies within an agile development context is referred to as agile test automation. Our objective is to assure that any delivered software is critical bug free.

As Agile methodologies now dominate software development lifecycles, it is essential that each project has its own test automation strategy. This blog post will explain how to design an agile test automation strategy in 2023. It will cover the areas of intent, scope and risk scoring to help teams decide what to automate. 

Understanding the Purpose of Agile Test Automation

When starting to adopt an agile test automation strategy, it's important to understand the organisation's goal. Below are some considerations, some of which may be important factors for your company and can help in defining what your team is trying to achieve. 

Let's dive into some of the most common reasons for embracing agile test automation: 

• Automation helps testers focus on important exploratory testing tasks and saves time by eliminating repeated manual, lower-risk testing.

• It allows testers to contribute more strategically and identify problems more efficiently.

• Automation regression testing provides quick feedback on application quality.

• It promotes better communication and teamwork between testers and developers.

• Agile test automation allows the addition of example driven positive and negative scenarios for a rapid increase in test coverage

Identify What to Automate

While automation can streamline testing processes, it is essential to identify what should and shouldn't be automated. 

Testing principles such as avoiding exhaustive testing, focusing on code complexity, and being context-dependent can help testers make informed decisions about which tests to automate. Exhaustive testing is impractical and frankly impossible, even with automation capabilities. Instead, testers should strategically allocate their time and effort to the automated testing of complex areas of code or high-risk functionalities. 

What to Automate

• Critical function tests in high-risk custom code elements

• Non functional cross-browser and cross-device tests 

• Tests that must be run against each build in each environment

• High setup, long running, laborious manual tests

• A test that needs extensive data-driven iteration to be fully validated

What Not to Automate

• Tests for single use

• Unpredictable tests

• Usability assessments

• Exploratory tests

• Something that must be tested immediately

Things to Consider

In mature agile organisations, 70-80% test automation is common, but it is still risk-based and tests are written when they are strictly necessary and add value.

Risk Scoring and Critical Test Packs

Simply put, an application feature’s risk score is determined by multiplying its ‘Impact of Failure’ rating (1-5) by its ‘Probability of Failure’ (1-5) rating. 

Probability or Likelihood Considerations

Is it especially complex?

Were the requirements well drafted?

Is it new, has it failed before?

Are we using a proven development team?

Impact Considerations

Which customers are impacted - internal or external - both?

Will this damage our brand?

Will this impact our revenue?

Will this cause legal compliance issues?

It is the quantitative statistic that makes risk-related test prioritisation easy. 

What is the Importance of Knowing Your Functional Risk Score?

Testing on any project requires comprehensive risk analysis and functional understanding. Understanding your feature risk scores helps you to become aware of the potential risks that should be appropriately mitigated by sufficient test coverage. When building automated tests, prioritise the creation and execution of highest risk score tests first before addressing the lower score tests. 

Conclusion

To succeed in test automation in 2023, it’s important to have a plan. Decide which processes to automate and understand why. Plan and execute well, use the automation pyramid and CI/CD pipeline. Transparent reporting and traceability can help demonstrate quality. Ongoing inspection, adaptation and training are critical for success. Agile test automation can help create high-quality software.

Our second article of the series will cover additional tools and considerations for agile testing automation. Our discussion will include cross-browser and cross-device testing, visual and accessibility testing and the importance of programming language choice in automation. We will also take a closer look at the CI/CD Pipeline.

We invite you to join us for part two, where we will delve deeper into these areas.