Designing an Agile Test Automation Strategy in 2023 - A Comprehensive Guide Part ll
This is the second part of our three part series. If you haven’t, make sure to check out
parts one and three of this guide as well.
Introduction
In today's fast-paced software development world, mastering agile automation is crucial for organisations to stay competitive and deliver high-quality products quickly. Testers play a crucial role in this process by closely collaborating with developers and infrastructure teams to build and manage testing environments.
In this article, we will explore test planning and execution, the automation pyramid, CI/CD pipelines and key considerations and strategies for testers to contribute to the automation efforts within an agile environment effectively.
Planning and Execution Considerations
Good planning and execution are important for successful automation.
Considering your test framework choice, data management and the skills of your current testing team is crucial. Think about your skills and available resources before diving headlong into solution choice.
What are Automation Frameworks?
Any good automated testing strategy must include a solid automated testing framework. Implemented correctly, they can deliver a significant return on investment (ROI) for QA teams adopting an agile method delivery, by improving test repeatability and consistency and lowering test maintenance, test cycle expense and manual testing workloads.
Before choosing an automation solution, these questions can help you explore your tooling requirements to help refine your selection.
Should I build or buy?
Is it open-sourced or licensed?
What technologies does it need to support?
What kind of non-functional testing must it support?
What is the budget for training?
What is the learning curve like?
How easy are the tests to maintain as the application changes?
Does it give enough comprehensive reporting?
What is the Testing Environment?
The test version of the application against which software testing takes place is referred to as the testing environment. It is typically a scaled down, controlled environment, designed to part way resemble the eventual production environment, allowing testers to check the functionality, early performance and behaviour of the programme or system under test.
You’ll likely need support to build, manage and test against 3 or 4 different environments and testers should work with the Infra team to do this. Ultimately, Testers should own the environment deployment process and decide which features get deployed and when they get deployed for testing. This ensures a stable environment for good quality repeatable testing.
Ensure you have sufficient production-like sanitised data for easy running of tests (products/inventory, content, test users, test payment methods etc.)
Key Considerations and Strategies
The Test Automation Pyramid
The automation pyramid, pioneered by Mike Cohn, provides a framework for structuring automated tests at different layers of the software stack. Unit tests form the foundation of the pyramid, followed by API tests and UI end-to-end tests. Achieving a balanced ratio of tests at each layer allows testers to then focus on manual exploratory testing, ensuring comprehensive coverage and high-quality software.
CI/CD Pipelines
To release new features quickly, it's important to establish a CI/CD pipeline. A new feature can be checked in by a developer and the environment can be automatically rebuilt. Testers, developers and infrastructure teams work together to make sure the latest, priority driven and correct automation tests are then triggered for execution against the specified target environment and reported on. For extremely mature Agile organisations, a successful feature commit, rebuild and test run can lead to immediate production feature deployment.
Infrastructure, Developer and Tester Collaboration
Testers need to work closely with infrastructure teams to set up and manage pipelines for deploying new features to testing environments. By taking ownership of the deployment process, testers gain better control over the environment and ensure alignment with sprint intentions.
Additional benefits
Collaborating closely with developers also enhances testers' product knowledge by providing them with insight to the inner workings of the application stack, enabling them to adopt a white-box or glass-box approach to testing and facilitating the improved questioning and probing within their exploratory testing.
Building the Right Team
To succeed with agile test automation, companies should hire testers who have a desire to step up and above their current manual testing remit and become agile test automation engineers. Testers should be open and excited to embrace new technologies to improve their testing skills.
Results of the train-the-trainer approach
Encouraging a train-the-trainer approach can facilitate knowledge sharing and enable rapid upskilling within the testing practice. Do you have a natural test-lead to drive the team to an agile way of working and train the trainers?
Tools for Agile Automation
Choosing the right automation tools is essential for effective agile testing. While there are various tools available, testers should consider factors such as mobile app support, desktop app compatibility, visual testing capabilities, accessibility testing and language support.
Evaluation of tools based on their ability to integrate into the existing ecosystem and provide comprehensive reporting features should also be a priority.
Quality and Traceability
Effective test automation is meaningless without comprehensive and real-time reporting. Agile testing demands quick access to test results, enabling stakeholders to make informed go, no-go decisions promptly with all of the facts. By establishing traceability between test reporting and source requirements, organisations can provide transparency regarding test coverage and clearly demonstrate the extent of testing performed for that release.
Results
Investing in a quality reporting platform empowers teams to deliver meaningful insights that go beyond simple and arbitrary pass and fail metrics. Application context, risk and test coverage clarity enable stakeholders to assess application quality rapidly and with great confidence.
Inspecting, Adapting, and Upskilling
To evolve and grow an agile test automation strategy, organisations should maintain, improve and increase their automated tests, fixing any inconsistent tests and addressing any environmental challenges. Do not continually tolerate failing tests.
Tests only fail for 3 reasons
they have either detected a bug
they need to be more resilient
They are running when they shouldn’t be against a feature that is no longer relevant
Ensure the test team is invited to requirements discoveries, backlog prioritisation and sprint planning. Determine the risk profile for your new feature. Explore manually first, raising any obvious issues and then build your new automated tests.
Conclusion
Testers play a key role in the successful implementation of automation testing in an agile development environment. By working with infrastructure teams and developers and using the right tools, testers can integrate their risk driven testing into the CI/CD workflow. Balancing automation with manual testing ensures comprehensive coverage and aids in the delivery of high-quality software delivery. Agile automation principles empower organisations to release features quickly while maintaining quality.
