Test Strategy vs Test Plan: A Comprehensive Guide
In software testing, two key documents can play a crucial role in shaping the testing approach for a company and its projects - the organisational test strategy and the project level test plan. Understandably, there is often confusion both in and out of the test team about each document’s purpose and exactly what should be contained in either one. Despite their similarities, it is essential to note that they have distinctive objectives and varying areas of emphasis.
In this blog post, we will explain the difference between a test plan and a test strategy, their importance, and how they complement each other.
Organisational Test Strategy
This is a high level, concise but technical testing guide that provides clarity and structure on how testing should be planned and executed within a company.
The Organisational Test Strategy exists at a corporate level, providing generic testing protocol for projects within its scope. It is not designed to be specific to any given project.
For a smaller company, the test strategy may cover all testing activities that may arise throughout delivery. A global organisation spanning numerous industry verticals may see fit to produce a number of Test Strategies as warranted, due to the differing nature of the respective development streams or the criticality of the products and software that each delivers. Likewise, differing areas of the business may author separate test strategies if one follows a waterfall model and another employs agile delivery methods.
Regardless, a strategy should contain practice and procedures for any type of anticipated testing sub-processes that will commonly exist at the underlying project level. For example, whilst not project specific, it can make test phase strategy statements about unit, integration, acceptance, accessibility, performance and automated testing amongst others.
Advantages of a Test Strategy
A widely adopted test strategy within an organisation can be a good indicator that testing and quality are held in high regard and that all senior stakeholders are bought into the core concepts.
All underlying projects will benefit from a clear testing mandate from the very start. Senior executives and product owners should be able to see a commonly applied approach to testing across multiple deliveries that have all fallen into line with boundaries established in the overarching test strategy.
If a new project delivery team is struggling to establish its testing ways of working, it should simply be able to refer upwards to the strategy for clear guidance.
Key Components and Steps For Drafting a Test Strategy
Follow the below instructions to create a robust test strategy.
Introduction, Scope, References, Glossary
Project-wide organisational test strategy statements
The strategy is defined for the specified scope. Incorporates guidance that is relevant for all underlying test sub-processes to be performed for a project within the scope of the organisational strategy.
Generic risk management
Outlines an approach to risk management to be adopted within the testing activities.
Test selection and prioritisation
Describes the organisation’s approach to selecting and prioritising test execution, in the form of prioritised test procedures. Test procedures consist of prioritised test cases, derived from prioritised feature sets via prioritised test conditions and coverage items.
Test documentation and reporting
Identifies the documents expected to be produced during testing for the test project as a whole. Outlines when these are produced and the relevant signoff process. This is tightly connected to the test process that is specified in the policy.
Test automation and tools
Outlines the organisation’s approach to automated testing. Identifies the testing tools to be used throughout testing.
Configuration management of test work products
Describes the configuration management to be performed for the work products from testing; describes how these work products are to be identified, traced, stored, and made available to stakeholders.
Incident management
Outlines how incidents should be triaged throughout testing.
Test sub-processes
Identifies specific lower level test phase sub-processes to be undertaken within organisation testing falling within the scope of the strategy.
Entry and exit criteria
Outlines the rules to ascertain when testing should start and stop.
A test sub-process consists of the following processes:
Test design & implementation
Test environment set-up & maintenance
Test execution
Test incident reporting
Different entry and exit criteria may be defined for each of these individually, or for selected ones, or for the entire sub-process as a whole.
Test completion criteria
Outlines how the company decides that the testing for the test sub-process to be finished.
Test documentation and reporting
Identifies the test documentation, including reporting, used for testing activities in the test sub-process.
Describes when each document or report is prepared and the associated approval process. This is tightly connected to the test process specified in the policy.
Degree of independence
Sets the level of autonomy of the testers. States how this testing group is technically, managerially, and financially independent.
Test design techniques
Identifies specific test design techniques to be used during test design and implementation within the test sub-process.
Test environment
Identifies the test environment for the test sub-process; may state where specific types of test should be performed, and identifies groups or organisations responsible for the test environment.
May identify the origin of test data, state where particular types of test data are located, and groups or organisations responsible for test data.
Metrics to be collected
Outlines the metrics to be collated and managed throughout testing sub-processes.
Retesting and regression testing
Documents the retesting and regression testing strategy, activities and conditions for any lower level test phases.
After developing an organisation test strategy, review the following in order to create a project level test plan.
Test Plan
The Test Plan provides a test planning and test management document. Some projects may have a single test plan, while for larger projects multiple test plans may be produced. Individual test plans can be utilised across multiple projects (at the programme level), or to a single project (project test plan/master test plan), or to lower level test phases (system test plan, integration software test plan, performance test plan, accessibility test plan, unit software test plan). If more software test plans are created, a mapping tree may be produced to aid documenting relationships and the information contained in each.
The Test Plan describes the decisions made during the initial planning and evolves as re-planning is performed as part of the control activity.
Advantages of a Test Plan
A test plan holds enormous importance in software testing and clarifies the testing objectives, scope, and deliverables, enabling the testing team to align their efforts with the goals of the project.
Some benefits of using a test plan:
A test plan offers a structured and systematic approach to testing software.
A test plan allows you to use resources wisely, schedule tasks, and manage your time throughout the testing lifecycle.
A test plan assists in reducing the risk, and identifying the issues early.
Key Components and Steps for Writing a Successful Project Test Plan
Examine the product and its requirements
Testing a product without prior knowledge can be troublesome. Examine the product's requirements, design, primary features, and operation in great detail.
Review the Organisational Test Strategy
A successful test plan requires a valid and authentic test strategy. Create a thorough test strategy with an overview of the testing methodology, approach, and strategies.
Project/Test Sub Process
Identifies the project(s) or the test sub-process(es) for which the plan is being written and other relevant contextual information.
Objective and Approach
Clearly state the aim and approach of the test plan. It involves verifying specific functionalities, validating system performance, and ensuring compatibility.
Test Items and Test Scope
Identifies the test item(s) for the testing covered by this plan including their version/revision or reference where this information can be found. Scope summarises the features of the test item(s) to be tested. It will also document any test exclusions and reasons why.
Assumptions and Constraints
Outlines any assumptions and constraints for the testing documented within this plan. These may include regulatory standards, the requirements in the Test Policy and the Organisational Test Strategy, contractual requirements, project time and cost constraints, and availability of appropriately-skilled staff, tools and/or environments.
Stakeholders and Communication
Documents the stakeholders and their interest in the testing and outlines how communication with stakeholders will be undertaken.
Risk Register, Product Risks, Project Risks
Highlights the risks reviewed by the testing within this plan. This should include any relevant risks that may be specified in the Organisational Test Strategy. Provides an exposure level for each risk based on its impact and probability. Provides recommendations to treat the risks. This section may reference where a separate risk register can be found.
Documents test-related product risks and provides guidance to triage each risk.
Highlights project testing risks and with associated resolution where possible.
Test Deliverables
Identifies all documents that are to be delivered from the testing activity or equivalent information to be recorded electronically, for example in databases or dedicated test tools.
EXAMPLE The following documents could be included:
Test Plan
Test Design Specification
Test Case Specification
Test Procedure Specification
Test Data Readiness Report
Test Environment Readiness Report
Incident Reports
Test Status Reports
Test Completion Report.
Test Data.
Test tools created as part of the testing activity may also be included.
Test Design techniques
Documents which test design techniques can be employed.
Entry/Exit Criteria
Specifies the requirements that must be met before testing can start, such as the availability of a particular build or the conclusion of a list of required development tasks. Describes the conditions under which the relevant test organisation considers test execution activities to be complete.
Metrics
Describes the metrics for which values are to be collected during the test activities.
Data Requirements
Specifies all relevant test data requirements for the project or test sub-process (as appropriate).
Environmental Needs
Includes environmental requirements that ensure testing operations may be carried out successfully.
Retesting and Regression Testing
Specifies the conditions under which retesting and regression testing will be performed. May include anticipated test-cycle estimates.
Suspension and Resumption Criteria
Specifies the criteria used to suspend and resume all or a portion of the testing activities in the Test Plan. Identifies who is responsible for suspending and resuming testing activities. Specifies the testing activities that may have to be repeated when testing is resumed.
Deviations from Strategy
Records any Test Plan content that deviates from the Organisational Test Strategy. Identifies the authorities responsible for approving deviations, where applicable.
Activity and Estimates
Identifies all necessary testing activities based on the test process to be used. Any retesting should also be considered alongside any general dependencies.
Staffing, Roles, Activities and Responsibilities
Describes the staffing requirements for the testing covered by this plan. Documents an overview of the primary and secondary teams undertaking the testing roles with their responsibilities.
Hiring and Training Needs
Identifies specific requirements for additional testing staff that are necessary for the test project or test sub-process. Specifies test training needs by skill level and identifies training options for providing the necessary skills for the staff needed.
Schedule
Documents testing phases and project checkpoints defined in the delivery schedule. Summarises the overall schedule of the testing activities, identifying where activity results feed back to the development, organisational, and supporting processes. Specifies the schedule for each testing phase based on estimates, resources and dependencies.
Test Plan vs Test Strategy: Major Differences
| Parameter | Test Plan | Test Strategy |
|---|---|---|
| Purpose | A detailed document that defines specific activities, tasks, and resources required for a phase of testing | A high-level guide with statements that define how a company will plan and perform different types of testing across verticals, teams and products |
| Key Elements | Objectives, Test Items, Test Phase, Scope, Risk, Roles, Criteria for Entry and Exit, Test Environments, a Schedule. | Documentation, Testing tools, Configuration management, High level risk management, Test team autonomy |
| Level | Project level | Organisation level |
| Ability to Change | Frequent, on demand, by project | Maybe once or twice per annum as team or technology changes |
The points above highlighted clear differences between the test plan and test strategy.
Worth reading: Software Quality Management Best Practices
Conclusion
The test strategy explains the strategic approach and principles driving the testing activities inside for an organisation. A test plan offers the general structure and specifics of the testing effort for a particular delivery or phase of testing within a project. The test plan and strategy are both critical components of solid software test planning. They serve different roles but play a valuable part in ensuring a successful testing procedure.
