Organizations are under constant pressure to get flawless and bug-free applications and to go to market faster with their software products. Organizations are leveraging new development methodologies such as Agile, Test Driven Development, Behavior Driven Development etc. to reduce time to market without impacting the quality of the application. It is interesting to note that all these methodologies put software testing at the heart of product development. Given the increasing complexities of the IT ecosystem, Automation Testing has become of paramount importance to achieve development agility. This is a prerequisite for faster releases and to ensure that the product is flawless. Automated testing ensures increased test coverage and gives a clear picture of actual outcomes versus predicted or expected outcomes for a piece of code. By implementing a standardized testing process that leverages automation, development testing time can be reduced significantly while still ensuring that the final product matches the user expectations.
Just like product teams have to focus on software development lifecycle management, testing teams too need to look at Test Automation Lifecycle Management. This is a methodology that helps testers take a structured approach to automated testing and helps them design a process that helps them avoid unplanned adjustments in the test process and maximize test coverage by optimally using their test resources. This lifecycle covers many stages as under:
In order to develop a strong test automation suite, it is essential to conduct a feasibility analysis to determine which test cases are well suited for automation. It is also essential to identify which modules of the application have to be automated, how they should be automated, and which tests should be conducted manually. During this analysis, the testing team should understand what is expected of the application and then decide which automation testing tools to use in the automation process taking into consideration the team size and the costs for tools. If the testing team does not have advanced programming skills, they can evaluate scriptless test automation tools to achieve the same test coverage and testing outcomes.
Test Planning, Design, and Development
Once the feasibility analysis is complete, the testing team has to create a strategic test plan. This consists of identifying which framework to use, creating a test suite for automated testing in the test management tool, creating a detailed test plan that outlines the application background, and the limitations, risks, and dependencies between the application and the tool. Finally, the testing team has to get this evaluated by the invested stakeholders to ensure that the software testing suite that is created is comprehensive and takes into consideration all essential end points of the application.
When planning the test cases it is important to define the test steps, display the test data, identify validation criteria, store test artefacts, define the roles and responsibilities, project the test schedule, identify test acceptance criteria, define the test procedure format standards, and ensure that the test plan provides complete traceability from requirement to test analysis. Implementing a test design and plan that follows design standards allows testers to develop test scripts that are reusable for incremental software evolution.
Execution of test scripts and test analysis
Some points need to be kept in mind during test script execution. During automation testing, the need is to ensure that the test script covers all relevant functional requirements of the test case and the script is stable so that it can perform in multiple test environments and browsers. Finally, the testing team has to analyze the test outputs and assess the time taken to complete the test cases and to generate a comprehensive report that highlights the requirement coverage, incident status, and test execution metrics.
Defect Lifecycle Management
It is also important to have a strong defect life cycle management process. The defect life cycle is related to the identification of a bug during the testing process and is the journey of that defect through its lifetime. Defect life cycle management involves the linking of the defects to the test cases and the software requirement.
A bug has to go through several stages from identification to fixation:
1. The ‘New’ State – When a bug is identified and posted for the first time, it is given the ‘New’ status.
2. The ‘Assigned’ State – Once the lead tester identifies the bug as genuine it is assigned to the corresponding developer.
3. The ‘Open’ State/- During this stage, the developer is working on fixing the bug by analyzing the defect
4. The ‘Fixed’ State – Once the developer has identified the problem with the bug and made the necessary code changes and has verified those changes he assigns the ‘Fixed’ status to the bug.
After this, the fixed bug is re-sent to the tester for evaluation wherein the tester conducts retesting of the altered code and verifies if the bug fix has taken place. In case the bug still exists, the tester ‘reopens’ the bug and ‘closes’ the loop only once he/she feels that the bug no longer exists in the piece of code. We shall take a deeper dive into Defect Lifecycle in a subsequent blog.
When planning a test automation lifecycle it is also essential to take into consideration the expected path of evolution of the product. Thus, testing professionals have to identify which test cases will remain relevant, which can be reused and which will become redundant in the light of product iterations, upgrades, and fixes. To increase the lifecycle of an automation testing suite, testing professionals thus should steer clear of monolithic test plans and instead have smaller and independent test cases to increase the efficiency of the automation initiative. Identifying test automation maintenance needs like preventive, corrective and adaptive maintenance help in extending the life of an automated testing suite.
It seems apparent that just as the product evolves, so does the test strategy, the test automation, and even the defects – the need is to manage the complexity across the entire life cycle for each of these!