It’s no secret that many software development teams now favor the agile methodology as compared to the traditional waterfall methodology. Agile, being more adaptive, makes product development more efficient by breaking down the development process into smaller, iterative cycles or sprints. In order to follow this approach, it becomes essential to test often and test fast and keep the focus on continuously improving the product from iteration to iteration. Considering that there are new features that are added after each sprint, the testing process in such development environment has to be agile as well to enable continuous development of the product.
In Agile development developers need to ensure that each new change that is implemented during each sprint works in harmony with the other parts of the application and does not impede the application functionalities. Given the sheer volume of tests that need to be done during product development in agile, looking towards test automation becomes not only necessary, but also inevitable. Test automation facilitates the frequent build and release process both for system and integration tests. However, implementing test automation in agile development needs careful planning as the test object keeps changing continuously with the sprints. So what should we focus on when defining a test automation strategy for agile?
Identify what to automate
As much as we would want to, automating all tests is neither possible, nor desirable. Instead of automating all the tests from the word ‘go’, we need to determine what we need to automate first. A few things to consider when determining this are:
1. Which functionality is critical to the business?
2. Which functionalities will be used most frequently by the users?
3. Which tests need to be run using different configurations?
4. Which test cases need to run several times using different test conditions and data?
5. Which test cases will show the expected result fastest?
In the traditional Test Automation pyramid, a large portion, sometimes as much as 80% of the automation is centred on UI tests. These functional tests explore the application via the GUI. Given the comfort of the testers around functional testing, the traditional test automation strategy places little emphasis on the middle and the lower tier of the test automation pyramid. However, considering in agile development the application changes frequently, the UI changes and this inevitably impacts the automation by impacting the stability and increasing the time and maintenance costs.
When executing test automation in agile, it, thus, makes sense to flip the pyramid and place a greater importance on unit level automation, using that as an entry point for automation. The Service and Middle Tier sees approximately 20-40% of automation and consists of Acceptance, Integration, and Component tests. UI testing, thus moves to the upper tier of the pyramid.
In agile development, unit tests need to be run after every build. Running automated tests at the lower level allows developers to check their work as they proceed, identify where the bugs are, and implement the changes. Bugs have a much shorter lifespan here since they don’t have many places to hide.
Once the unit tests are complete, testers need to check that all the components of the product are functioning in harmony with one another with the help of API/ integration/ component testing. The business logic of the product is tested here without even going to the UI. Automating this test layer requires skill as they demand fast test execution. However, once despite being slower and more complex than unit tests, they are not as brittle as UI tests and hence lend themselves easily to automation.
Considering that releases in the agile product development are frequent and fast, it can be easy to base the automation suite only on unit tests as this is the best code base to build the suite on. However, doing this limits the scope of automation and the immense value it can bring to product development. For automation to deliver value, it has to address system tests, integration tests, and the obvious functional and regression tests as well. To do this effectively and to avoid technical debt, taking the ‘test first’ approach ensures that testing is integrated into the product development process from the very start.
A robust test automation suite is critical for the success of agile development. However, it calls for taking an iterative approach with testing. Creating meaningful tests, selecting the right test automation tools and keeping an eye out for maintainability and execution ease are critical for success. In closing, we can say that given the planning requirements of this approach, the foundation of a great test automation strategy starts with taking a ‘whole team approach’ – where the developers and the testers have to work together as a team to deliver the promised benefits of the agile approach.