A colleague had asked the following Q in several Testing and Test Automation focused LinkedIn groups about the state of Functional Test Automation today and the responses were Overwhelming (yes – with a capital O).
Title: What’s wrong with Functional Test Automation today?
(Download the document with all valuable responses)
“Most software projects today include a consideration for automating functional testing. There’s enough anecdotal and other evidence out there though that many of these projects just don’t deliver the expected benefits. Why do you think that is? Some thoughts that came up while talking about it in our company are an increasing degree of isolation from the business needs, adding automation as an after-thought and expecting to achieve 100% automation. What do the testing & automation experts in this group think could be the main issues?”
Given the depth of feeling displayed and the trouble people had taken to put down their own opinions it seemed like a good idea to collate these responses and offer them for everyone as a kind of digest. Going through the responses people have expressed there seems to be some kind of consensus forming around the following areas as being most to blame for the somewhat less than stellar success of Functional Test Automation today:
- Expectations mismatch: Management goes into Test Automation projects with expectations of all sorts of miracles to follow. This has an impact on the design, development and execution of the test strategy. If the expected results are not clear then what do you measure against to check if the whole effort has been worthwhile?
- Test automation Myths – 100% test automation: A lot of people have mentioned this – clearly not everything in testing can be automated and efforts to do so are doomed to failure even before they start. Wasted effort, project overruns and a bad name to the entire Automation effort will inevitably follow.
- Test automation – Preparedness – Approach, Strategy, Tools, Framework: People seem to agree that the Boy Scouts know what they are talking about when they say “Be prepared”. Planning for the Automation effort is critical and the plan should cover everything from the tools and technology to the strategy.
- Test automation maintenance: The feeling was strong around this point – the fact is the Automation suite, one developed and put in place, needs constant attention just like any other product. Failing to make an allowance for the maintenance of the suite could prove extremely problematic.
- Who should automate – Developers or Testers with functional knowledge: An interesting point came out several times about who should be responsible for the Automation? Assuming the automation suite is a product then should it not be developers with functional knowledge who take this up?
So there you have it folks – the voice of the testing community on the state of the functional test automation game today. All-in-all download this invaluable resource – so much information from so many experienced people in one place is truly rare to find!