To automate or not automate, this is the question...

Whether 'tis nobler in the mind to suffer
The slings and arrows of manual testing
Or to take arms against a sea of automated test
And by developing them. To die, to pain--

.....


Anybody that has tried automation knows that automation needs to be done carefully. Managers, convinced by automated tools salespeople and Start Treck voice activated computers will try to force you to automate everything. The arguments will be the usual ones of increase productivity, results reliability and speed. However the land of automated tests is littered with corpses and failed projects.

Why? Well, based on what I have seen the foremost reason is that test automation are projects that need to be though, managed, staffed and planned as any other software development project. To be useful they need to answer the following questions:

• Will the test be done a limited number of times or it is a test that needs to be executed regularly?
  
• The requirements for the application under test, will change soon? change with time? are still undefined?
  
• Is the code under test architected in such a way that a change in a component will not affect our automated test?
  
• Can the code under test be automated with an off-the-shelve software?
  
• Have the code under test be instrumented? Does it have hooks that can be used?
  
• Is the automated test code modular?
  
• Is the automated test code easy to maintain?
  
• There is time allocated to document your automated test?
  
• Are you testing your automated tests?
  

These are basic considerations that need to be answer before proceeding with an automation project. However, before this considerations it is fundamental to consider the first point Ihave made: Any automation exercice as a project in itself. This means it need to have the appropriate level of staffing, project management supervision, methodology... Failure to plan the automation project appropiately will incur the same risk as any other project: overworked staff, last minute problems, cost/time over runs...

So, my sincere recommendation to anybody trying to automate the testing of their new piece of software: treat it seriously as a subproject of your main project