To automate or not automate, this is the question...

Whether 'tis nobler in the mind to suffer
The slings and arrows of manual testing
Or to take arms against a sea of automated test
And by developing them. To die, to pain--

.....


Anybody that has tried automation knows that automation needs to be done carefully. Managers, convinced by automated tools salespeople and Start Treck voice activated computers will try to force you to automate everything. The arguments will be the usual ones of increase productivity, results reliability and speed. However the land of automated tests is littered with corpses and failed projects.

Why? Well, based on what I have seen the foremost reason is that test automation are projects that need to be though, managed, staffed and planned as any other software development project. To be useful they need to answer the following questions:

• Will the test be done a limited number of times or it is a test that needs to be executed regularly?
  
• The requirements for the application under test, will change soon? change with time? are still undefined?
  
• Is the code under test architected in such a way that a change in a component will not affect our automated test?
  
• Can the code under test be automated with an off-the-shelve software?
  
• Have the code under test be instrumented? Does it have hooks that can be used?
  
• Is the automated test code modular?
  
• Is the automated test code easy to maintain?
  
• There is time allocated to document your automated test?
  
• Are you testing your automated tests?
  

These are basic considerations that need to be answer before proceeding with an automation project. However, before this considerations it is fundamental to consider the first point Ihave made: Any automation exercice as a project in itself. This means it need to have the appropriate level of staffing, project management supervision, methodology... Failure to plan the automation project appropiately will incur the same risk as any other project: overworked staff, last minute problems, cost/time over runs...

So, my sincere recommendation to anybody trying to automate the testing of their new piece of software: treat it seriously as a subproject of your main project

Do not give [free] honest advice/assesment in job interviews

Ok, this is a rant, so feel free to skip it!

About a year ago I had a job interview for a contract position. The project was complex, ill organized and with high probability of failure. A brief description of the project is:

Company A starts a job for a customer. They were going to adapt their exisiting product, based on US business processes, to the Canadian business processes. After a year, and no progress, company B purchases company A. Another year pass with nothing to show. It is now two years after the project has begun and the customer still has nothing. When company B realizes the mess, decides to cut their loses and hires company C to finish the project. Customer sets up a team to speed the project that will coordinate internal resouces with company C, who incidentlly are located in a different time zone.

At this stage I interviewed for the QA position in the coordinating team. Among other responsibilities, it includes testing the code that Company C is producing. There were no procedures, vague specs and the group of developers used the word Agile methodoly to describe what is a cowboy approach. The PM, working for the customer, has no carrot no stick over company B or company C. My assesment of the situation was somehow bleak, and I proposed measures like change control, regular risk meetings, clearly defined [and achievable] goals for each iteration... I know I sounded a bit negative, but definitely I didn't wanted to sound like all was fine: it wasn't.

A year later, and because Vancouver is an small IT town, I have learnt they hired another person that had 'Don't worry, be happy' as mantra. Six month after the hiring the PM realized it was not working. So, they started to do risk meetings, put in place a change control and each iteration has a small number of features that had to be implemented. It has took over three year for the customer to have something that is operational and can be put into production.

Conclusion: next time I want the job I need to smile, assuage all fears with 'It's going to be fine' and send a bill for any advice I may give during the interview.

Adding/modifying work items in Team Foundation Server -Part 2

In the previous blog I explained how to create a 'Steps to reproduce' control. Lets now add this control to default Bug Work Item (WI). In this case, I want to add 'Steps to reproduce' as a new tab.

If you go to the Layout tab, you will get something like the following image

It is easy to read. The TabGroup refers to tab structure that previously I have qualified as not meeting the tester’s needs. Now that we have created the 'Steps to Reproduce' control, we can add it to the layout.

To add a new tab element, right Click the ‘TabGroup’ folder and choose ‘New Tab Page’. A new ‘Tab Page-New Tab Page’ element will appear at the bottom of the ‘TabGroup’ list. On the right pane, change the label to something more meaningful, for example ‘Steps to Reproduce’. Note: this simply adds a new container, in this case a tab, where we can place control.

Now, we are going to add the previously created 'Steps to reproduce' control we created in the first part of this tutorial to this new tab. To add this control to the tab, right click the ‘TabPage- Steps to Reproduce’ and choose ‘New Control’. The property pane appears, change the label to ‘Steps to Reproduce’, and choose the Visiphor.Steps we created for the field Name field.

Accept everything, and if you click the ‘Preview Form’ button, you will get the ‘Steps to reproduce’ tab added to the tabs section

If you want to move this tab to be the first, right click the ‘TabPage – Steps to Reproduce’ folder and ‘Move Up’ as many times you want.

At the end, you can build a bug work item like

That fits much better my needs as a tester!

Notice the new tab 'Steps to reproduce' contains the control 'Steps to reproduce'

Hope it will work well for you too.