04 June 2007

Second part of the list, as promised! I feel a bit less strongly about the recommendations in this second list. In some cases, I would advise a 'menu approach' that will allow the team to pick and choose.


As before any omments, suggestions and real life stories will be most certainly welcomed!





ActionDescriptionWho is responsible?Phase
Code CoverageThe new Visual Studio suite offers the possibility to explore all the possible paths of a piece of code. If all the lines of code are executed at least once in a test environment, it is less probable that they will contain critical errors. Also, if there are areas of the code that are never accessed, it would point to areas where maintenance can be compromised.DevelopersDevelopment
Tracing requirements to Use Cases/Test Cases This would add overhead to the QA operation, however, it may help to keep all the requirements in sight and also limit their scopeTester/PM

Planning, Development, Stabilization

Sign on at each milestone. My proposal would be that at each milestone the PM MUST send an email to the team announcing it. This means that a change in design (Planning) after the ending of the Planning phase will not affect developing. The change will affect next iteration (Thinking in MSF context).
This may seem draconian, but may help architects, PM and Customer to be more conscious when they are breaking the process.

PMEnd of each phase inside the iteration
No switching of resourcesEvery time that resources are brought in/out of the project, there is a learning curve and information that is lost. A project with a big resource turn-over risks to have a larger amount of undetected issuesPMAll phases
Representative Test environmentSometimes, due to the complexity and expense of the production system, the test environment is significantly different from the Production Environment. If the environment are similar, the test effort twill be much more accurate
PM, System Engineering, TestingStabilization
Risk ManagementCore to the MSF, it should include the customer/customer representative. The extra overhead is well compensated by the managed approach to problem when, if they, appear. PM, EverybodyAll phases
Real and represenative test DataIf the customer supplies real and representative test data, it is easier and more probable to spot problems/code deficiencies during the test process PM, TestingDevelopment, stabilization
Customer agreement on qualityCustomer should take an active role defining what they expect of the product. PM

Envisioning

Formal Security Review (if applicable)For project that require a high level of security, in addition to the usual test procedures –encryption, use of certificates, users rights…- it also should be submitted to deliberate attacks to try to break into the applicationSys Eng/TestingPlanning, stabilization, deployment
Cross disciplinary Peer review of documentsThis could identify issues that are more obvious to another team.EverybodyAll phases

25 April 2007

Despite the name, most of QA groups/departments in the software industry are not QA departments at all. They are a testing group that design, implements and execute test plans, test cases and test scripts. Nothing wrong with that indeed!. However, the next step to improve the quality of the software is to go beyond the testing phase and create a culture of quality among all the participants of the software development process.


How is possible to begin this culture of quality? Ideally you will implement any of the existing models (CMMI, ISO...), however, given the amount of effort and time (so, money!) involved, it is usually difficult to get committed support from senior management. A group that incidentally looks at testing and QA as non-core/non-fundamental software development process activity.


Here at Visiphor the Test Group is reinventing itself into a QA group by pushing the following list of actions that are easy to implement. Incremental change is powerful if steady. You do not need to apply all the items, you can pick and choose or add your own. However consistency is important. If you pick one of the items, you need to keep it during all the development cycle: it doesn't make sense to peer review version 1, 2 and 4 of a document but not versions 3, 5 and 6 of the same document!


Here is the list. Comments, suggestions and real life stories gladly accepted. I would love to read your comments and what has worked (or not!) for you.

ActionDescriptionWho is responsible?Phase
Peer review of documents (architects)Each document is reviewed at least by another member of the Architects team Architects ArchitectsPlanning
Peer review of documents (developers) Each document is reviewed at least by another member of the developers teamDevelopmentPlanning, Development
Peer review of documents (tester)
Each document is reviewed at least by another member of the testing team
Testing Planning, development, stabilization
Peer review of documents (systems)Each document is reviewed at least by another member of the systems engineer team Sys. Eng. Planning, development, stabilization, deployment
Solution Setup (developers)
A senior developer creates the Visual Studio solution(s), project structure, and namespace framework.
DevelopmentPlanning or Development
Source Control Repository Setup A senior developer adds the solution(s) to the source control repository. Development Development
Code Reviews (developers) Similar to the peer reviews of documents, the code for the application is regularly reviewed by another member of the team or by an automatic tool. Development Development
Kick off meeting. Also known as Launch Meeting.Everybody involved in the project needs to be present. The purpose of this meeting is to fulfill several of the MSF principles of common goals, vision, foster communication…. PM Planning
Setup and use of Issue tracking system
An Issue Tracking System needs to be set up for every project, in addition, it needs to be the central collaboration tool that each team member uses to identify, record, triage and solve bugs and change requests. Testing Planning
Sign on at each iteration/Marking Iteration boundaries The beginning of each new iteration (as defined in the MSF process) should be marked by a team meeting with the same purpose as the Launch meeting. Iteration ends should at least be marked by an email from the PM, and in some cases by a post-mortem-like team meeting. The goal is to keep everybody informed, and with the same objectives. PM Through the lifecycle of the process
Dedicated Test environment The Test environment should be used exclusively by testers and not shared with development. PM/Sys Eng/Testing Development, stabilization
Unit test Developers, should unit test each piece of their code either manually or using automated tools Development Development, stabilization
Define the Scope for every project. It is important to define what is in the project and what is out. Gray areas will always exist but need to be minimized.PM, Development, TestingEnvisioning
Customer involvement in the testing effort of User Interfaces The rationale of this point is based in the premise the customer is the best to judge the user interface: both the information contained, the user-friendliness, the work flow and the business logic behind it . In this context, if the customer is involved in the testing, the quality of the end product will be better. PM Stabilization, deployment
Project will proceed in a continuous stretch with not time gaps in between. When a project stops, and then restarts, a lot of the knowledge accumulated is lost. Projects done as a continuum will have better chances to reflect better the customer original intentions.
PM Through the lifecycle of the process
Post-mortem meetings a the end of each iteration In a multi-phase or multi-iteration project, this practice should help to improve process and correct problems that have appear in previous phases PMEnd of the project
Setup and use of Change Request Procedure The team, PM and customer should set up a procedure to deal with new/changed requirements. I should include a risk assessment for the requests. Testing, everybody Planning
Setup and use of Requirements System A centralized requirements repository will reduce the dispersion of information and ensure all the member of the team share a common list of requirements.
It may allow the possibility to generate metrics and be able to show to the customer all the requirements changes.
BA, Testing, PM Planning
Data dictionaryA spreadsheet or document that describes the name, meaning, type and other information about the fields of a database and how they transform
(if applicable, as not all the projects may recommend a data dictionary)
BA, Development, testing Planning, Development
Monitoring of the systems for unauthorized changes This monitoring should be associated with clearly defined consequences for intentional unauthorized changes Everybody All phases

Soon I will post a second list of recommendations that are a little bit harder to implement or less applicable to all projects.

Note: The "Phase" is in the context of the Microsoft Solutions Framework (MSF), and are orientative. Some of the tasks do not have a clearly defined phase in which they need to be done.

11 November 2006

I have read the interesting article about Michael Porter's ideas titled 'Why Do Good Managers Set Bad Strategies' (
http://knowledge.wharton.upenn.edu/article.cfm?articleid=1594
). The answers are interesting, although I have some concerns about the premise: Excellent managers can be very bad strategist and vice-versa (In WWI Germany was very well managed, but its long-term strategy proved disastrous, while the new and chaotic Soviet Union became a World power).

However, it is his statement of "Strategy has to do with what will make you unique" what make me scratch my head. In the current context of the software development industry, specifically in the context of a small professional services firm like the one I am working on, what we can bring that is unique?

Perhaps it is a self-interested answer. However, it is clear for me that currently that major differentiator among our competitors is Quality, used in the broadest possible terms. Quality understood as the set of processes that will be able to deliver in time in budget a solution a business need in accordance to the customers' expectations. Notice this would include managing the customer's expectations!

After more than ten years in the software industry, still surprises me that so few are trying to exploit this niche as differentiator. It still surprises me that many managers, visionaries, captains of industry and so on see QA as a luxury or fat to trim. In most of the cases, it is a simple marketing investment.

In an ad from Mercury, they say, "80% of applications are deployed untested. 100% of customers really hate that.” So, if this is the state of the industry, let me return to my suggestion and to the point of this lucubration: what about developing a strategy that will focus in the QA of your product as a way to make it unique?

The last but not the least. The article also includes the quote "If you don't pursue a direction for two or three years, it's meaningless." Putting the quote in the QA context: you should not waste energy and efforts implementing a lukewarm QA strategy. Do not set a policy and then constantly break it. For QA professionals it is frustrating and you will not achieve any good.