A few weeks ago, while looking for some other information, I stumbled across the power point slides of a conference session on Test Process Improvement that I decided was "not a good fit" for me. Yeah, I walked out... about 10 minutes into it.
The premise was "If you don't have a Process, you need one. If you don't have a Process, you have Chaos and Chaos is bad." Following the obligatory introduction, and some seven minutes of what appeared to be gratuitous assertions, I said, "Enough" and walked out.
Having a Process is not a silver bullet. Simply having a Process will not magically fix your Chaotic environment. If you are trying to impose Process on the organization wearing your "Tester" white hat or the plate mail of the Quality Paladin, good luck. Most places where I've seen Chaos rule, its because someone with a lot of scrambled eggs on their hat likes it that way. (I wonder how many metaphors I can pull into one paragraph? Better quit there.)
However, if you have a Process and no one follows it, the question should be why not? My previous blog posts (Part II and Part I of this thread) talked about how the "problem" might not be the real problem and how you need to seriously look at what you are doing before you can fix what might need fixing.
When you look long and hard and honestly at what you and your group is doing, when you find the places where what is done varies from what The Process says, you must determine why this difference exists.
I suspect that it will boil down to a matter of relevance. The official Process has no relevance to the reality of what actually is needed in those situations. If it is a one-off, then there may be something that can be tweaked. If it is a regular occurrence, then the value of The Process comes into question. If it doesn't work, why pretend it does? Why bother having it at all?
Granted, The Process may have been relevant at one time and things may have changed since it was introduced. However, nothing is permanent. Change is inevitable. Even The Process may need to be updated from time to time.
When you do, look to the Purpose your team is to fulfill. Why do you exist? What is your Charter? What is your Mission? Do you have a Mission? I'll bet you do, even if you don't know what it is.
To start, look to what Management expects. If a boss-type is telling you that the Test Process needs improvement, try talking with them. Discuss with them what they believe needs to be improved or where the gaps are. This may become the basis of the group's Charter.
The Quest that you are expected to follow.
What are they seeing as "broken" that needs to be fixed?
If the gist is "there are too many defects being found by customers" ask if there are specific examples. Anecdotal evidence can paint a compelling story, yet without specific examples, you may never be able to find hard facts Is this a hunch or are there specific examples? Are these defects, as in they should have been found in testing?
Maybe these are aspects of the application that the customers expected to behave differently than they received? If they are, why is that? How can that be? How can the expectations be so different than what you believed it would be? After all! The Design and Requirements, that you based the tests on, matched perfectly!
Let us ask Dulcinea how these things can be so different than what they appear to be?