On Software Quality and Software Testing

The last week or so I have been deep, very deep, into considering the relationship between Quality of Software and Software Testing.  In this, the conversation has been more at the Meta level, something akin to ASQ's view on quality in general.  (Fair warning disclaimer, along with being a software tester I am also a member of ASQ - American Society for Quality - these folks.)

Interestingly, that relationship helps me when I challenge assertions, usually gratuitous, often fundamentally flawed, on something published by the ASQ or something Deming said or wrote.  Its interesting sometimes to lean into the table and say "Can you explain what that means?  I'm not making the connection between what you are asserting here and my understanding of what {insert quality buzzword} means.  Its possible we have a different understanding of the concept and I'd like to address that to avoid future problems and potential future conflict." 

The response often comes back citing some authority, for example, Six Sigma or some concept championed by ASQ.  Interestingly, that was recently coupled with the ideas of Context Driven Testing and AST - Association for Software Testing (Ummm, for those who don't know me, I'm a member of that, too.)  Oftentimes I will then, when it is clearly an attempt to assert a position by citing authority, say something in as non-threatening a manner as possible along the lines of "I'm a member of ASQ and of AST.  I have read the white papers and books on Six Sigma (or whatever else they are asserting, usually out of the recommended context) and I'm not sure how they align with what your are saying.  I would like to understand what you are saying better.  Can you explain it or would you prefer to have that discussion off-line, maybe over coffee?  I'll buy."

I find people will be much more open to such discussions if I buy the coffee and /or bagel to go with it. 

And yeah, I realize that I am doing my own version of citing authority by making the above statement.  It does serve to get their attention and blow away the smoke screen that is intended to be set up.  Well, maybe not so much removing the smoke screen as is bringing high-powered radar into the mix - I can see their position in spite of the smoke screen.

Where am I going with this? 

Many people I meet use the terms "testing" or "quality assurance" or "QA" interchangeably or in conjunction with each other.  You get statements like "Let me know when this has been QA'd" when they mean "tested."  Then there is "QA Testing."  Do NOT get me started on that.

The idea of "testing improves quality" is often the response to the question "Why do we test?"  The bit that gets left out, possibly because it seems obvious or maybe because people are oblivious to the idea, is that testing improves quality only if someone acts on what is learned from the testing. 

If something is not changed as a result of testing - configuration, code, processes - maybe all three - maybe other stuff as well, then will "quality" be any better?  What is the point of testing?

If people want confirmation that things "work" then by all means - run the happy-path scenarios that possibly were used for unit testing, or build confirmation testing, or maybe in the CI tools - but don't confuse this with "testing."

The point of testing is to learn something about the system or piece of software at hand.  It is usually not to prove anything.  It is rarely done to prove something is "right."  It may be done to check certain behaviors - or to see if specific scenarios behave well enough for a demonstration - or even, in a limited sense, validate some piece of functionality. 

However - if there are any variances found, the testing identifies those variances - nothing more.

Testing does not make anything better.  Testing does not improve quality - ever.

Testing provides information for someone to decide that action needs to be taken. and then someone must act on that decision.  Then the quality may improve. 

It is taking action after testing is completed that improves quality - not testing.