Tuesday, May 17, 2016

On Quality Engineering and Testing and Defect Prevention

Some time ago, I wrote a response to a post I read extolling the virtues of "Quality Engineering" over mere testing. (You can my response here.) Since then I have received some emails and been in some conversations on the topic. I've also seen a variety of threads on twitter related to the discussions I've had with others.

This, then, is some more of my thinking around the topic, based on what people have said to me - mostly trying to convince me of the error of my thinking.

It was explained to me that Quality Engineering is, at it's heart, prevention of bugs and problems in software. Thus, a Quality Engineer is not looking for bugs, instead, a Quality Engineer focuses on bug prevention - keeping them from being created in the first place.

"A good QE works to avoid bugs in software."

That was precisely what I was told by a very nice young lady. It struck me a bit like "A good AO (Automobile Operator) works to avoid potholes in the road." Apparently I was not amusing to her (maybe she lived in a city, as I do, where there are myriad potholes on nearly every road.)

There were several examples presented.

One involved a QE finding a problem in a planned change to a DB table. The QE prevented a problem by identifying the flaw in the development group's intended change. Their work flow consists of proposing DB changes, reviewing it with the development team, then the full Scrum team, then presenting it to the DBAs for review. It was in the Scrum team review that the QE identified the problem.

Another involved a QE identifying a problem in the design of some changes to an application. Again, the QE spoke up and raised an issue during review of the design with the Scrum team.

The third example was a QE speaking out over requirements that seemed contradictory. The reason was simple. They had not been understood and were noted down incorrectly.

Each of these were presented to me as examples of what a good Quality Engineer does. They prevented bugs from being created.

Except...

My response was, in each of these cases, the QE found a problem or inconsistency and raised the issue. They did not so much prevent a bug, as they did find a problem (bug) in a spot other than the working code. They found the problem earlier in the course of software development.

This, to me, is part of the role of testing and why testers need to be involved in the early discussions.

Taking the next logical step, including a tester who is familiar with the application in the initial discussions could benefit the entire process by helping other participants think critically about what the story/change/new feature is about.

By engaging in these discussions and exploring the intent and nuances around the request, recorded notes and conversations on the work, a tester might be able head-off issues while they are in the "bounce ideas around" mode - while discussions are happening around what terms or concepts mean.

In an Agile team (whatever flavour your group uses) if people are engaged in working toward better quality software, the role of a critical thinker is necessary - whatever you call it.

Some folks tend to get rather, emmm, pedantic over how words get used. Here's what I mean...

Each person in a team is trained to do something. Usually, they are better at that than the other activities needed to be done. Ideally, each person can contribute to each task that needs to be done - but their expertise in certain areas is needed to support and lead the team when it comes to doing those tasks and activities they are trained in particularly.

Some people are trained, and very good, at eliciting and discovering requirements. Some are trained in building a usable design. Some are trained in developing production code. Some are trained in database design. Some people are trained in assembling components together into a working, functioning build and/or release.

Testers have a role in each of these tasks.

Testers can help requirements be defined better.
Testers can help the design be better.
Testers can help the person writing production code write better code and execute unit tests better.
Testers can help with DB work (this may shock some people.)
Testers can help verify and validate the builds are as good as they can be.

Testers can test each of these things. It is what we do.

Getting to a position where testers are trusted, welcome and encouraged to participate fully in each of these tasks takes time, effort and gaining the trust of others on the team.

People tell me that testers only test code.

Those people have no idea what testing can be in their organization.

What some people are calling Quality Engineering tasks, from what I have been told (very patiently in some cases) are testing functions.

Think.

Test.

7 comments:

  1. Excellent post, and I realize an error in my own thinking... I have been saying that testers prevent defects ... when I should have been saying that testers (by being good question askers) help prevent defects in the software.

    I have been saying for a long time that testers test more than software. They test ideas, assumptions... which is what I call testing early. I've also been trying to promote that testers are product testers these days, not software testers, but not having much luck with that .... yet.

    ReplyDelete
    Replies
    1. Yes Janet - "Testing early" identifies defects in places before they get to the actual software. The "question asking" is the key to this.

      Thanks for the kind words...

      Delete
    2. Janet - if all we talk about is about testing SOFTWARE projects, what about all the business projects that are about much more than software?

      Delete
  2. I did not see a link to the original Quality Engineering article, so I am not sure if this is discussed or not.

    I do believe there is a place, beyond what is _traditionally_ called testing, where an engineer (read tester) of whatever name you might choose, helps the team find potential quality issues of a larger nature. Are developers following good practices for writing and maintaining their code? Are check-in and release practices allowing users and the team to know what changes are occurring in the software? Who is evaluating risk from the perspective of the team, users, support? Is the team lead effectively communicating tasks and their priority to the team?

    This is larger picture stuff that someone down in the trenches of daily development will not see on their own. Someone can be watching these things and letting the team know when it is, or may be, impacting the quality of the software without this kind of analysis impacting the daily tasks of the team.

    That being said, this is all still testing. It is just the kind of testing that we do not think about in the trenches. I agree wholeheartedly with the statements made in this and the previous post.

    ReplyDelete
    Replies
    1. Fair enough. In general, I'd agree. In the original article, much of what you mention is under the realm of Release Management and Configuration Management. The idea of "testing" practices in those contexts seems to be limited to software itself and not to anything else. (If I can find a version that does not require a paywall, I'll post the link.)

      Of greater concern to me is the idea of "preventing defects" vs "finding bugs." They are ALL bugs - some are in code, others are in documents, some (many), as you point out, are in practices.

      The best we can do is raise the flag and try and get someone's attention to fix it.

      Delete
    2. As well as test people move towards shift-left (code), shift right (test in the wild), project management, scrum masters - there is also contexts where the tasks of the "test person" is trending towards configuration and release control. #beenThereDoneThat

      Delete
  3. Hi Pete,

    Very nice read!
    I tend to agree with your views and wording.
    I often use these words in the same manner. However, lately I've started to struggle with how we seem to abuse 'Testing'. We give it such a broad and all-encompassing meaning.

    To me, it feels artificial.
    Testing in a narrow sense is trying to validate and falsify hypotheses. It is about observing something of note or having an experience, construction a theory about that observation or experience and then experimenting to find out if the theory stands. It also holds a purpose to uncover important information.

    To me, this is testing at its core. (of course there's much more to it.)

    Testing of conversations and ideas or brainstorm sessions might fall into the same process, but that really doesn't fit for me.

    Being critical during planning and while working out concepts is what I call being an involved and responsible team member. Everyone on the team can and should participate.
    Testers are generally better at being critical though, so that might be an explanation of why we feel that's our role.

    ... and we push ourselves into that role?

    I've been writing about my own experience with naming and activities and would like to know your insights on it if you'd be interested.
    https://wordpress.com/stats/post/382/enquiretst.wordpress.com
    It doesn't cover prevention or detection of early bugs, but it can definitely fit in there.

    Kind regards,
    Beren


    ReplyDelete