Friday, April 18, 2014

On Testing and Wars of Religion

I'm sitting in my rocker with a nice glass of wine next to me.  Today is Friday, the 18th of April, 2014.  In Western Christian Religious calendars today is Good Friday.  It is also the anniversary of the Battles of Lexington and Concord which Americans may remember as the beginning of the American War of Independence.  Paul Revere and several others had made their midnight ride the night before.  Alas, that is another story.

The determination of how the date for Easter was to be calculated was one of the early points of contention in the Western Christian tradition.  This is an interesting point. Keep that in mind.

The granddaughter in high school had school today.  The interesting thing is that she goes to a "Christian" school.  My lady-wife found it interesting that she had school on a day of such significance in the Christian religious calendar.  I smiled and gently said "It is because of the type of Christian the school is intended to educate."

We talked on this a bit.  Simply put, I had many conversations with people of this particular sect of Christianity.  When I was young, they were the majority if kids in the neighborhood.  There was one family who were Greek Orthodox, one family around the corner who were Jewish my family and then several families of this sect.  I smiled because I remember so many times being told "You're nice and your family is nice but you're still going to Hell when you die."

OK, so consider being 11 or so and being told you would go to Hell when you died because of the way you and your family go to church.

I remember asking my mother on what was going on.  Her response was something to the effect of "They can't imagine being wrong in anything, and since we go to a different church, we must be the ones who are wrong and that means we are going to Hell."

These conversations, so many years apart, have left me thinking this evening and finding the similarities with conversations I have had with certain testers to be notable and quite disheartening.

Testing Must...

Simply put, I have been given a list of items which "Testers Must..." do if they are "really" testers.  These seem to fall under one of several forms of fallacy.  This is generally expressed as "No real <blah> would ever <do thing>."

One item commonly mentioned - Testers Must Verify Requirements.

Really?  Must?  In every circumstance?  If Tester 1 verifies requirements and does that in a day or so, what am I to do? 

I understand that when the contract says you must "provide traceability between tests and requirements" that you need to be able to do this.  Is there one and only one way to document tests and show traceability?

If there is one and only one way to document tests does this imply that any other way to document tests is wrong?

If it is wrong is it bad?  If tests are wrong as they are documented, how can we execute them and be certain that we are doing things right?

What if we are not wrong?

Does this mean we are right?

When we are challenged in our beliefs about testing, do we respond as 11 year old children or do we respond as thinking, mature adults?

What must testing do?  Are we certain?  Do we agree on this?

Based on conversations I recently had and articles I recently read, I am certain we do not agree.  When people condemn others for not agreeing with them, I get a little sad.  When I am condemned for not agreeing with them, I ask "What is it that makes you certain you are correct?"

I find that question to be challenging for people to answer.

If people can not logically explain why they believe things they believe about testing and they can not logically discuss the implications, the result sounds much like the wars of religion from 400 years ago.

Of course, in smaller ways, those wars continued through my youth.  In some places they continue.  Likewise, the wars, and condemnation about doing testing "differently" also continue.


  1. Man, I really need to start my own blog. Why? Because each of you excellent bloggers keeps writing about topics that I had intended to write about! This was a nice, thoughtful post, Pete.

    I recently wrote to someone about my own thoughts on the similarities between challenging ones religious or professional beliefs.

    There are many beliefs. There are political beliefs. There are religious beliefs. There are professional beliefs. Taken to a silly extreme, there might even be beliefs about the peanut butter.

    JIF believers: Ours is the one, true Nut.
    Skippy believers: Everyone else is going to shell!
    Peter Pan believers: Our nut lives in the sky (second star to the right, and straight on till morning).
    Homemade believers: We are one with mother-nut.
    Peanut-allergy sufferers: There is no nut.
    Everyone above: Nutella, take that stuff back to Hollywood!

    Regarding my own religious beliefs…to be a bit coy, “I used to believe one thing, and now I believe another”.

    Regarding my own professional beliefs…to be a bit coy, “I used to believe one thing, and now I believe another”.

    In both cases, I had a “crisis of faith” and was moved to challenge my beliefs.

    Challenging ones beliefs is a very tough process. The journey is filled with doubt, fear, intrigue, trepidation, wonder, joy. The end result may be that you retain your original beliefs, that you alter your existing beliefs, or that you adopt totally new beliefs. If the journey is completed, then each result is valid. But, I think that many do not complete the journey because it is so difficult. Some of the potential results mean that you might be wrong about other beliefs, too. This is too earth shattering for some, who opt to abandon the journey and stick with their original beliefs – not out of enlightenment, but out of blindness.

    So, I now have more (but not unlimited) patience for those that might have different professional views about quality and testing. I understand that they might be in the middle of their journey. They may be actively challenging their professional beliefs and struggling with the fear and doubt and implications that they might be wrong.

    What makes some challenge their beliefs and/or finish the journey? I think it boils down to will and curiosity. But that is a long post for another time.

    But, I think we can all agree: Creamy over Chunky.

    1. This comment has been removed by the author.

    2. Hello!

      Thanks for the (remarkably long) comment!

      Two questions for you!

      First - On beliefs, you said you "used to believe one thing, and now ... believe another." Are these things completely different or are they extensions and growth from the earlier positions? To paraphrase, "When I was a child I had the beliefs of a child. Now I am a man (adult) and I put childish things behind me and have the beliefs of an adult."

      Second - See how easy that was? What is stopping you from developing and posting ideas in a blog? Do you want to change the world? Your corner of it? Maybe just yourself? What is stopping you?

      All the best!


  2. [Are these things completely different or are they extensions and growth from the earlier positions?]
    Completely different.

    For example, I used to believe that you could not have a defect without an explicit, documented requirement. I'd say things like, "How can you know if something's wrong, if you don't know what's right?" and "We are testers, not designers." That belief sees testers as idiots - robots that cannot think for themselves.

    Now, I believe something completely different.

    [What is stopping you from developing and posting ideas in a blog?]
    The more I think about this, the bigger my answer gets. The short answer is: I'm having trouble getting over the idea that I'd be "just another voice in the crowd". Anyway, I'll reply to you privately about this specifically to avoid distracting from this worthy blog post topic. You can post none, some, all of our future correspondence here, if you think it might be helpful to others.


  3. Too Bad, Damian. You're prompting another blog post. Short answer: Never Fear being another voice in the crowd. The more people thinking and sharing ideas on testing, the better.

    Old drumming axiom applies here: There's always room for one more!

  4. hehehe....this is so true. I came to this understanding a few years ago in an online discussion with someone who was an X-type of Test Methodology user. He would not change or adapt, even when challenged with a weakness in his own methodology. I began to think this was like a religious belief, where you have a specific kind of faith in what you do, and how you do it, so even when you are confronted with a fault in the methodology you find any means to cling to it. I agree with you that we really need to keep an open mind on methodologies, I used to cling to Waterfall because it's what I learned when I started, but then realized there were other methods I could interpose and now I have all kinds I can use as the situation permits. We need to be adaptable otherwise we stagnate and die. Thanks for this, I've had this in my head for awhile but you put it better than I could, and you usually do!

  5. Hi Pete, can you clarify your penultimate paragraph? What are you implying is the result, and how does it relate to the Catholic / Protestant wars of the 1600s?

    I do think that one has to be careful in two ways. First is in assuming that the lack of a logical foundation for a belief makes it less real or valid to the person holding that belief. If a client wants a UX that goes against common notions of good design, we may feel obligated to point out problems, but I don't think we should consider ourselves responsible for changing his mind.

    The other thing to be cautious about is equating one's ability to articulate reasons for holding a belief with the truth or value of that belief. I'm not a theoretical physicist, so I need to rely on my experience of gravity rather than a mathematical model, but my ignorance of the law does not provide me an excuse to ignore it.

    1. This one, yes?

      > If people can not logically explain why they believe things they believe about testing and they can not logically discuss the implications, the result sounds much like the wars of religion from 400 years ago.

      Should we not expect people to be able to express themselves logically about their profession? About their craft? I do. If people are willing to discuss the question, and develop, then so much to the good. If they cannot or will not, then there are other possibilities.

      Why do they do the things they are doing? Because someone told them to and no more? Or because they have found value in it?

      For your suggestion, if a Customer wants a UX that does something unusual, why not ask about their choices - if only to understand what they want and and why they want it? Are they asking for this because they have not thought about alternatives?

      Or are they doing so because of their sovereign lord's desire they do so?