Monday, May 2, 2011

What Looks Like Is Wrong May Not Be What's Wrong: Testing Lessons From House, M.D.

So a few weeks ago, I was trying to figure something out.  I was getting weird error messages and strange results and generally odd behaviors when trying to run a test.  Now, the odd part was, the behaviors shown and the error messages and the data stored in the DB simply did not, well, match. 

There was no way those pieces could all go together.  What could be going on?  What could be causing these really unusual symptoms? 

Now, I don't know what anyone else does in these situations.  I know that the method I use for tracking down problems like this can best be described as "Housian" - Well, maybe "House-ian."

Yes, I turn to that ever so patient and thoughtful, gentle tempered example for all testers, the title character of "House M.D." Gregory House.  The cuddly, lovable, always cheerful soul dispensing folksy wisdom in an gentle, kind way.

OK - If you ever watched the show, you know I'm, well, fibbing a bit.  He isn't most of those things, usually. 

What he is, is methodical.  He may not be the gentlest soul, but there is something about him.  You see, when problems simply can't co-exist - I think of it as two objects not being able to share the same space.  I sort of learned that once upon a time.  The core question is what are you going to do about it?  How are you going to find the real problem that is triggering the first in the observable problems? 

House uses a team of people to bounce competing ideas against.  That can work in some circumstances, like when you have a short time to figure out what is wrong with a critically ill patient on television, and a commercial break is coming up.  What I do like is to keep ideas flowing.

Sometimes I find myself looking at a symptom or two and trying to find a relationship.  Is one causing the other?  Can the "real" problem be something less than obvious?  Maybe another set of problems is masking the real issue?  Maybe the real issue has nothing at all to do with what I am seeing? 

Yeah, I know.  It kinda depends on what the symptoms are, right? 

So, with this particular set of problems, I found myself thinking, "What would House do?"  Other than belittling all the ideas from the Greek Chorus of supporting doctors, a common tactic is to focus on something that I can control - one symptom or set of symptoms and carry on with that.  Maybe I'll find something related to what the core problem is. 

So, I tired that.  Nope.  Nothing appeared to make sense.  Back to square one.

Then something dawned on me.  Maybe the clue to the real cause was being masked in the flood of stuff in the logs.  Maybe I was ignoring the one piece of information I needed.

So I took a look at the logs again.  (Folks who watch the show will see this as the "Hey, look at this little spot in this easily ignored place on the patient's body" moment.)

Bingo - there it was.  The one little error message in the midst of spectacular dumps of... stuff.  There was the clue I was ignoring all along. 

Thanks, Doctor House.  We were able to save the patient.  Umm, project. 

2 comments:

  1. Nice!

    I've been wondering when "House" would be used as a comparison - there are a lot of similarities to the complexity testers face (lots of information from many angles)- and how to work out what's relevant and of what's relevant, what's significant.

    Then it's down to hypothesis and testing (after forming various hypotheses) - the differential diagnosis is something most good testers do without thinking about the name/label.

    I was reading "How Doctors Think" - which is an excellent illustration of cognitive errors in medical practice and thinking that it was just like "House" (in terms of illustrating errors in thinking and using the wrong information) - and that, in turn, has many similarities to the issues and problems testers face.

    In the book there's a quote about a doctor who says that the MRI is a wonderful tool but also a terrible burden - as it gives so much data and information. If you look closely enough you'll find problems with everyone. Flood of data - yes, I recognise that! (What's important to the customer/patient/stakeholder right now type of question.)

    So, following the "House" analogy, I assume you wouldn't advocate breakin-and-entry to discover the secrets of your stakeholder/developers? :)

    Good post!

    ReplyDelete
  2. Thanks Simon! Great observations. As far as breaking into the stakeholders/developers "domiciles" I would never suggest running a code sniffer looking for words like "buggy" "kludge" "doesn't work right" or "problem." That might be inappropriate. Then what would Cuddy say? ;)

    ReplyDelete