On Metrics & Myths or Your Facts are From the Land of Make Believe

Its been a quiet week in Lake... Oh wait.  I'm not a famous radio personality with a show centered on a town that does not exist.  I'm a tester.

Sometimes though, I feel less like someone from Lake Wobegon, MN and closer to someone from Brigadoon.  Both are fictional, mythical if you will, and both have certain charms and appeal about them.  Except for one minor point.  Out of context, they make very little sense. 

So, the last several weeks I have been working away on studying metrics and concepts around them and things of that ilk.  The cause of that was the combination of "training" required by the day-job, and getting the new set of metrics for the "Scorecard" - yup - Metrics applied to the individual, team, group and department.  Oh my.

So, I went digging though my notes and found a variety of ideas, some good and some less than good, from a variety of sources, some reliable and some less than reliable.  Some of these we just plain contradictory.  Some had ideas that, in and of themselves seemed reasonable, until you considered the assumptions and presumptions that must be made and taken in for the numbers to actually make sense.

I found myself rereading articles by Cem Kaner, Doug Hoffman and others cautioning against misusing metrics.  I likewise found learned discussions around how metrics can be relied on if you take emotion out of the equation and look just at the hard, empirical data.

Then I saw a tweet from Michael Bolton, recommending the writings of  Laurent Bossavit as being worthy of  consideration.  So, I followed the link and began reading.  What I found was a fellow who had written an e-book that seems interesting.   Don't take my word for it.  His Twitter handle is @Morendil.  Search for him and begin reading.  Or, check out his e-book - Cool title - The Leprechauns of Software Engineering.  Find it here:  http://leanpub.com/leprechauns  You may not agree with everything, but much is worth your consideration.

Where was I?  Oh yeah.  Metrics.

Matt Heusser and I had an interesting chat last month while on a flight to New York.  He asked me my view on metrics.  I responded that my general view was that most people misuse the term and the concept. 

I believe that metrics should serve to address questions we are seeking enlightenment on (kind of like testing, no?)  A painfully large number of companies focus on stuff that is easy to count, without looking to see what that information might tell them - beyond the obvious.

I believe that most people trying to address questions with these "metrics" really don't have a good idea what the questions they want to ask are - and so they settle for what they can get easily.  Things like bug counts, test cases, test cases executed per day, failure rates and things of that ilk.  Instead of looking for things to help constructively help their staff, their people, do their work better, it is easier to look for control metrics.

They'll misquote Drucker or Lord Kelvin or - heck - maybe they've just heard so many truisms (that aren't really true) that are misquotes that they accept them at face value - an awful lot of us do.  They'll look to change behaviors by making a big deal about metrics and ... well, stuff.  What they get may not be what they intended to get.

Be careful in dealing with metrics - not all is what they appear.

Be careful when playing with dragons for you are crunchy and good with ketchup.

Where Has Pete Been, or Time Flies When You're... really busy

I can not believe so much time has passed since my last blog post. 

OVER A MONTH! 

I have several in the works and will be posting one shortly after this - I promise. 

Since the last update, I've been crazy busy at the day-job.  I spent several days in New York with Matt Heusser hanging with some really smart people and talking about testing.  I made some new friends and chatted with people I knew by reputation and cyberly - but met in person for the first time.

One night I participated in a panel discussion hosted by the Financial Services Special Interest Group of AST - the Association for Software Testing.  What an interesting experience - lots of different view points and ideas around some seemingly simple questions. 

We had a great time.

I've also had some interesting conversations when we got back - lots of interesting things happening in testing all around the world, it seems.

I had, at one point, signed up as a "contingency instructor" for the AST's BBST Foundations Course - translated, there were folks signed up, a head instructor and two assistants, and I agreed to help out if one of them could not do the class.  As it is, I got an email this last week asking if I was willing to help out.  So, I'll be doing some online instruction. 

There have been a stack of problems facing day job stuff - projects that simply were not working right - lots of work there.  Did I mention that before?

Oh yeah, I had "unexpected oral surgery" shortly after getting back from New York and I'm like WAY behind in my writing.  Sorry folks - I'll get there - I promise.

I also have been trying to catch up on the music writing I've been trying to get done.

And of course, being a husband, dad and - Oh yeah, my neice got married as well. 

So, sorry for the backlog - I'll get caught up soon. 

I promise.