On Missing the Mark or What Bagpipe Bands Taught Me About Software Testing

For the third year in a row, I am not where I spent some 30 Memorial Day Weekends.  The Alma Highland Festival is a two day affair on the campus of Alma College, in, Alma, Michigan.  At one point in my life, I took the Friday before off from work, loaded my kilt, drum(s), coolers of adult beverages, anda small cooler of fruit and sandwich makings into the car and drove the 90 minutes or so that it took to get there.  I'd then camp out in the parking lot until we could get into the digs that would be ours until Sunday evening.

There is a two-day pipe band contest, one Saturday and one Sunday.  At one point they had enough bands to stretch from end-zone to end-zone on the football field, with bands lined up every 5 yards or so.  This year they have some 25 or 30, divided into 5 "grades" or levels of experience and expertise.  . 

I got an excited text message yesterday from a band I have been helping out by teaching their drummers once a month or so this last year.  They had played really well and were looking forward to hearing the results at the end of the day.  There were a total of 10 bands in the grade they were competing in and they hoped for a good placement, at least in drumming, if not overall.

A few hours later, I got a very sad text message.  "What do I know?" wrote the sender.  "We ended 9th drums. I thought we did better."

I have a response that has become almost "canned" I have used it so often with so many beginning bands.  It goes something like this:

I would not be surprised that you played well.  You have been working really hard and the improvement shows.  What we don't know is how hard the other bands have been working.  Since it is hard to listen objectively while playing yourself, then comparing yourself to every other band, how do you know you did not do the absolute best you could?  Even if you did, how do you know that the other bands did not do the same?  What if their "best" was simply better than your best for the day?  If you were pleased with how you played, accept that as part of the reward for the hard work.  Recognize that the real point is to improve your level of play and be able to know you gave nothing away for the other bands to capitalize on, and beat you.  If they outdrummed you today, congratulate them, have and {adult beverage} with them and a laugh or two, then work all the harder to get ready for the next contest.

It is a model I've used for years, with every level of band I've played with or worked with from the absolute beginners to Grade 2 - one step away from the god-like heights of Grade 1, the top of the field.  Sometimes, it is hard, other times, it makes things a bit easier to take.  A fair number of times, it is also true.

What does this have to do with software testing?

It is reasonably related - No matter how hard you try and no matter how carefully you work, you will not find every defect in the system.  Full Stop.

No software tester or test team can find every defect.  That is a simple fact.  Some folks feel devastated when a defect "gets away" and is found by the customer or users.  What information did you miss that lead to you not exercising the exact scenario?  Was there any reason to suspect you should exercise that exact sceanrio?  If the choice was to exercise that scenario and not others, what would be the impact of doing so?  What bugs might have been released instead of the one that was?  How can you know?

Contrary to those who cite "defect free" as the target of good testing, you can not possibly exercise every scenario, every combination of environment and every combination of variables to cover everything.

Learn from the defects that get through, examine your presumptions then see if, given what you know now, and the results of the decisions made, would you have made the same decisions you did about testing?  Can you apply these lessons in future projects?  If so, have a nice cold {adult beverage} and move on.

When the results are less than optimal in pipe bands or testing, if you learned something apply that and move on.  Berating yourself or your fellows does no good.

CAST 2012 Speaking and Emerging Topics

Remarkable as it may seem, we are quickly drawing to the deadline to submit a proposal for the Emerging Topics track at CAST 2012. 

CAST 2012, to be held July 16-18. CAST 2012 will be held in San Jose, California at the Holiday Inn San Jose – Airport (formerly Wyndham Hotel).  

The Call For Participation is up (here). There are three basic types of presentations:

Interactive Workshops (140 minutes);
Regular Track Sessions (70 minutes - at least 25 minutes for discussion);
Emerging Topics (20 minutes - at least 5 minutes for discussion);

The deadline for Regular Tracks and Workshops is past.  The Emerging Topics deadline is quickly approaching.  We are beginning to review the Emerging Topics submissions that have already been received.  If you intend to be at CAST this July, and are interested in trying your hand at presenting on something you are passionate about, involves testing and THINKING - SUBMIT A PROPOSAL!

Emerging Topics track are great - Short, 20 minute snippets - enough for you to present the core of an idea and answer questions. That's about it.  You can submit through the same process the Regular Track and the Workshop sessions are submitted.   OFFICIALLY - the deadlines  for THOSE submissions is June 18.  We are starting the process of reviewing - and contacting people who submitted.  If you are thinking about it - SUBMIT THIS WEEK!  Better yet, SUBMIT TODAY!

The information you need to know about submitting proposals is on the website at the link above. If you are a Thinking Tester, I encourage you to consider attending CAST. If you are interested in telling people about your ideas, I encourage you to consider submitting a proposal.

All the cool kids will be there.  Why not you?

On Testing, Crossing the Rubicon or Nailing the Theses to the Door

The last ten days or so, I've had a series of conversations that have left me baffled.  Do people really listen to the words they speak or write?  Do people understand what words mean?

Whilst being "gently informed" in a variety of ways that I "just don't get" testing.  Apparently I'm a heretic as far as at least these four people are concerned.

The details of the words varied, not by much, but they varied.  The expressions used were actually quite similar.  This surprised me because the environments they each work in are totally different.  Not just different companies, different development environments, but different intended development methodologies.

In short, they said "Our goal is to improve the quality of the software by doing good testing.  Testing needs to verify that everything is right."

I responded that I viewed testing as much more than that.  Testing is neither confirmatory nor destructive.  Instead, testing is an interrogative activity that allows us to discover the true behavior of the software and how it reacts or responds to various stimuli - data, keystrokes, sometimes, simply time.

Rather than breaking software or confirming the requirements have been fulfilled, or showing what doesn't work or... whatever, we can focus on learning how the software really behaves.

No, I was told by five different people, I was wrong.  Plain and simple.  Wrong.

That is NOT testing, they said.  Testing is proving the software conforms to the requirements.

So, all the requirements I asked?  Yes.  All the requirements.

Do we know the requirements?  Of course we do - they've been fully documented in the requirements definition phase (Even the guy that said they were Agile said that. I guess we disagree on more than one thing.)

Now, one of the benefits of the education I had lo those many years ago, is one retains certain things.  One of the things that is in my mind is Martin Luther's statement at the Diet of Worms - the one where he defended himself at an ecclesiastical court.  Some sources, now considered remarkably questionable, simply because the words don't appear in the earliest transcripts, have Luther saying "Here I stand.  I can do no other."

Now, I think it makes sense.  Its a great line.  It is the kind of thing we wish could be dealt with and directed and - well - a defiant refusal to submit; to not go down without a fight.  It is often portrayed as a defense of his heartfelt beliefs that he could not, and would not, gainsay.

Others may point to that as the sin of pride.  One can not admit to that they are wrong. I'm not sure. It could be something of a mix of these things.

In the end, I'm not sure it matters. The simple thing is, you can compromise on some things, maybe a lot of things.  Fundamentals though, tend to be closely held and tend to be inflexible.  At least for people who really believe them.

Are my views on testing firmly held?  Some of them certainly are.  I'm also open to discussion to see if we use different terms to mean the same thing.  I can be reasonable, after all.  Then there is the concept of developing understanding - that one's views can develop and evolve over time.

In the end, a breaking point will be reached.

I don't know the point that will be.  Mostly, it is a different point for each and every person.  What that point is, for anyone reading this, I do not know.  I know my point.

When that point gets reached, each individual must decide for themselves.

When Caesar (Julius - the real Caesar) crossed the Rubicon River in 49 BCE, leading his Legions into Italy proper and initiating the Roman Civil War, he is reputed to have said "Alea iacta est."

The die is cast.

At what point do we say "Enough?"  At what point do we refuse to compromise? 

Here I stand.  I can do no other. 

On Too Much Process or Too Much, Meh, Whatever

Process - noun - 
   1. a series of actions or steps taken in order to achieve
       a particular end
    (Oxford English Dictionary)

So, a fair number of people have heard me discuss excitedly (find the euphemism for rant) on how too much process stuff gets in the way of actually getting things done.  A fair number of times I pretty well am set in the idea that process should not be a controlling thing or a limiting thing, but a guiding thing.

There is a significant difference between the two of them.  Some people don't want to accept that.

Kind of like some people define "Agile Software Development" as 1) no documentation and 2) no testers.

When people allow the aspects of process to overtake the entire point of what is intended - typically the facilitation of what the process is supposed to, well, facilitate, where the process becomes more important than anything else, you don't have one problem, you have a collection of them.

The opposite end, well, makes bedlam look reasonable.

Having no controls and no processes in place can, and I suspect will, lead to its own problems.  Point of codifying processes, the "How we do stuff" is centered around making sure nothing gets missed: no steps get left out, nothing critical that will impact customer experience (like .JAR files not being where they are supposed to be) and other small things like that.

A process can also help us be certain that everyone who needs to know about something actually knows before that thing happens.  People, myself included, often talk about process as something that is ungood.  I've tempered this over time to be more like, process can help us, it can also hinder us - and the specifics of the situation will no doubt have a direct impact on which one it is.

What often gets lost in the shuffle when talking about process, or some development methodology or approach, is that the point of most software processes is to get things done that need to be done, and make sure people know - I kind of said that a bit before - I know.  But if you look carefully at the second portion of that, the bit that process helps "make sure people know" then that sounds alot, to me at least, like communication.

People forget that communication is less about what is said and is more about what is understood - what is heard on the receiving end.

If you have good communication and people succeed in hearing A when A is said, or written in a report or email or... yeah, you get the idea, then process forms a framework to operate within.  When people don't communicate, process may help - but I suspect it just adds to the noise.  More emails and reports to ignore, more meetings to sit in and do something else whilst they are going on, more of the same Dilbert-esque pointy-haired-boss stuff.

Even companies that quite publicly talk about their lack of formal process have processes - they have rules that people work within - frameworks that guide them  and their activities.

I suspect where I draw the line for processes that are useful and those that get in the way is the willingness of the staff - the people who are directly impacted - to follow and adhere to the given processes.

I prefer processes that are organic - that grow and develop based on the experience and relationships among the people doing the work.

I object to processes which are imposed by someone, or a group of someones, who have never actually done the work (except maybe as an exercise in school or a single project in the "real world") but have read, or attended a workshop or talked with someone at "the club"on some best practice that involved some stuff.  Whatever that is.

If people want to have a thoughtful discussion around what can work for their organization, team, company, whatever, I'd be extremely happy to participate.  If you tell me things must be done this way because of some study or best practice or whatever, don't be surprised if I ask what was studied and what practices were compared to determine which one was best. 

On Managers and Mothers and Bush Trimmers

As I'm writing this, it is the evening of Mother's Day in the US.  Because the weather was lovely, much time was spent in the garden this weekend mowing the lawn, trimming bushes, preparing the vegetable garden for the summer. I then took my lady-wife and mother to dinner where we had a lovely meal with nice conversation, a very drinkable bottle of wine and cannoli for the mothers in the party.

It has been a remarkable spring. 

The apple blossoms are done - no surprise there.  This is fairly normal for us.  The pear blossoms are done.  This is unusual.  Lilacs are done - not terribly unusual.  The tree peony are done.  Tulips are done - again not that unusual as they often are.  Its a little odd that the late blooming variety are done as well, but, that is not far from the realm of possibility.  Let's see.  Hmmm.  Oh yeah, iris are in bloom and there are peony ready to bloom.  Those two are unusual for mid-May in Michigan.  July is closer to the expected.

Oh, the stuff that is done?  They all bloomed within a few days of each other.  By mid-April.  That is not only unusual.  It is remarkably unusual.

Right then.  What does this have to do with anything?

On Shrubbery

Well, a few years ago, some friends were over visiting a day or so after I had trimmed the bushes in front of the house.  He (Dan, the male in this married couple, and father of a bright boy, married to the boy's mother)  made a comment, asked a question really, "Who trimmed the bushes?  They look very... zen."

I had.  I tend to do that a couple of times a year, three depending on the particular year and how much the bushes have grown.  You see, left to their own devices, shrubs - there are a couple of good-sized arbor vitae, a healthy juniper and a couple of others that I have no idea what they are.  The thing is, when it comes to trimming bushes, shrubs whatever, some of them can be trimmed into shapes really easily and will do quite well.  Others will struggle, sicken and eventually die.

As a shrub grows,and is untended for some time, it will do what ever it pleases.  It will stretch for light and water and... right you get the idea.  The thing is, if the thing is left alone, it will grow quite... big,  These were quite large the first time I tried to prune them.  How do you prune something that is probably larger than most people have them, and still allow them to be healthy and thrive whilst doing what they are supposed to do?

How does one get the most out of anything, help it develop to its full potential and achieve the goals one sets for it?

Well, with bushes, its pretty easy.  You can trim the recent growth, prune the large branches that are sticking up and neaten up the edges.  Maybe give it a nice round shape.  The thing is, from this point you can direct its growth from this point by trimming given branches one way or another - yet always allowing the bush to be a bush. 

So, this got me thinking.  It is, after all, Mother's Day.  Never having been one you see, how does a Mother guide her children in their development, growth, etc.?  I have my ideas, but, like I said, I'm not a mother - grandpa, yes, dad - well, I've played one now for an awful lot of years.  But, how does one teach young people to grow into, well, mature, positive people?

On Mothers

How does a good mother raise children?  Well, I can talk from observation of how the lady-wife and my mother tried - they focused on getting the best from their kids all the time, encouraging them to try things, if they wanted, and sticking through the commitment after starting.  Backing out after starting when things are hard or unpleasant or, boring, was discouraged.  Firm, yet loving might be a good short description.

My siblings and their spouses are using (or have used) this general approach with their kids.  Other folks we know are trying as well. Finding the balance is a challenge.  Learning what will motivate people and what gets them all excited can be a challenge - as any mother (or dad) with more than one child.  The fact is, one-size-fits-all style approaches simply do not work for raising kids - or bushes for that matter.

Some folks tell us that mothers always will do the right thing for their children.  Maybe in a movie.  I don't know.  There are plenty of examples of mothers who did a terrible job and were amazingly horrible at being mothers.  Let's face it - some people are not cut out to be mothers.  (Same with fathers, but my mind was wandering around on mothers today.)  They may want to be mothers, or think they do, but really, they have not considered what it means to be a mother at all, how much work it is and how long that work will continue.

The lady-wife tells people that becoming a mother is a life-changing event that will continue for the rest of their lives.  Once they become a mother, as long as they are alive they will be a mother.  Its a pity that some who chose to do so are not very good mothers.  I think sometimes people forget what their role is.

This got me thinking.

On Managers

Why do some managers try the one-size-fits-all when it comes to career-development, professional-development and generally assisting their staff to improve and grow?  Why do some people believe that the motivational devices that work for one or two people will work for everyone?  Yes, I know - it is up to the employee to seek out and strengthen their career - it is their career after all.  Still, managers can encourage them and show support to them for learning more and broadening their skill-sets, even if it includes learning concepts that may not be directly applicable to what they are doing right now.   After all, the way technology changes, what is to prevent them from learning something that will be helpful to them, the team and the broader company on the next project or a future project? 

Compared to mothers, managers have it easy.  They can become an "un-manager" by changing jobs, quitting, retiring, getting fired... lots of ways.  I wonder if people who want to be managers before they actually are managers would still want to be a manager after they learn the  reality of how much work it is.  Then again, if becoming an un-manager was as difficult as becoming an un-mother after being a mother (or a manager) I wonder if that would change how badly people want to be managers.  

Still, we have our own bias, don't we?  A fair number of people presume that because they wanted to become a lead and then a manager and then some higher level of boss, that the really good testers want to do the same.  Possibly the really not-so-good testers do as well.  I suspect this is because we see this path (novice >  experienced > really experienced > lead > manager > bigger manager > knight > Earl > King) as the only way to measure success.  Because "moving up" in the organization is what we are told is what we want to do - and some people really DO want to move up - then that is what everyone wants to do, right?

Or are biases and personal views getting in the way.  Mom wanted to take ballet when she was a little girl so darn it, her daughters will take ballet.  They only THINK they want to do soccer or karate.  But that can't be part of the one-size-fits-all thing right?  I wanted to be a manager so Joe must want to be a manager too!  I'll get him going and he can move up instead of being just a tester.  That is not part of the one-size-fits-all thing either.  Right?

So, Managers and Mothers - consider what it is that you want for your people - either staff or children.  Is that what you want or is that what they want?  We can't always get everything we want - I know - there's a song about that, right?

The question boils down to this - are we trying to shape out children and our staff in a way that they can be shaped or are we trying to force them into something they can not be?   Shaping and pruning them to enhance their growth will result in an amazing thing of beauty - a well-formed person.  If we force things they can not support, there is a chance they will wither and never achieve the potential that is theirs.

To all the good mothers in the world, I salute you.  Likewise all the good managers.

Bushes, shrubs and hedges?  My hedge clippers and pruning shears are cleaned and sharpened for the next time I need them. 

On Managing Testers, or Being a Coach

I had a revelation recently that I wish I could have had some time ago, like years.  It may have made me a better employee, and in my forays at lead and manager roles would have made me better at both.

While my thoughts here probably don't rise to the level of great writing that can be found for the betterment of leaders everywhere, it represents the sum of experience and belief demonstrated by many managers and leads I have worked for and with.  I trust that some may find it useful and help them manage testers perhaps better than they might without such thoughts.


1.  Encourage Learning.  This is important.  There is a significant difference between "training" and "learning."  Encourage your staff to look for ideas new to them and try things they have read about.  Don't bog them down with mandated, "This is the only way things are done" type of rote memorization.  Allow them to explore areas of interest to them, then gently guide them with how these can be applied to their current and future work.

2.  Discourage Self-limiting Behavior.  Kind of a contradiction, eh?  But really - anyone who has been around the block more than once has seen more than one person set themselves up for failure.  Maybe they like it.  Maybe they don't know any other way of doing things.  Help them discover what they are good at, what they enjoy doing and what they want to do and maybe other ways of doing things. 

3.  Encourage Participation.  I have found that one great way for people to learn and appreciate things, is to have a hand in the doing of it.  I know everyone is busy and is scheduled at 110%.  Well manager, MANAGE - encourage your staff to get involved - then make sure they have the time to do it without missing their kids school programs and family vacations and trips away and holidays and... right.  You get the idea.  The hard part about being a boss is telling other people "No."  And yet, we need to do that all the time, right?  This is just one more reason why you must tell someone "No," or if that makes you uncomfortable, try my favorite alternative "We can not work on that right now.  We should be able to start on it by ..."

4.  Discourage Lockstep Conformity.  I know we must all follow the process or whatever is in place at the company.  Sometimes it takes people with a lot more pull than we have to change the rules.  Well, sometimes bending the rules to let your people flourish is important too.  Allow them the space to be creative in approaching problems.  Allow them time to THINK.  Remember when you'd think about something deep and technical?  I still find a proper cup of tea, with a bit of sugar and a little milk, just sitting and sipping it, can help my brain unravel a problem that is in my way.  Allow your people the same room for creativity and innovation - particularly if there is no Time Reporting Code for that on their time sheets. 

5.  Encourage Innovation.  I find this tends to flow from what happens in number 4.  Sometimes thinking leads to a new approach.  This approach can take a little tinkering to get sorted just to see if it will work.  Give your people, that's what they are after all, people, the flexibility they need.  Don't force it on them, but if they want to try - let them.  Then help them learn and clear the path of roadblocks in ways only a Manager can.

6.  Discourage Disparaging Comments.  Negativity can be a serious drain on energy and morale and productivity.  If one of your people is down, a lot - see if you can help.  It might be nothing you can fix, but maybe just letting them know you are concerned is a help.  Then lighten their load, if possible, so they have the energy to deal with whatever is weighing them down.  If they are just being, well, negative for the sake of being negative, talk with them about that too.  Everyone gets down.  Everyone has a bad day (sometimes a week!)  Everyone has times when they'd rather be at the beach or the woods or... whatever.  We just need to get the job done.  Then let them go to the woods or beach or wherever.  Just keep an eye on folks - If they know you have their well-being in mind, more than just what they do at work, they may turn around and surprise you.

7.  Encourage Growth.  This one is a mix of "mentor" and "coach."  I know some people look to mentors for some level of guidance and teaching.  If you can do that for your people - fantastic!  Otherwise, keep an eye out for people who CAN do that - then get the mentors and students connected and allow things to happen - don't force them.  One thing I have learned is this.  Most technical professional types have a failing, somewhere.  One common one I have found is how they relate to other people - technical peers and non-technical as well.  It is amusing that so many people consider testers "non-technical" still, compared to some, most of us are.  Compared to others, we're technical rock stars.  Help them to navigate these waters.  Help them to learn to do them better.  Help them to learn to be a better team, and give them transferable skills they can use anywhere.

8.  Don't be the Control freak.  Like this guy. If you're my boss and you do those things, I'll quit.  Promise. 


Lessons Learned - I've seen these ideas applied often at various shops.  Where two or three are used effectively, the result has been stupendous.  The staff revel from one positive experience to another, the software gets better from one project to another, practices improve and, surprisingly at some shops, customer satisfaction went up - way up. 

Remember: Managers are the oracles of the Truth, but the people working for them are the spirit that make that truth relevant.  My wife used to say (when the kids were younger, at least, not in their 30's) "If Mama ain't happy, nobody's happy."  You can set the tone - will you be happy and uplifting or a burden to be endured?  By setting an example in attitude, work, learning and life balance, a good manager can do more than simply qualify for stock options and large bonus checks.