In Defense of the Obvious, Testers and User Experience III

I have had some interesting conversations over the last few months with testers and designers and PM types and experts in a variety of fields.  I ask questions and they answer them, then they ask me a question and I answer it.

That is part of how a conversation works.  Of course, another part is that when Person B is responding to a question by Person A, it is possible, if not likely or probable that A will respond or comment to B.

This leads to B responding to A and so forth. Most folks know this is how a conversation works.

It is not a monologue or lecture or pontification.  It is an exchange of views, ideas and thoughts.

So, do all conversations follow the same model?  Are they essentially the same in form and structure?  Do they resemble those pulp, mass-produced fiction books that follow the "formula" used by the specific publisher?  You know the ones.  Pick one up, change the name of the main characters, change the name of the town - then pick up another from the same publisher and SURPRISE!  Same Story!  Change the name of the characters in the second book to what you changed the names from the first book - and see how similar they are.

OK.  Software folks -  Are your perceptions of users (you know, people who use your software to do what they need to do) as fixed as the characters in the mass-produced fiction books?  Or are your perceptions of users more like the participants in conversations?

Some Ideas I have that may seem really obvious to a fair number of folks, but I suspect are either revolutionary or heretical to others...

No Two People Are the Same

OK.  Obvious idea Number 1 for software testers: No two people are the same.  Duh.  Says so in red just above that, right?  They are the same, right?  Really?  How many differences can you spot?  (Go ahead, try.  Its OK.)

Why do we expect the people using the system to be a homogenous group where they generally act the same?  Think of people you work with who use software - ANY software.  Do they select similar options as each other?  Do they have the same interests?

Do they like the same coffee?  Do they do the same job?  Do they want to do the same job?  No, wait.  When you read the last couple of questions, what was your answer?  Do they REALLY do the same job?  Or do they do the same general function?

Are they doing something similar to each other?  Umm - similar is not the same, right?  If these are question you don't want to deal with - or maybe don't know the answer to - How are you designing your tests?

How are you designing your systems?

What "users" are your "user stories" emulating?

I had a bizarre chat fairly recently.  Boss-type said "We fixed this by using personas.  We can emulate people and mimic their behavior."

OK, says I to myself, reasonable idea and reasonable approach to formulating various scenarios.  They can be very powerful.  "Really," says I, out loud, "tell me about some of them.  Sounds like it could be cool."

"Sure!" says the very proud boss-type, "We have Five of them: One for each department."  Really? So, tell me more.  "Sure!  Persona 1 does the thing-a-ma-bob function.  Persona 2 does the dumaflatchey function.  Persona 3 does the whats-it function.  Persona 4 does the thing-a-ma-jig function (similar to the thing-a-ma-bob function but not the same).  Persona 5 does the whatever function."

So, a total of five personas?  OK, how many people are in each department?

"Well, the smallest department has 15 people.  The others have 75 to 100."

Really?  They are all the same?  They all do the same thing every time?  They never vary in their routine?

Do they all do the same thing your test scenarios do - in that sequence - every single time they go into the system?

Sometimes People Have Bad Days

Yeah, I know you thought that only applied to software folks.   Sometimes super-model types have bad day's too.  Of course, famous folk have bad days - then they get their picture in various tabloids and their "bad day" seems not so bad because all the attention because of the tabloids is worse than the original "bad day."

Bad days can impact more than just our coding or testing or a public figure's dinner plans.  Remarkably enough they can impact people who use the software we're working on. 

Sometimes people have too much fun the night before they are in the office using our software.  Their typing is less than perfect.  They are less accurate than normal in their work - they read things wrong; they invert character sequences; they simply don't notice their own mistakes.

Sometimes they had a really bad night instead of a really good night.  Maybe they were up half the night caring for a sick child.  Maybe it wasn't a child, maybe it was a partner.  What if it was a parent? 

The results may be the same outwardly, but what about the inner turmoil? 

"Is my child/partner/parent doing better now? Do I need to check on them? What if I call and they don't answer the phone?  If they are sleeping, I may wake them.  If they can't get to the phone, why not? Something could be seriously wrong?"

Will they be more irritable than they normally are?  Will that impact others in the group and cause their productivity to drop?

Sometimes the Best People Aren't at Their Best

What?  How can that be?  Aren't they like what the Men In Black are looking for?  Aren't they "Best of the Best of the Best" (sir)?

What if they are too good?  What if they get asked questions and are interrupted and step away from their machines for a minute or lock their screens while they help someone else?  What if their user session times out?

Let's face it.  Anyone can get distracted.  Anyone can be interrupted.  Is the system time-sensitive?  How about state sensitive?  The session can time-out mid-transaction, can't it?  Someone else has a problem so the expert locks her system and helps out the guy with a problem - what happens with her session when she comes back? 

Do you know? 

What if they get called into a conference room with some boss types to answer some questions.  If she signs in from another location, what happens to her first session?

And so forth...

These are not new ideas.  Do we know what happens though? 

Now some of you may be thinking to yourself "But Pete, this is not really UX kind of stuff, is it?"  That makes me wonder what kind of "stuff" they might be.

Do your test scenarios consider these possibilities?  Do they consider any one of them? 

Testing to Requirements

Ah yes.  I hear the chorus of "Our instructions are to test to the requirements.  Things that aren't in the requirements should not be tested.  They are out of scope."  Whose requirements?

The requirements that were written down by the BA (or group of them) or the ones that were negotiated and word-smithed and stated nicely?

What about the requirements that the BA did not understand, hence did not write down.  Or maybe he wrote them down but they made no sense so other folks scrapped them. 

Then there are the implied requirements.  These are the ones that don't make the documented requirements because they seem so obvious.  My favorite is the one about "Saving a new or modified record in the system will not corrupt the database."

You hardly ever see that, but everyone kind of expects that.  Right?  But if you are ONLY testing to DOCUMENTED requirements, then that does not count as a bug, right?  It is out of scope.  RIGHT?

NO?  Really? 

See? That is kind of my point.  You may be considering the experience of the users already.  You just don't know it. 

Now, broaden your field of vision.  Pan back.  Zoom out.  What else is obvious to the users that you have not considered before?

Now go test that stuff, too.

When Fortune Turns the Wheel

Much has happened since my last little missive.  It seems I've been busy, even by my own standards.

I recently wrapped up assisting with another installment of AST's BBST® (yeah, its now a registered trademark of Kaner Fiedler & Associates LLC) Foundations course.  There were a PILE of students in this instance.  Like many times, there were some students who were really well prepared and willing to dive into the exercises, and there were some who were less prepared. Or maybe less willing to participate.  Either way, the results were the same.  The result was some did really well, some had a harder time.

Oh.  I also wrote two presentations, facilitated a local tester meeting, presented one of the presentations at the local tester meeting and dealt with a whole stack of goodies at house stuff and garden stuff and life stuff.  Garden stuff?  Yeah.  Its August - now September.  We have more tomatoes and peppers and ... stuff coming in from the garden than I can remember seeing.  Oh.  Potatoes.  We'll be harvesting them soon.  It is crazy how much stuff we have coming in.  And general tending and watering and keeping the lawn mowed and all the normal family/household activities.

I'm also getting ready for a really, REALLY busy couple of months.  In October, I'll be attending and participating at PNSQC in Portland, Oregon. I'll be hanging out with some cool folks and learning an sharing there.  And then, just as I'm about  In November, I'll be at Agile Testing Days in Potsdam, Germany.

In the midst of this, I resigned my position with the company I  worked for.  I became an independent consultant.  I entered into a contract with a local company that was looking for a little help.  With this, one has all the joys of starting a new job (no matter how cool or how much fun the gig is, there is still a moderate amount of strain) and starting a new business while trying to keep your feet on the ground.  Oh yeah!

So, when the paperwork is finalized, I'll be excitedly announcing a new business name.  It seems I've been so busy that I'm still fumbling my way through the various legal documents.  Can't imagine what I've been doing that has gotten in the way.

So, yes.  A third installment on User Experience and Testers is in the works.  It will be out shortly.  I also have some notes I am trying to pull together to make another couple of articles that are still a bit cloudy.  So, there is more stuff coming. 

Thanks for reading.