Testers and UX and That's Not My Job

OK.

I don't know if you are one of the several tester types I've talked with over the last couple of months who keep telling me that "Look, we're not supposed to worry about that UX stuff you talk about.  We're only supposed to worry about the requirements."

If you are, let me say this:  You are soooooooooooooooo wrong.

No, really.  Even if there is someone else who will "test" that,  I suggest, gently, that you consider what a reasonable person would expect while you are examining whatever process it is that you are examining.  "Reasonable person" being part of the polyglot that many folk label as "users."  You know - the people who are actually expected to use the software to do what they need to do?  Those folks?

It does not matter, in my experience at least, if those people (because that is what they are) work for your company or if they (or their company) pay you to use the software you are working on. 

Your software can meet all the documented requirements there are.  If the people using it can't easily do what they need to do, then it is rubbish.

OK, so maybe I'm being too harsh.  Maybe, just maybe, I'm letting the events of yesterday (when I was sitting in an airport, looking at a screen with my flight number displayed and a status of "On Time" when it is 20 minutes after I was supposed to be airborne) kinda get to me.  Or, maybe I've just run into a fair number of systems where things were designed - intentionally designed - in such a way that extra work is required by people who need the software to do their jobs.

An Example

Consider some software I recently encountered.  It is a new feature rolled out as a modeling tool for people with investments through this particular firm.

To use it, I needed to sign in to my account.  No worries.  From there, I could look up all sorts of interesting stuff about me generally, and about some investments I had.  There was a cool feature that was available so I could track what could happen if I tweaked some allocations in fund accounts, essentially move money from one account to another - one type of fund to another - and possible impact on my overall portfolio over time.

So far, so good, right?  I open the new feature to see what it tells me.

The first screen asked me to confirm my logon id, my name and my account number.  Well, ok.  If it has the first, why does it need the other two?  (My first thought was a little less polite, but you get the idea.)

So I enter the requested information, click submit and POOF!  A screen appears asking the types of accounts I currently had with them.  (Really?  I've given you information to identify me and you still want me to identify the types of accounts I have?  This is kinda silly, but, ok.)

I open another screen to make sure I match the exact type of account I have with what is on the list of options - there are many that are similar in name, so I did not want to be confused.

It then asked me to enter the current balance I had in each of the accounts.

WHAT????  You KNOW what I have!  It is on this other screen I'm looking at!  Both screens are part of the same system for crying out loud.  (or at least typing in all caps with a bunch of question-marks.)  This is getting silly.

So, I have a thought.  Maybe, this is intended to be strictly hypothetical.  OK, I'll give that a shot.

I hit the back button until I land on the page to enter the types of accounts.  I swap some of my real accounts for accounts I don't have - hit next and "We're sorry, your selections do not agree with our records."  OK - so much for that idea.

Think on

Now, I do not want to cast disparaging thoughts on the people who obviously worked very hard on this software, by some measure.  It clearly does something.  What it does is not quite clear to me.   There is clearly some knowledge of the accounts I have in this tool - but then why do I need to enter the information?

This seems, awkward, at best.

I wonder how the software came to this state.  I wonder if the requirements handed off left room for the design/develop folks to interpret them in ways that the people who were in the requirements discussions did not intend.

I wonder if the objections raised were met with "This is only phase one.  We'll make those changes for phase two, ok?"  I wonder if the testers asked questions about this.  I wonder how that can be.

Actually I think I know.  I believe I have been in the same situation more than once.  Frankly it is no fun.  Here is what I have learned from those experiences and how I approach this now.

Lessons

Ask questions.

Challenge requirements when they are unclear.
Challenge requirements when they are clear.
Challenge requirements when there is no mention of UX ideas,
Challenge requirements when three are mentions of US ideas.

Draw them out with a mind map or decision tree or something.  They don't need to be be fancy, but they can help you focus your thinking and may give you an "ah-HA" moment - paper, napkins, formal tools - whatever.  Clarify them as best you can.  Even if everyone knows what something means, make sure they all know the same thing..

Limit ambiguity - as others if their understanding is the same as yours.

If there are buzzwords in the requirement documents,  as for them to be defined clearly (yeah, this goes back to the thing about understanding being the same.

Is any of this unique to UX?  Not really.  I have a feeling that some of the really painful stuff I've run into lately would have been less painful if someone had argued more strongly early on in the projects where that software was developed.

The point of this rant - If, in your testing, you see behavior that you believe will negatively impact a person attempting to use the software, flag it.

Even if "there is no requirement covering that" - .  Ask a question.  Raise your hand.

I hate to say that requirements are fallible, but they are.  The can not be your only measure for the "quality" of the software you are working on if you wish to be considered a tester.

They are a starting point.  Nothing more. 

Proceed from them thoughtfully. 

In Defense of the Obvious, Testers and User Experience III

I have had some interesting conversations over the last few months with testers and designers and PM types and experts in a variety of fields.  I ask questions and they answer them, then they ask me a question and I answer it.

That is part of how a conversation works.  Of course, another part is that when Person B is responding to a question by Person A, it is possible, if not likely or probable that A will respond or comment to B.

This leads to B responding to A and so forth. Most folks know this is how a conversation works.

It is not a monologue or lecture or pontification.  It is an exchange of views, ideas and thoughts.

So, do all conversations follow the same model?  Are they essentially the same in form and structure?  Do they resemble those pulp, mass-produced fiction books that follow the "formula" used by the specific publisher?  You know the ones.  Pick one up, change the name of the main characters, change the name of the town - then pick up another from the same publisher and SURPRISE!  Same Story!  Change the name of the characters in the second book to what you changed the names from the first book - and see how similar they are.

OK.  Software folks -  Are your perceptions of users (you know, people who use your software to do what they need to do) as fixed as the characters in the mass-produced fiction books?  Or are your perceptions of users more like the participants in conversations?

Some Ideas I have that may seem really obvious to a fair number of folks, but I suspect are either revolutionary or heretical to others...

No Two People Are the Same

OK.  Obvious idea Number 1 for software testers: No two people are the same.  Duh.  Says so in red just above that, right?  They are the same, right?  Really?  How many differences can you spot?  (Go ahead, try.  Its OK.)

Why do we expect the people using the system to be a homogenous group where they generally act the same?  Think of people you work with who use software - ANY software.  Do they select similar options as each other?  Do they have the same interests?

Do they like the same coffee?  Do they do the same job?  Do they want to do the same job?  No, wait.  When you read the last couple of questions, what was your answer?  Do they REALLY do the same job?  Or do they do the same general function?

Are they doing something similar to each other?  Umm - similar is not the same, right?  If these are question you don't want to deal with - or maybe don't know the answer to - How are you designing your tests?

How are you designing your systems?

What "users" are your "user stories" emulating?

I had a bizarre chat fairly recently.  Boss-type said "We fixed this by using personas.  We can emulate people and mimic their behavior."

OK, says I to myself, reasonable idea and reasonable approach to formulating various scenarios.  They can be very powerful.  "Really," says I, out loud, "tell me about some of them.  Sounds like it could be cool."

"Sure!" says the very proud boss-type, "We have Five of them: One for each department."  Really? So, tell me more.  "Sure!  Persona 1 does the thing-a-ma-bob function.  Persona 2 does the dumaflatchey function.  Persona 3 does the whats-it function.  Persona 4 does the thing-a-ma-jig function (similar to the thing-a-ma-bob function but not the same).  Persona 5 does the whatever function."

So, a total of five personas?  OK, how many people are in each department?

"Well, the smallest department has 15 people.  The others have 75 to 100."

Really?  They are all the same?  They all do the same thing every time?  They never vary in their routine?

Do they all do the same thing your test scenarios do - in that sequence - every single time they go into the system?

Sometimes People Have Bad Days

Yeah, I know you thought that only applied to software folks.   Sometimes super-model types have bad day's too.  Of course, famous folk have bad days - then they get their picture in various tabloids and their "bad day" seems not so bad because all the attention because of the tabloids is worse than the original "bad day."

Bad days can impact more than just our coding or testing or a public figure's dinner plans.  Remarkably enough they can impact people who use the software we're working on. 

Sometimes people have too much fun the night before they are in the office using our software.  Their typing is less than perfect.  They are less accurate than normal in their work - they read things wrong; they invert character sequences; they simply don't notice their own mistakes.

Sometimes they had a really bad night instead of a really good night.  Maybe they were up half the night caring for a sick child.  Maybe it wasn't a child, maybe it was a partner.  What if it was a parent? 

The results may be the same outwardly, but what about the inner turmoil? 

"Is my child/partner/parent doing better now? Do I need to check on them? What if I call and they don't answer the phone?  If they are sleeping, I may wake them.  If they can't get to the phone, why not? Something could be seriously wrong?"

Will they be more irritable than they normally are?  Will that impact others in the group and cause their productivity to drop?

Sometimes the Best People Aren't at Their Best

What?  How can that be?  Aren't they like what the Men In Black are looking for?  Aren't they "Best of the Best of the Best" (sir)?

What if they are too good?  What if they get asked questions and are interrupted and step away from their machines for a minute or lock their screens while they help someone else?  What if their user session times out?

Let's face it.  Anyone can get distracted.  Anyone can be interrupted.  Is the system time-sensitive?  How about state sensitive?  The session can time-out mid-transaction, can't it?  Someone else has a problem so the expert locks her system and helps out the guy with a problem - what happens with her session when she comes back? 

Do you know? 

What if they get called into a conference room with some boss types to answer some questions.  If she signs in from another location, what happens to her first session?

And so forth...

These are not new ideas.  Do we know what happens though? 

Now some of you may be thinking to yourself "But Pete, this is not really UX kind of stuff, is it?"  That makes me wonder what kind of "stuff" they might be.

Do your test scenarios consider these possibilities?  Do they consider any one of them? 

Testing to Requirements

Ah yes.  I hear the chorus of "Our instructions are to test to the requirements.  Things that aren't in the requirements should not be tested.  They are out of scope."  Whose requirements?

The requirements that were written down by the BA (or group of them) or the ones that were negotiated and word-smithed and stated nicely?

What about the requirements that the BA did not understand, hence did not write down.  Or maybe he wrote them down but they made no sense so other folks scrapped them. 

Then there are the implied requirements.  These are the ones that don't make the documented requirements because they seem so obvious.  My favorite is the one about "Saving a new or modified record in the system will not corrupt the database."

You hardly ever see that, but everyone kind of expects that.  Right?  But if you are ONLY testing to DOCUMENTED requirements, then that does not count as a bug, right?  It is out of scope.  RIGHT?

NO?  Really? 

See? That is kind of my point.  You may be considering the experience of the users already.  You just don't know it. 

Now, broaden your field of vision.  Pan back.  Zoom out.  What else is obvious to the users that you have not considered before?

Now go test that stuff, too.