I have had some interesting conversations over the last few months with testers and designers and PM types and experts in a variety of fields. I ask questions and they answer them, then they ask me a question and I answer it.
That is part of how a conversation works. Of course, another part is that when Person B is responding to a question by Person A, it is possible, if not likely or probable that A will respond or comment to B.
This leads to B responding to A and so forth. Most folks know this is how a conversation works.
It is not a monologue or lecture or pontification. It is an exchange of views, ideas and thoughts.
So, do all conversations follow the same model? Are they essentially the same in form and structure? Do they resemble those pulp, mass-produced fiction books that follow the "formula" used by the specific publisher? You know the ones. Pick one up, change the name of the main characters, change the name of the town - then pick up another from the same publisher and SURPRISE! Same Story! Change the name of the characters in the second book to what you changed the names from the first book - and see how similar they are.
OK. Software folks - Are your perceptions of users (you know, people who use your software to do what they need to do) as fixed as the characters in the mass-produced fiction books? Or are your perceptions of users more like the participants in conversations?
Some Ideas I have that may seem really obvious to a fair number of folks, but I suspect are either revolutionary or heretical to others...
No Two People Are the Same
OK. Obvious idea Number 1 for software testers: No two people are the same. Duh. Says so in red just above that, right? They are the same, right? Really? How many differences can you spot? (Go ahead, try. Its OK.)
Why do we expect the people using the system to be a homogenous group where they generally act the same? Think of people you work with who use software - ANY software. Do they select similar options as each other? Do they have the same interests?
Do they like the same coffee? Do they do the same job? Do they want to do the same job? No, wait. When you read the last couple of questions, what was your answer? Do they REALLY do the same job? Or do they do the same general function?
Are they doing something similar to each other? Umm - similar is not the same, right? If these are question you don't want to deal with - or maybe don't know the answer to - How are you designing your tests?
How are you designing your systems?
What "users" are your "user stories" emulating?
I had a bizarre chat fairly recently. Boss-type said "We fixed this by using personas. We can emulate people and mimic their behavior."
OK, says I to myself, reasonable idea and reasonable approach to formulating various scenarios. They can be very powerful. "Really," says I, out loud, "tell me about some of them. Sounds like it could be cool."
"Sure!" says the very proud boss-type, "We have Five of them: One for each department." Really? So, tell me more. "Sure! Persona 1 does the thing-a-ma-bob function. Persona 2 does the dumaflatchey function. Persona 3 does the whats-it function. Persona 4 does the thing-a-ma-jig function (similar to the thing-a-ma-bob function but not the same). Persona 5 does the whatever function."
So, a total of five personas? OK, how many people are in each department?
"Well, the smallest department has 15 people. The others have 75 to 100."
Really? They are all the same? They all do the same thing every time? They never vary in their routine?
Do they all do the same thing your test scenarios do - in that sequence - every single time they go into the system?
Sometimes People Have Bad Days
Yeah, I know you thought that only applied to software folks. Sometimes super-model types have bad day's too. Of course, famous folk have bad days - then they get their picture in various tabloids and their "bad day" seems not so bad because all the attention because of the tabloids is worse than the original "bad day."
Bad days can impact more than just our coding or testing or a public figure's dinner plans. Remarkably enough they can impact people who use the software we're working on.
Sometimes people have too much fun the night before they are in the office using our software. Their typing is less than perfect. They are less accurate than normal in their work - they read things wrong; they invert character sequences; they simply don't notice their own mistakes.
Sometimes they had a really bad night instead of a really good night. Maybe they were up half the night caring for a sick child. Maybe it wasn't a child, maybe it was a partner. What if it was a parent?
The results may be the same outwardly, but what about the inner turmoil?
"Is my child/partner/parent doing better now? Do I need to check on them? What if I call and they don't answer the phone? If they are sleeping, I may wake them. If they can't get to the phone, why not? Something could be seriously wrong?"
Will they be more irritable than they normally are? Will that impact others in the group and cause their productivity to drop?
Sometimes the Best People Aren't at Their Best
What? How can that be? Aren't they like what the Men In Black are looking for? Aren't they "Best of the Best of the Best" (sir)?
What if they are too good? What if they get asked questions and are interrupted and step away from their machines for a minute or lock their screens while they help someone else? What if their user session times out?
Let's face it. Anyone can get distracted. Anyone can be interrupted. Is the system time-sensitive? How about state sensitive? The session can time-out mid-transaction, can't it? Someone else has a problem so the expert locks her system and helps out the guy with a problem - what happens with her session when she comes back?
Do you know?
What if they get called into a conference room with some boss types to answer some questions. If she signs in from another location, what happens to her first session?
And so forth...
These are not new ideas. Do we know what happens though?
Now some of you may be thinking to yourself "But Pete, this is not really UX kind of stuff, is it?" That makes me wonder what kind of "stuff" they might be.
Do your test scenarios consider these possibilities? Do they consider any one of them?
Testing to Requirements
Ah yes. I hear the chorus of "Our instructions are to test to the requirements. Things that aren't in the requirements should not be tested. They are out of scope." Whose requirements?
The requirements that were written down by the BA (or group of them) or the ones that were negotiated and word-smithed and stated nicely?
What about the requirements that the BA did not understand, hence did not write down. Or maybe he wrote them down but they made no sense so other folks scrapped them.
Then there are the implied requirements. These are the ones that don't make the documented requirements because they seem so obvious. My favorite is the one about "Saving a new or modified record in the system will not corrupt the database."
You hardly ever see that, but everyone kind of expects that. Right? But if you are ONLY testing to DOCUMENTED requirements, then that does not count as a bug, right? It is out of scope. RIGHT?
NO? Really?
See? That is kind of my point. You may be considering the experience of the users already. You just don't know it.
Now, broaden your field of vision. Pan back. Zoom out. What else is obvious to the users that you have not considered before?
Now go test that stuff, too.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment