It was an interesting day in Lake Wobegon. I spent a fair chunk of the afternoon with a tester who was having a problem. She was working on writing tests that others would execute - except that the others never followed her scripts. The really missed the gist of most of the steps. She kept saying "Why can't they just follow instructions?"
My response was not very nice. "Why are you writing tests that make sense to you, but others have a hard time executing? What if you simplified what it was that each test was doing?"
"ALL THEY HAVE TO DO IS READ THE INSTRUCTIONS CAREFULLY THEN FOLLOW THEM!"
Sure. And if they are confusing? What if they don't know they are doing the wrong thing? How could they tell?
Here's the gist of my suggestions from this conversation, and I suspect it may help others in "scripted testing land":
1. Simply, simplify and then simplify. If you are writing tests you will not be executing, don't make them mathematical problems to solve - and then follow the equation to get the answer. Give the people executing the tests the information they need to do what you need them to do. Don't make them solve puzzles just to start the work.
2. State what the goal of the test is. Explain it briefly and what the purpose is. Explain they why of what they are doing and they may understand better. They may also think of things of value which you had not considered.
3. Be open to suggestions. I don't know everything. I bet most folks don't either - no matter how much they want us to think they do. Other people may make suggestions. They make me a better tester.
4. Be open-handed. When folks (like lower level testers) make a good (or great) suggestion - let them know you appreciate it. When it pays out in rewards - make sure the greater portion of the credit goes to those who made the suggestion.
5. Be honest with yourself. Delusions are deadly.
6. Keep calm.
I sometimes upset people without intending to. Sometimes, well usually, I feel bad about upsetting them. Unlike some folks I don't say things to intentionally cause pain.
If you are working in a shop where I am working, and where I have been tasked with helping people test better - remember that I will never do anything to hurt you out of cruelty or malice. If you think I am - say so - then be ready for some more.
My role is to help you improve, just as I turn to other folks to learn. Usually that pain, speaking from personal experience, is a sign of growth.Your old ideas are passing away. They are making way for new ways of thinking and that can hurt. A LOT!
When the grandkids were living with us and something happened, they were always afraid we would be mad. No - not usually. Upset maybe. Sometimes were got frustrated. But we were not mad - or angry. They'd look at us in amazement - "You're not mad at me? Really?" No. Sometimes we were a little hurt, but not mad.
"Honey, you have only been on this Earth 12 years. I can't expect you to know everything. Sometimes things happen when you don't intend them to. I'm not mad. Let's talk about what happened and see if we can find another idea in case this comes up again."
My wife gave me that tremendous gift. It works with granddaughters, grandsons and (ready?) Junior Testers.
We can't expect you to know everything someone who has been doing this for many, many years knows. And when we help you it is because we want you to grow and blossom. We're not mad at you.
Just remember to keep your tests as simple as you can. That way less experienced folks will understand. And maybe us old folks will understand as well.
Subscribe to:
Post Comments (Atom)
I'd like to suggest something a little different.
ReplyDelete0. When they're not doing what you think you're telling them to do, treat that as information.
That is, instead of rebuking them for not following instructions, observe them to find out what the problem is. Maybe the problem is that they're trying to test, rather than to check.
Related to your (2) above, before you state what the goal of the test is, ask what the goal of the test is. The goal of the test, presumably, is to reveal problems in the product. Why are you giving them an explicit set of instructions at all? Will a prescriptive set of actions do that? Or could a more open-ended assignment make productive use of natural human variability to expose problems
Be careful of simplification, too. A simple process—or a simple test (check) may not reveal problems that a more elaborate test might. Simplification can be good when you're looking at something simple, like a particular function or a relatively small and uncomplicated task. But don't underestimate the value of complex actions, rich scenarios, and galumphing.
Finally, I'd like to suggest that Google fixes the usability problems in Blogger, but that's fodder for another LONG series of posts.
---Michael B.
Broadly speaking, I agree. About the (2) I think "Purpose" or "Intent" might be better. Let me think on that.
ReplyDeleteThe rest is, alas, an on-going cycle. I've seen some really good work and some that is less good. I am a firm believer in /galumphing/ and use it. My concern is that instead of exercising one or two things at a pass, these tests exercise many things. The result is that when something behaves unexpectedly, tracking down the behavior is a huge challenge.
I prefer to establish how "function A" behaves, then "function B" then "function C". From that point on, the real fun begins. How does "function A" behave when "function B" is also in play? What about C? What about all 3?
Pressed for time right now. Your comment is deserving of more thought.
Thanks for contributing.
P.