On Testing Purpose and Documented Requirements

It has been a really crazy busy four weeks at the day job.  Somehow, I ended up with several "challenged" projects in my lap.  Yesterday was rather strange, in fact.  I got in at my new "normal" time of 7:45 and realized that there was no reason for me to be there that early.  All the troubled projects were sorted out, there were no emergencies or top-priority bugs waiting further investigation.  There were no outstanding tasks to do before getting my first coffee of the morning.  Everything was kind of... dull.

The cool thing was a comment that a Project Manager said to me.  She looked at me and said "On Project K and Project M (two of the troubled projects) you came in and looked at the tests and skimmed the requirements then talked with the business process experts.  Why did you do that?  The test lead who started on the project did not do that."

This got me thinking on past projects and companies and clients and places where things did not work well.  Each time someone mentioned something like that above statement, part of me controlled myself.  Another part smiled to myself thinking "And that explains the state of testing for those projects." 

What I actually said was "It is good to know what the documented requirements are.  It is also good to know what existing tests have been set up or run already.  It is essential to know what problem the project is intended to address.  It is crucial to know what people are doing now and what they need the software to do for them." 

When you sit down and actually speak with people in a conversation, not an inquisition, not in a meeting with loads of people who don't really care about their work, you can learn things that may not otherwise be learned.  When your energy is one of "please help me to understand so I can help you better" - as a servant, not a superior, walls come tumbling down and this weird thing happens:  People tend to share information freely.

Too often, IT folks come in as the experts who know how to fix problems that these lesser being have.  I find that offensive.  My handful of accounting and finance courses lo those many years ago do not qualify me to know more than the CFO, nor even any of the folks working for him.

So, rather than go on a 10 minute rant, I kept it simple.  (Yes, I sometimes can control myself.)

"But isn't that what the BA is supposed to do?"

Oh, excellent question.

Certainly a good BA can gain that information and pass it on to the team.  However, with each layer or remove introduced in the information flow, something changes.  Information may be lost.  Other things may be added.  The "clarification" may critically change part of the message.  If I can get as close to the people doing the work as I can, I often learn stuff that is important to me as a tester that other people shrug off as unimportant.

If I can understand their needs better, I can exercise the software more efficiently.  If the tests I write and run do not explicitly exercise "the requirements" are they really required?  Are they interpretations of something?

Sure, there may be some things that are important, like "don't corrupt the database" or "this needs to be reflected in the ziggidy-splat system" or "this value should match what is on this other screen."  Do these things make the user's list of expectations?  Do they make the documented list of "requirements"?

In having these conversations, I find out what the answer is.

If you only test the "documented requirements" how do you know that you are providing information the clients/customers/people using the software will be interested in?  If the things they are interested in are not in the "documented requirements" what is the value in the documented requirements?

If you can, exercise both.  If you can't exercise the stories.  Then have folks from outside of IT sit down with you and go through it as well.