The Automated Testing Trap

A brief sidebar at the latest local Tester meetup (#GRTesters) caused a certain amount of thinking and remembering. This is the result.

Before venturing back into contract work, I was working for a large company where bosses had very definite ideas about "testing" and "automation" and "code." Some of them were quite reasonable. Some of them resulted in me, and others, pushing back really, really hard on their assertions.

I got responses like "Have you read the book {big boss} talked about? If you have, you would understand his message." Yeah, except I knew the authors of that book. We had presented at the same conferences and had taken each other's workshops. I also knew the people their work was build upon, whom they cited. So, I'm sorry. I still have no comprehension what this means.

I got pretty well ignored after that. Then I read the "new" job descriptions for people doing what I did. Oh my.

I pushed back really hard on those - You SAY you want certain skills, and somehow you ignore the skills needed to make use of the skills you say are required. I think there is a disconnect.

The Problem of Automation

To begin, I am a fan of using automation for testing. I want to use the best possible tools to make sure the best possible outcome. I want to use tools that help me do good work.

I see more and more positions for "Automation Testers" that focus on the primary development language, characteristics of the stack, tool set, Git repository and on and on. Many of these read like the job descriptions I was speaking out about a few years ago. I have kind of been pushing hard against these types of characterizations for some time. (I think that is why some folks consider me to be "anti-automation.")

Please, don't misunderstand me. Those things can be important, in some contexts for some organizations.

To paraphrase the Wendy's commercial from 1984, "Where's the testing?"

By placing the emphasis of the job descriptions, notices and search terms into the easy to understand structure used for developers, there is a disconnect. By focusing on development skills at the cost of all other skills, for example, actual skill and experience in testing, you are limiting the role you seek to fill.

More importantly, you put the product and your organization, at risk - if not potentially in mortal danger.

On Situational Awareness

After a bit of consideration, I had a thought.I realize it does not fit ALL tech type managers. Still, many I have met clearly have a mindset something like THIS when it comes to testing. It is reflected throughout development teams.

I don't place the blame for this on the "code camps" many developers are coming from. I do not place this on the colleges and universities where many more have come from and continue to come from.

Instead, I think it is a failure on many levels to understand what Software Development as a whole is. I think it is a complete failure in "situational awareness." People are not aware what the full picture is. They are focused on their little corner of it, if that.

The failure, as Sherlock Holmes would put it, is "You see, but you do not observe."

Part of this is failing to recognize there are many forms and types of "Automation." Something like a typical CI environment, running low level tests that are effectively base-line tests for core functionality, is one aspect. Building meaningful regression tests to exercise software function to cover known behaviors is another. Adding to and removing from both of these sets of test suites takes time and careful consideration.

Discounting the need for experience in testing over experience writing code is a disaster waiting to happen.


On Testing

Good Testing is not a box to be checked.
Good Testing takes careful, considered work.
Good Testing is crucial to understanding how your software product behaves before your customer gets their hands on it.

The issue I see in many, many organizations is most "leaders" of the software organization have no idea what Good Testing is, nor what is involved in achieving that. I wrote a series of blog posts on this some time ago.

Start here: You Call THAT Testing?
Then go here: The Failure of Testers
Then go here: The Failure of Management
And then go here: Testers Rising to the Challenge

Finally, there is this: Why Don't Managers Understand What Good Testing Is?

I have spent the last many, many years trying to help people understand that thinking, good, testing is impossible to be removed from good software development. Apparently it is easier to talk accept the glossy paper snake-oil people sell while sipping craft beer and artisanal cocktails.

"Automation" will not help your company, team or project. Automation driven by informed planning, driven by people trained in testing can likely help a great deal.