Considering Conferences: A Reflection

The last week I have had a series of interesting thought/introspection sessions. The kind where I pour a decent measure of good whisky sit back and look out the window and think. It helps to have a nice garden to see when I look out the window - and since the house does not have a fireplace and it is too bloody cold out to have a fire in the garden, well, staring into the fire is out of the question right now.

The gist of my thoughts? Let's see.

After speaking at many conferences a year on for several years, on various topics, then filtering down to a few conferences each year, and now one or two conferences a year - I have observed things that make me wonder why I should keep going and speaking at conferences.

Don't get me wrong. I really enjoy the conversations and sharing ideas that happen. Most of these, for me, happen in the hallway or "after hours."

Perhaps it is just that I've grown cynical with regard to the desire of so many people to belong to the cool-kid-club.

What do I mean?

For example - Cool-Kid A and Cool-Kid B talk about an exercise or a book and how awesome it is. They write about it and talk about it. In a short time, loads of other people are jumping in - recommending the same book or exercise. IN a year or so - SCADS of people are talking about how awesome it is and how cool it is. A year later they are all onto something... else. Typically endorsed by the cool kids.

People stopped talking about what they KNOW - what they have experienced and can demonstrate and instead talk about ... whatever the cool kids are talking about. Presentations - even keynotes - get more hand-wavy and full of dancing unicorns (the kind that are on the screen and don't really have a bearing on what is happening with people in the room) and truisms get spouted and straw-men get setup and then slain and people applaud and talk about what a great presentation it was.

My question is - What was the presentation about and how will it impact your work when you get back to the office?

Is the information really new to the people who think it was awesome, or does it merely confirm their beliefs on a subject. Are they so shallow as to not be able to consider going to a presentation they disagree with? Are they afraid of learning something new or taking on material that might lead them to reconsider what it is they think or believe?

I feel the same way about the cool books or exercise - the cool kids advocate.

How has this changed what you do as a tester? NO - Don't give me more bullshit answers about how it changed how you think. LOADS of things can change how you think. Good whisky can change how you think!

How did THIS book or THIS exercise change how you do your daily job? Because frankly, I suspect much of it is little more than mental self-indulgence (I had another word but a proof reader said it might be too explicit for some folks...)

And don't even get me going on so many bullshit keynotes I have heard. The abstracts were... harbingers of hand-wavy crap. Somehow the reality of the presentation so many times has been... more hand-wavy than the abstract. I find it actually quite depressing.

Why?

When I read proposals for conference presentations, I see a series of similarities and "sameness" among them. There are the ones that regurgitate ideas from 20 years ago. There are the ones that claim a new take on a topic - and when I dive in I find them to be regurgitating ideas from 20 years ago.

I see the cookie-cutter, instant solution to all your problems so frequently, I swear (as in take an oath) when I review proposals for conferences, I will not go to any conference that has people presenting such shallow rubbish - ever.

What is one of my warning signs? If the speaker or potential speaker cannot carry on a conversation for 40 minutes ON THE TOPIC THEY ARE PRESENTING ON - do they have any experience in the area or are they babbling theory only? Do they have any real-world applications of what they are presenting? Or...

I'm not sure why this is. I am not sure why people latch onto certain ideas and never vary from them. I am not sure why hand-wavy bullshit is so appealing to people, except for one small thing - it takes no effort to think about because there is nothing there TO think about.

I expect keynote speakers to be exceptionally well versed in their subject. When talking with them, either in advance when preparing a program or over drinks at a conference, I expect them to be able to cite specific experiences - nothing that would violate an NDA, but something like...

There was this problem where... The possible solutions I saw and presented to try were... The team tried this approach... The results were... 

IF they worked - fantastic. What was it that made the solution work? Why did this work? If not - then why did it not work? Talking over drinks (or over the phone) I want to know if the person has "been there and done that" to steal an over-used phrase, or if they are telling a fairy tale/war-story.

When I try to coach new speakers - to help them find their voice - the ground they are going to stake out as their own, I try to help them find a message that is uniquely their own. I try and help steer them away from the ground so often trod on by others and show a little used path they can take.

I expect that speakers will give me something I had not thought of before, some insight I have not considered or some point of view I can make use of when I get back to the office.

Avoid the band-wagon cool-kid trends. Speak and write on what you know and are passionate about.

That might just convince me to read what you write and maybe even go to the conference where you are presenting the idea.

Until then, be well and be.

End to End Testing and Integration Testing Revisited

After a crazy busy "black Friday" at work,  I was hiding at the small local coffee shop enjoying a really good cup of coffee. It was really very good coffee. Very nice.

I was half way though my first cup (I limit myself to 2 at this shop) looking out the window, glad I was not part of the mob scene of young people debating going to the local bar, the fancy coffee shop or replenishing the supplies from the "party" store next door. (There were loads of parties Wednesday night and Friday in the area.)

A guy I sort of knew from the local test meet up pulled up a seat and began talking about how awesome things were at work. He was really really proud and needed to tell someone about it. Their End-to-End Test Automation had been up and running for 3 releases and it was AWESOME! They made some tweaks and it ran clean and fast and all you had to do was click "Start" and it ran.

OK - Sounds pretty cool so far.

It has 300 steps - well, closer to 400  but, you know... (More coffee for me, this might be a long conversation.) And the first few times it ran it found a bunch of bugs and they could get them all fixed and now it is running really really clean!

Wow - OK, that sounds line something. So I ask the next question that seemed logical to ask...

So, this has been running for 3 releases? You found a bunch of bugs in the test environment early on, right? (Yeah, we did - saved our butts.) Cool. That's good. So, how about bugs in the  production environment? Got any cropping up there?

Well, there are always bugs in production, right? Something always goes wrong and there really isn't anything we can do to prevent them in advance. You know how it is. There's one the keeps cropping up. Seems like this reappears every few weeks. We just fix the data and move on.

Hmmm. So, is there a way to recreate the scenarios in the test environment and see if you can head off these issues you had in production? Is there a way to maybe isolate why you  need to "fix" data every few weeks?

Well, it's really complicated. We don't control that part of the data. Another team does. They claim their stuff works fine and the problem is with our stuff, so it is our problem.

Ouch. That kind of sucks. (Thinking about breaking my rule and going for a third cup of this really good coffee.) Ummm, what about integration testing? Is that a thing for you? or, not really?

Well, all the teams are testing their stuff in the System Integration Test environment we have. But we never seem to find these kinds of bugs there.

Hmmm. Yeah. That can be frustrating (wishing I was drinking a nice red wine at this point.) What would it take for that to happen?

We can't do that. We're Agile. We don't do that kind of thing. But hey - I see my wife outside - Gotta go! Later - See ya next month maybe at the meeting!

I sigh. realize I'm half way through my third cup of coffee and I hear my friend the Unicorn start laughing. YEAH - the Unicorn! I had not seen him in MONTHS.

He wasn't there when you talked about "Pulp Fiction Integration Testing" was he. Too bad.

(I blogged about it a long time ago here.)

So the Unicorn and I chatted for a while.

The problem we both have with much of the "automated end-to-end testing" stuff is that thinking humans tend to stop thinking and allow things to run on auto-pilot. If everything is green at the end there are no problems and anything that comes up in production was something that could not have EVER been anticipated.

Except often times they CAN be anticipated. Running tests in an "environment" does not make them "integration" tests. It does not mean they are telling you anything of value.

I might suggest this idea - try getting other teams involved - Other people who deal with systems or applications that interact with yours. Try comparing notes. Try setting up scenarios emulating what actually happens when the software is released to the wild.

Can you evaluate the touch points? Can you monitor the database or the logs and see if there are any warning signs popping up? What happens if you keep those multiple scenarios running for an extended period of time? What happens when OTHER scenarios get introduced into the middle of this?

What happens when something "impossible to happen" is introduced? Because all those things that "no real user/customer/person" would do? Oh yeah. They'll do them They'll do stuff because it makes their life easier - or they think it will - until they do it and stuff hits the fan and goes EVERY where. Then their life will get very unhappy.

Then they'll look at you for why you screwed things up.

Then you'll need Mr Wolfe to help clean up the problem.

Better you see the problem in advance and cut down those calls to Mr Wolfe.

I finished my coffee. The Unicorn finished his and we wished each other a Happy Thanksgiving and went our separate ways.