Sunday, December 3, 2017

End to End Testing and Integration Testing Revisited

After a crazy busy "black Friday" at work,  I was hiding at the small local coffee shop enjoying a really good cup of coffee. It was really very good coffee. Very nice.

I was half way though my first cup (I limit myself to 2 at this shop) looking out the window, glad I was not part of the mob scene of young people debating going to the local bar, the fancy coffee shop or replenishing the supplies from the "party" store next door. (There were loads of parties Wednesday night and Friday in the area.)

A guy I sort of knew from the local test meet up pulled up a seat and began talking about how awesome things were at work. He was really really proud and needed to tell someone about it. Their End-to-End Test Automation had been up and running for 3 releases and it was AWESOME! They made some tweaks and it ran clean and fast and all you had to do was click "Start" and it ran.

OK - Sounds pretty cool so far.

It has 300 steps - well, closer to 400  but, you know... (More coffee for me, this might be a long conversation.) And the first few times it ran it found a bunch of bugs and they could get them all fixed and now it is running really really clean!

Wow - OK, that sounds line something. So I ask the next question that seemed logical to ask...

So, this has been running for 3 releases? You found a bunch of bugs in the test environment early on, right? (Yeah, we did - saved our butts.) Cool. That's good. So, how about bugs in the  production environment? Got any cropping up there?

Well, there are always bugs in production, right? Something always goes wrong and there really isn't anything we can do to prevent them in advance. You know how it is. There's one the keeps cropping up. Seems like this reappears every few weeks. We just fix the data and move on.

Hmmm. So, is there a way to recreate the scenarios in the test environment and see if you can head off these issues you had in production? Is there a way to maybe isolate why you  need to "fix" data every few weeks?

Well, it's really complicated. We don't control that part of the data. Another team does. They claim their stuff works fine and the problem is with our stuff, so it is our problem.

Ouch. That kind of sucks. (Thinking about breaking my rule and going for a third cup of this really good coffee.) Ummm, what about integration testing? Is that a thing for you? or, not really?

Well, all the teams are testing their stuff in the System Integration Test environment we have. But we never seem to find these kinds of bugs there.

Hmmm. Yeah. That can be frustrating (wishing I was drinking a nice red wine at this point.) What would it take for that to happen?

We can't do that. We're Agile. We don't do that kind of thing. But hey - I see my wife outside - Gotta go! Later - See ya next month maybe at the meeting!

I sigh. realize I'm half way through my third cup of coffee and I hear my friend the Unicorn start laughing. YEAH - the Unicorn! I had not seen him in MONTHS.

He wasn't there when you talked about "Pulp Fiction Integration Testing" was he. Too bad.

(I blogged about it a long time ago here.)

So the Unicorn and I chatted for a while.

The problem we both have with much of the "automated end-to-end testing" stuff is that thinking humans tend to stop thinking and allow things to run on auto-pilot. If everything is green at the end there are no problems and anything that comes up in production was something that could not have EVER been anticipated.

Except often times they CAN be anticipated. Running tests in an "environment" does not make them "integration" tests. It does not mean they are telling you anything of value.

I might suggest this idea - try getting other teams involved - Other people who deal with systems or applications that interact with yours. Try comparing notes. Try setting up scenarios emulating what actually happens when the software is released to the wild.

Can you evaluate the touch points? Can you monitor the database or the logs and see if there are any warning signs popping up? What happens if you keep those multiple scenarios running for an extended period of time? What happens when OTHER scenarios get introduced into the middle of this?

What happens when something "impossible to happen" is introduced? Because all those things that "no real user/customer/person" would do? Oh yeah. They'll do them They'll do stuff because it makes their life easier - or they think it will - until they do it and stuff hits the fan and goes EVERY where. Then their life will get very unhappy.

Then they'll look at you for why you screwed things up.

Then you'll need Mr Wolfe to help clean up the problem.

Better you see the problem in advance and cut down those calls to Mr Wolfe.

I finished my coffee. The Unicorn finished his and we wished each other a Happy Thanksgiving and went our separate ways.

Sunday, November 26, 2017

The Brightest Software Witch of Her Age

Folks who know me know my geek-interests are broad and varied. That is part of why I was intrigued when I saw a conference track talk at Agile Testing Days with a tie in to Harry Potter. So I grabbed my Hogwarts alumni lapel pin and Gryffindor house tie and made a point of packing said items for the conference.

If you read the Harry Potter books or watch the movies, you may remember toward the end more folks than Professor McGonagall, Ron and Harry realize just how bright Hermione is. She is referred to, to her face, as "the brightest witch of your age." Hermione is smart, intelligent, wise beyond her years, resourceful, dedicated, passionate and absolutely focused.

She represents all that is best in the world of wizardry and witchcraft. (Remarkable in that she is Muggle-born - neither of her parents had magical abilities.) Indeed, she represents the very future of that same world.

Last week, I believe I met the future of software at Agile Testing Days.

I met three bright, intelligent women in Potsdam. Each have their own strengths and characteristics. Still, each demonstrates the traits that make Hermione, well, Hermione.

Let me introduce you to them -

First, the person mentioned above with the link to Harry - Marianne Duijst.

Marianne is one of the plethora of talented testers from the Netherlands I have met over the years at Agile Testing Days. Of these three, Marianne is the likely candidate for "muggle-born" in that she came to software through a round-about route. (Don't get me wrong, I have known many talented software people who are muggle-born.) She did MA level studies in English Language and Culture with an emphasis in modern literature and theatre.

Now, THAT is some tech-heavy academic credentials (says the guy with enough university level credits in music and history studies to have majored in either...) Really. She taught for a time, then moved into software - development then testing. She also does some stuff as a Scrum master and coach.

My conversations with her reveled an ability to connect ideas and thoughts in unusual ways. Not just odd, "oh, liberal arts major" ways - where she's more butterfly than focused professional. Not at all. These were of the nature where an old guy, reasonably well read, was gobsmacked by how she was making connections from hugely diverse literary forms and works, and drawing connections to working with software.

The session she gave was outstanding. I believe she will go very far in shaping the world around her and making it better for all.  Find Marianne's website here: http://marianneduijst.nl/   - she can be found on twitter here: https://twitter.com/@marianneduijst
**
Cassandra Leung I met online while doing a podcast. I was excited to meet her in person at the conference and found her to be as personable and intelligent as my first impressions of her made her out to be.

Cassandra is from Glasgow, Scotland, currently working in Germany. She moved into software testing after doing a bit of a stint as an IT recruiter. We had some fun and informative chats on twitter before the conference.

The conversations and discussions with Cassandra at Agile Testing Days were more of the same. Practical, applicable ideas in place of the hand-wavy crap so often encountered of the "well, it depends, doesn't it?" ilk.

One key point I observed, at the start of her talk, Cassandra said something to the effect that she was not "an expert" in test automation - then proceeded to explain ideas and concepts more clearly that many self-proclaimed experts I've heard. She carries herself with an air of confidence that is remarkable.

How remarkable? Cassandra made and placed piles of "tester bingo" game sheets - pages with the names and pictures of speakers and other conference attendees, to encourage people to meet and start conversations. How cool is that?

If I were to mention anything negative, she claims she does not like coffee. I'm a little baffled. I thought coffee fueled most test efforts.

Find Cassandra at her website - http://www.cassandrahl.com/ or on twitter at https://twitter.com/@Tweet_Cassandrahttps://twitter.com/@Tweet_Cassandra
**
I met Sabina Simons  the first time at a conference a few years ago. When I saw her at ATD, I had this strange sense of having met before, but could not place it. That turned out to be a minor detail.

Sabina impressed me with her solid, straight-on approach.Originally from Quebec, she lives in Kitchener, Ontario and works at D2L She has moved from Quality Analyst and tester, to development, to Test Strategist, and was recently promoted to Development Manager.

Our conversations about software, problems with software, problems with roles and, frankly, life, were full of profound consideration - something I often associate with people who have "looked into the abyss" and considered their own purpose steadily.

Her presentation was solid - here's what we did, these things worked and these were the mistakes and unexpected consequences we encountered,,, Here's how it was handled. As with other people who are leaders, she shouldered the burden of the errors to allow the team to move on to do better work.

I've met very few new managers/team-leads with an attitude like that. I was impressed.

I'm not even jealous that she did one of the last PSLs that Jerry Weinberg hosted. That many of her lessons contain "Weinberg-isms" is no surprise. What is notable is how she has taken them on - internalized them - and not simply spouting them unthinkingly as so many do.

Find Sabina on twitter here: https://twitter.com/@sabina_s_simons
***
These three women each embody different aspects of the strengths and capabilities of Hermione Granger.

When I first got into software, I had visions of changing the world. I suspect these three just might do it.

The brightest witch of her age, indeed.

Tuesday, November 21, 2017

Considering Agile Testing Days 2017

I'm sitting in my favorite rocker with my feet up, a glass of very drinkable red wine, the dog worn out on the floor by the footstool thinking about what a week last week was.

I was in Potsdam, Germany for the 9th Agile Testing Days. I have been there for six years. This is astounding to me. I have seen the conference grow and develop into something... amazing.

Unicorns are a big deal with this conference now. I was at the conference my first year when unicorns became a thing - and speakers went out of their way to add unicorn images to their slide decks. I was one of them. I THINK I had the only image of a dead unicorn at the conference that year... ok, yeah.

My first year - I was bowled over. I met so many amazing people I had followed on-line, on twitter, reading blog posts, reading their articles, sometimes their books. And you know what I found out? They liked having a good time and meeting people and talking with people and hanging out and liking the same things everyone else did. I spent an amazing evening sitting with Lisa Crispin, Janet Gregory,  their husbands, my wife and younger daughter at the Speaker's Dinner. and we talked about all sorts of astoundingly normal things.

But - these were the authors of Agile Testing! THE BOOK people turn to - and then there was Huib Schoots and loads of other people I stumbled into and met and got to know. And they were good people who worked hard and shared their ideas openly with people  they had just met. We compared notes and told stories that had lessons and -

Then there were the organizers - Jose Diaz - whose crew did a fantastic job of EVERYTHING. Jose demonstrated what a leader does - all the credit goes to the crew. Any problems were on him.

There were people willing to think and question and think on the responses and consider and talk about it over drinks long after the sessions had ended. And this continued unto the wee hours.

And you know what? This has not changed - it has grown. It has matured. This year, there were a bunch of people there I had no recollection of meeting before. There were people I know were there in the previous years, but I simply had not met. At the same time, there were people I had met and spent excellent time with in previous years - that I simply never saw this year.

Having said that - there were just under 700 people at the conference this year. That is an astounding number. I'm not sure how many countries they came from, but I know the speakers alone came from every inhabited continent in the world. Yeah, it has become a global thing.

The word used a lot last week was "community."

I know I am not the brightest star in the universe of software testers. I work hard to contribute however I can, and encourage people I meet who are clearly above average. I firmly believe they will achieve great things and do things in ways people like me never imagined possible.

Last week I felt extremely privileged to spend time in conversation with Masters and Colleagues and Rising Stars of this craft we call Software Testing. Among them - George Dinwiddie - Yeah - GEORGE FUCKING DINWIDDIE - Many excellent conversations, ideas from his presentation and thoughts on a huge range of ideas. Ray Arell - Holy CRAP! How can you not know that name - go look him up. He's awesome. Alex Schladebeck - if you have not followed her work, you really need to. She is astounding. (I'm running out of superlatives...)

My friends - I mean that deeply and surely - Chris George - he's a good man and a fantastic tester. Huib, whom I mentioned before. ANOTHER good man - he works hard, plays hard and gives excellent support and insights to people who take the time to listen and share ideas. He really is a good man, a good coach and advocate for people wishing to do good testing work. Perhaps this is why he won this year's Most Influential Agile Testing Professional Person (MIATPP) award.

Of course - Tony Bruce. I met Tony at a workshop quite a few years ago - I knew who he was before - but the meeting was exceptional. My first evening, while a "confusion" with my room was being sorted - Tony stuck his head out from the hotel pub and called me in for a beer. "Pete! Let me stand you a pint!" After a very long day of travel and frustration at the end, he reminded me why I like this conference so much - the people.

Not just the speakers and the sessions, though these tend to be excellent, but the people in attendance. The people are what makes this conference special to me.The individual events - that things that stand out to many people were listed in my daily "Live From..." postings, not just this year, but pretty much every year I've been there. Year after year it is the small things that I don't write on directly that leave the deepest impression.

The number of people I ran into from previous years hoping to have conversation with this year were many. Alas - many were brief in the hallway or between sessions. Some folks I had intended to seek out, and never saw them at all - a sign of the growth of the conference I think. Others, I had gotten messages about meeting them - looking them up - serendipity smacked me about the head more than once and these conversations happened - sometimes briefly and sometimes in depth.

Toyer Mamoojee was as awesome as I was told. Claudio Perrone gave an awesome keynote - and was as personable in the hall (literally) as he was on the stage. Cassandra Leung had a powerful story to tell, then a more powerful story over lunch. Angie Jones - I enjoyed dinner the last night with her and some others. Matt Heusser - colleague, collaborator on many projects, enjoyed our brief chats and of course, dinner Friday. Anastasia Chicu - a delightful, exuberant person with much to tell the world. Ash Coleman - my partner in crime last year making videos - how delightful to see her again when she had such a strong message to give. Deb Hartmann - whom I only brifely saw - in an open space my very fiest day - this woman sits down and I'm thinking "Is that her?" and it WAS! Bart Knaack - David Evans - Yeah DAVID EVANS - Good guys - had a lovely breakfast conversation with David. What a n excellent way to start a day at a conference.

I know I am missing people. I kept seeing Fanny Pittack across the room. Gitte Klitgaard - who always puts people at ease simply by being her. Guna Petrova - if I had her level of energy, look out world! Meike Mertsch - the hug giver, and also excellent sounding board for ideas.

Sabina Simons, from Ontario (originally Quebec) - brilliant thinker. Signs of growing into an excellent leader. Mike Sutton - Yeah - HIM - a good, trusty man. Of course, Marianne Duijst - another fantastic thinker and excellent idea person. Rodrigo Cursino, whom I only saw across the room. Susan Bligh - wonderful, delightful person. Trinidad Schmidle - very, very nice. Viktorija Manevska, who was at the same tale for the Speakers Dinner with me - where we met and continued to chat through out the conference.

The PILES of people I did not chat with or more than a few words - Selena Delesie, Samantha Laing, Karen Greaves - and the list goes on. So, yeah - the people make this conference what it is....

==

Last year, I was horrified to hear people I respected tell stories of abuse in the work place and conferences. That was my last night at ATD before heading home. I was filled with a mixture of revulsion and rage Eventually that led to this post in February, 2017 (after my daughter read the 1st draft and said "You're screaming. People won't listen to you scream." She must get her good sense from her mother... http://bit.ly/2l0WkAy

This year, I heard and was in moving conversations of where there was a single common thread - people in positions of power took advantage of that to get what they wanted and inflict harm on others - workplace, conferences, school, home. The correct word in each of these cases is "criminal activity." No, my screaming isn't ended - it is simply toned into a very hot fire.

I had conversations with people who made a point of always seeing a bright future, no matter what had happened before. I saw people open their wallet and look in, then pull all the bills/notes they had and put them in the collection box for #saveLinnea. (https://www.savinglinnea.com/)There's always more money - they print more every day. Giving something as common as money to help save someone as uncommon as she is, and her parents are, seems only right.


My take? Some people are right proper asshats. Other people are good, well intended folks who sometimes make mistakes. Yeah, there is a difference.
 
**

And the ideas. I heard people express concepts and ideas in ways I had not considered before. This helped me see a different line of thinking and a different way of carrying on the same information.

I saw people get up and give their first presentation at a conference (some solo, some 1st time EVER) and be absolutely terrified - and still get the message delivered despite shaking voice at times, and knees perhaps a bit more wobbly than they would like to admit. And they were fantastic.

I saw people get up and tell terribly personal stories about themselves and their hopes and dreams for the future.

The spirit of community, the courageous energy and giving hearts and the desire to improve, individually and each other are what set this event apart from other conferences I have attended.

This year's conference was astounding. I can hardly wait to see what next year's - the 10th ATD will look like.

With luck I will see you all there - and manage to make the time to have a conversation with every person.

Friday, November 17, 2017

Live from Potsdam, Day 4 - Agile Testing Days 2017

Friday, 17 November came and I realized I had a full night's sleep!

With every intention of attending the "late keynote" and "Women & Allies in Agile" sessions last night, I sat down for quiet time - and woke up at 9:30 PM, and I decided that it meant it was time to go to bed - so I did - and overslept. Missed lean coffee but had a wonderful breakfast chat with David Evans and  Marianne Duijst

Huib & Stuart are announcing the auction of the each sheet of sketchnotes for day's keynotes. Proceeds will go toward the fund me for Linnea Nordström, the 6 year old daughter of Kristoffer Nordström (a fantastic tester and many time speaker here at ATD) -  who is suffering from brain cancer who is going through an experimental cancer treatment that is not covered by insurance. https://www.savinglinnea.com/

**
Roya Mahoob is presenting "Computer Technology as Agent of Change for Women in Developing Countries." Wait, what?  In 2013 she was named as one of the 100 most influential women in the world. I should mention that Roya is involved in leading the Afghan Girls Robotics Team.
Photo by Sabina Simons, used with permission

In 2010 she formed Afghan Citadel Software - yeah, a female computer entrepreneur -  from Afghanistan. In 2013 she moved to New York because of threats. I should mention that Roya is involved in leading the Afghan Girls Robotics Team.

"Through technology, we are reshaping how entire communities view women."

When she was young, there was an open field across the stret from her house. After the Taliban took over the country, the Taliban gather all the books from the town she lived in that were not Islamic or "approved" in that field and burned them. In her words, her view of the outside world went to ash.

The image of the fire stayed with her and instead of destruction, "The Taliban loved the darkness. Ideas that were not theirs were forbidden." The fire that remained with her though, was the fire of her imagination.

When the first internet cafe opened, her brothers were allowed to go in, girls were not. "I was told that the cafe had this box-machine that connected you to the world."

One day, she made a point of going in very early and was amazed. She saw the computers and saw the world could be opened to her through technology and the internet. 

Becoming the first female tech CEO in Afghanistan was nothing but challenges. Corruption, interference by religious conservatives, interference by social conservatives, all were obstacles to any form of success (ahem - Important lesson here for the world!)

"I believe deep inside all of us is a desire to change the world."

She was asked to arrange/sponsor a Afghan robotics team made up of students to compete at an international robotics competition in Washington DC. There were many applicants for the team, but only 6 who had family permission to travel to the US and had any idea what robotics involved. After being rejected for Visas twice (TWICE!) by the US, until Congressional and White House pressure pushed them through the Visa process - they came and won a silver medal.

They demonstrated that with support and education, even from countries considered non-technology centers, girls have as much potential as any boys.

Girls everywhere deserve the rights to select their own future and deserve their own voice - as well as access to the same education and technology many in the world believe should be  reserved


Jose is giving tokens to each of the girls.

Dear Lord - one of them has the guts to address the conference. My heart is breaking at the story. Notes on this later. (Nope - not posting the notes. sorry. )Not just the suffering inflicted in their own country, but their treatment by certain governments (ahem) because of where they were from and their religion.

"My parents support me and because they support me I can be here."

Her father was killed by Daesh for supporting her education and trip to the USA.

And a standing ovation.

OK - I left out that they are building a tech school. I'll post a link to more info when I find it.

I'm a little overwhelmed by this and taking the time to assemble thoughts and clean up blog posts. Sorry, Matt, we'll talk later.

**
**

In George Dinwiddie's talk on improving gherkin.
 
Major point, in BDD & TDD, "I don't write automation to detect bugs, I write them to prevent them." This is a sense that some people expect things from software development to be simple. Yet the point of the 3 Amigos is to help determine not just what was needed, but hwat really was possible.

The point of BDD is to do 3 things - Elicit/discover requirements is part of it the rest? well...

If you are writing tests the same way you are doing it manually, consider you are doing it wrong.

If you start out with THIS -
Why not do THIS instead?
Cut it into pieces - and get the result Liz pointed out (ahem)
 

Helpful hint from Liz Keogh
Liz Keogh reacting to her "hint" being in the slide deck
Instead of detailed... stuff - distill to the essence of what you are trying to get to. Because - keep it simple. Don't worry about the UI - Worry that the message is the same - for example - Does the kitchen timer indicate when the time is up?

"Every word in a test should be directly relevant to its purpose" - Marick
"Tests should contain every detail that matters, and no detail that doesn't matter." - Emery

"I blame the British (for US companies tendency to not reimburse for alcohol at meals). Australia got the criminals, we got the Puritans. Someone else having fun should not be allowed."

Focus on the rules - focus on the intent - Look at the interesting cases that MIGHT have something of value FOR them. REALLY!


**
Very good lunch! Conference attendees are dwindling - fewer & fewer.
Alas.

Jose is giving some announcements - he seems really tired - I think we ALL are!
And they are auctioning the sketch art murals of the keynotes - 1st one - started at 100 euros and ... sold at 225. They just passed 200 Euro for the 2nd... and that ended at 250.

The third is at 175 and going... up...

And NOW -
2nd keynote of the day...

Maaret Pyhajarvi is giving us Leaning with Osmosis.

Maaret won the MIATPP award last year.  She is presenting on some ideas that seem inline with much of this conference. She has been constantly growing and changing herself and how she presents information. She describes herself as a polyglot feedback fairy and a polyglot programmer.

Osmosis is movement through semi-permeable membrane without recognizable amounts of energy being spent.

While in school, she worked very hard to be prepared for class. She was regularly called on in her Swedish class to translate passes from Swedish. While her classmates were amazed that she could translate it, they did not know that she spend a great deal og time preparing to be able to translate the passages.

For a long time, she was the only tester by profession - and the only woman on the team. She was the only one to get excited about bugs. She was also the only one who was not really excited that the developers were able to do... something... with only, say, 3 classes.

And then there was this comment -"Women only write comments in code."

ummm - right,

Testing is generally a good environment, given the gender balance among the testing community.

Still, things changed...

Her daughter came to school age. And Mareetp bought her class a book on programming. And - SOMEhow, her friends all liked doing programming - so she tried it to. And she LIKES it. THEN - they tried pair programming - partly because they were doing it naturally.

Then she met Woody Zuill and the idea of mob programming - where 1 person is typing and loads of smart people are working with them. At first, she rather rejected it - because - well - the team did not collaborate. very well. The team rejected the idea - "it's just silly." and other responses.

But trying it at meetups and with random people helped.

Finally, she asked a co-worker to try it with her, so she could learn. THey went from typing letter by letter to moving forward fairly freely - she had seen the developers doing things and had a strong sense of what was happening and what developers did on a regular basis, given the situation...

Soon, she found that she could try things and it would be OK. THEN! There was less of yours and mine attitude, and it became an "OURS" attitude.

Then  - the began finding something interesting - Correcting things on the spot, by the mob finding things - removes ego from the environmant. It also helped learn what was relevant. While she slowed them down by inexperience, the slower pace allowed for more thoughtful thinking.AND she was able to do some exploration during these sessions

Programming is like writing - Getting started is easy and it takes a lifetime to get good at.

The point of mob programming is that you can get the best out of your team - and learn from each other all at the same time - and everyone takes on ownership

(and a breakdown of mob programing roles - driver, navigator, etc.,)
HUGE POINT - Best ideas win over credit!

YES!

Learning in a mob is a way of moving toward learning things you did not expect ot learn.
Mob techniques can work in testing - ET, Automation, Unit, Perf & Security - which makes perfect send (to me) Mareet explains how she learned about programming from mobbing with her team, and they learned about testing by mobbing with her.

Very, very good.

Discomfort can make you learn - the idea that doing something that made you uncomfortable might mean that you can learn something new - or  relearn...

***
OK, a word of apology. My battery was completely flatlined after that last sentence. Shutdown Now. I need to go back and check photos, etc., I missed getting the last of Mareetp's talk - I have some photos and tweet's I'll try and get in place to rebuild it, but...

Big Takeaway from mob programming (and testing) from a former intern brought on full time after a short period of pairing and mobbing:
And a BOOK! Leanpub - check it out.
The battery was SO drained, I could not get Janet Gregory's closing talk either. GOOD THING THE PHONE HAD POWER! Someone tweet-stormed her talk! I'll be incorporating that in here later.

In the mean time, you might get a sense of it here:
https://twitter.com/PeteWalen/status/931541792692072449

**
The FINAL sketchnote page was brought up for auction after Janet's Keynote. All of these are magnificent tributes to the skill of the speakers and the craft the artist Stuart Young- find him on twitter @Stuartliveart. Bidding atarted at 100 Euros and went up, stalling around 250. Until Jose Diaz very gently said into his mic "1000 Euros" - pandemonium wasn't in it.

All in all some 2700 Euros (I saw some figures of 3000+, not sure which is accurate) was raised through Agile Testing Days for #SavingLinnea.

We broke up into groups, some heading home, some to dinner before heading out the next day. Conversation at dinner was long and excited (much to the surprise of other patrons in the restaraunt - maybe that is why they had us in a room by ourselves.
Yes, I AM smiling!
***

In the meantime. I had a number of conversations that last day and evening.Dinner with friends, return to the Dorint to pack up and return to the pub for the farewell drink - the parting glass as it were.


**
I'm sitting in the airport in Dublin at the moment finishing a bottle of porter going over pictures, notes and smiling at the memories. Pulling things together for a recap later in a new blog post.

Thursday, November 16, 2017

Live from Potsdam, Day 3 - Agile Testing Days 2017

Morning came at the usual time in Potsdam on November 16, however many of the conference attendees missed its arrival as they had been having much fun following the Agile Games and ATD Cabaret. Much singing and music and jokes and fun was had, and SOME (ahem) were feeling the effects a bit more than others....

Still, that is one of the aspects of this conference, and many conferences. People getting to know each other, swapping stories, asking questions, sharing ideas and sometimes sharing advice.

One aspect of this, beyond the evening events and other random conversations that grow organically at events like this, is Lean Coffee (introduced at ATD by Lisa Crispin & Janet Gregory). I mentioned this in the Day 1 blog and today, after inhaling a quick breakfast (someone was up later than intended... ahem) I wandered in to see how Lean Coffee was going, and was pulled into moderating a table. OK! Dive in head first!

What is Lean Coffee? It is a time blocked method of discussing ideas, problems or exploring questions where people bring topics, vote on them, then discuss them until a resolution is reached OR the energy for the topic has been expended. The idea is to share ideas, investigate things the original person had not considered and come to a take away.

**

I've moved into the main hall for the morning events. Jose is going over "lost & found" items (including someone's tablet - THAT will be missed!) Ash Coleman is talking briefly about the events later today around inclusion and diversity. AND we're getting ready for this morning's keynote, The Pothole of Automating Too Much with Paul Holland.

Paul is kicking off his message that (for the opening aspects at least) is challenging the conventional wisdom held b y many: Automated tests are not likely going to give you all the benefits that many people believe it will.


Automated scripts will only look for the things it is programmed to look for. It takes a vigilant human to find other things. He uses a metaphor around the roads of the United States for software.There are so many raods (paths) we can't really "test everything.


Right.

Problems - 
There are more kinds of problems that an automation can be programmed to recognize...
  -  Vigilant testers can observe and evaluate a very large variety of outputs and also vary the inputs let us find more problems than we can predict.

There are more checks to automate than can possibly be written;

Some things are too difficult to automate effectively;
  -  Complex pass/fail algorithms;
  - perhaps it can be dome more quickly by a human;

Investigating reported failures takes a long time;

Automation is expensive to build and maintain;
  - High cost to value ratio;
  - Sunk cost problem (so much spent so far, abandoning it is hard)

What about -
A strategic mixed approach:
  - Automate critical paths;
  - Automate paths with the highest use;
  - Do NOT write automation for all failures found in the field;
  - Consider the cost of automation vs benefit -
     Difficulty to create;
     Difficulty to maintain (frequency of changes)
     Difficulty to analyze failures;
  - Augment automation by performing human testing
  - Automation is excellent at showing that the code CAN work;

**

Grabbed some lozenges, and a coffee and headed into Toyer Mamoojee's session on "Coexistence of Legacy and Cutting Edge Technology Systems". AND - the conference staff bring in a birthday cake for him! Yeah. these folks are awesome.


I met Toyer Monday evening, after having been told I needed to find him and have a chat - and we did! Bright tester from Cape Town, South Africa. I am really looking forward to this.


Here we go -

For this discussion Toyer is defining legacy systems as:
Business Critical;

Lack of support;
Skills shortage;

... and a couple others I was too slow to type...

so why keep them?
Customized over time to the Org;
Change could negatively impact business

Their environment looks a bit like this:


Problems?
Oh yeah...

Deployments are manual (very manual);
Achitecture - Too much logic build into legacy systems to easily update/decouple;
Processes - Slow running processes, including tewsts;
Delivery Process - Some teams less Agile than others...
Skills - Different systems required different skills.


Solutions -

Their approach (over time and not this neatly - Pete comment - they never are...)

Deployments - Automate, automate & Automate;
Achitecturally - Built APIs, Micro-services, Rewrite complex batch processes;
Processes - Faster Running processes, quicker test execution;
Delivery Process - Functionality, Freeze, E2E meetings, SOS (Scrum Of Scrums)

Skills - Internal vs External recruitment (right skills for the right system)

Testing -

Automated Testing -
  Automate what you can with commercial tools for legacy;
  Open Source for Cutting Edge/newer tech
  Push to get CI/CD;
  Automate at different levels!

E2E Cross System -
  Centralized E2E testing tool;
  Daily SOS meeting
  E2E Meeting before stories get developed (measure the impact on teams)

So, what was the result?
Originally, they had Excel for test case tracking  & QTP.
and now...



And the result?
Cross skilled workforce;
Everyone engaged, relevant & up-to-date;
Get the right people on the right system;
Maintain/expand buisness competitive edge with less risk;
Faster delivery time - monthly releases instead of quarterly

**

Some good chats, dealt with some minot work items AND slide into "An Alien Visiting MY Office" with Viktorija Manevska. (a little late... sorry)

Starts a new job - AWESOME! And she hears they are going to do Scum - AWESOME! And she learns how they are doing it - AWESo... uhm

So, she went and took on the task of reading and learning about Scrum. She found the typica; "THIS WILL SAVE THE WORLD" articles and then some that were less than enthiusiastic.
So, she looks at what is around there. And how people look at things -
So, since she knew that things weren't quite right - so she decided to get a certification (cultural thing - certifications lend greater authority - your mileage may vary).
Except, things still weren't quite right. Stuff wasn't happening the way it seemed like they were supposed to be.

Except - Scrum is not a problem fixing framework, It is a problem FINDING framework.

Key question...
Is the value of the product developed by Scrum?  -  Hmmmmmmm

Except Scrum is a bit like the blind men and the elephant...You see scrum based on how it is exposed to you. Hmmmmm.

So, if we know what the customer wants and we have some ideas on how to make this happen, does the development method/model really matter?

So, if we focus on the end result, what can happen? Working together, not worrying about the ritual forms, what if we share ideas and collaborate and not worry about what to call it? Instead of doing a daily stand-up, particularly when people are all working in the same room, what happens if we just do stuff?

THEN she lands a new position (same company) and finds out that.... not so happy. The rainboes & unicorn thing was more a tangled mess. She began looking at items, making mind maps and researching the intent of the such things and.... "that is not your job - you are a tester."

Quotes Paul Feyerabend - "The only principle that does not inhibit progress is: Anything goes."

So, she let them go and do their thing... Allowing people to fail, then supporting them when the problems arise can help them begin to change their minds from fixed positions to a more flexible one. By working with them to find shared spaces - they began coming to her to find potential problems before deployment, then pairing then... everywhere.

By looking at things from an external viewpoint, like an alien, she was able to see problems, then share the means to help people see things the same way.


**

LUNCH!

**
Slight change in plans. After an amazing chat with Cassandra Leung over lunch, have I mentioned how remarkably bright she is? Right - I must rest a bit... and deal with stuff from work - yeah, day-job stuff. I hope to spend some time in the open space shortly.

See you all in a bit!

**
And Congratulations to Jose Diaz - the master behind the conference - for officially becoming a German Citizen today!
**

Sometimes taking a nap is an excellent thing. Other times, having a nice relaxing cuppa (or 3 or 4) and conversation with interesting people Can be just as good, if not better. Thanks to Rob Lambert, Alex Schladebeck and Chris George for giving me much to think about.

Open Space is an excellent way to explore ideas and have a chat with people about items you may not have considered.
**
Sliding into the LAST track talk of the day (refreshed and ready to go!)
Our Journey to Reduce Manual Regression Tests - presented by Thomas Fend and Trinidad Schmidle.

They work at Bachmann Electronics, a company with a broad product portfolio, long lived products, a development team of around 40 developers and 12 testers. The majority of their software products are released at the same time, typically once a year.

There is a fixed issue verification system that tends to go straight to the tester. The were massive numbers of bugs being reported, however, when the development team reported a as "fixed" it went straight to the test team, not to the person who reported or raised the bug.

By changing that loopm so the person who raised the bug had first crack at the fix, or at the lease, had the change explained to them, they were able to streamline the process and reduce the amount of rework. THIS lead to more time to work on more new features.

Now, with a common acceptance criteria definition, the team & product manager are in agreement as to what needs to be done.

Then there was the "feature veto" issue - Features are completed very lat with little time for thorough testing. When everything else seems to be better, the tester might raise bugs, get very unpopular and get told to leave them alone.

By getting testers involved early in the process, this has been reduced.

Then there was automation. Yeah. Some tests are executed each sprint. The BIG ones are run after feature freeze - once a year. Not all the tests are fully automated - there would be manual configuration before starting some tests. Because, well.... yeah.

Each sprint, testers present test results in the test department. As an incentive, the number of tests being run each product cycle, through automation.  Result was increasing to around 65% of the tests.

This helped improve stability, helped find bugs when there was time to fix them, and help find and fix "blocker" bugs.

There was limited time to make regression test really happen. So, they pulled in developers to help. This was PL, but...

By working together, working on some basic action - the time to regression test dropped dramatically. They also reduced distractions during "test weeks:, no meetings, etc.,

By introducing a mix of static and dynamic code analysis to check out the changes that have been made, they were able to restrict the interruptions, limiting meetings, encouraging test pairing, etc., They were able to reduce the amount of effort to do regression tests and do them far more efficiently.

Conclusion - Through communication, teamwork, detailed planning and focusing on things needed RIGHT NOW, they were able to make this improvement where they can run the full suite in under 2 weeks!

***

And NOW the lights come on in the room. It is REALLY hard to see to type with no lights (ahem)...

Right - something else is coming up, - AH yes - the closing keynote of the day with Liz Keogh.

**
Jose is back up with a few announcements and introducing Liz...

Liz Keogh will be speaking on "How to Tell People The Failed and Make Them Feel Great."

And asserts that Failure is Essential. Whilst on a project, a couple of developers had tried to install a tool to help do testing. It took a couple of days and they gave up. And got yelled at. And new rule was implemented that NO one did anything without getting permission from the BAs. Except - they made that rule for EVERYTHING.  All creativity ceased. And it became a far less fun place to work.

Enter Cynefin - (the new version)



Obvious Domain - sense-categorise-respond ; Best Practice(s) -
Complicated Domain - sense-analyse-respond ; Good Practices ;
Chaos (Chaotic) Domain - act-sense-respond ; No effective constraint ; Novel Practices
Complex Domain -probe-sense-respond ; Enabling constraints ; Emergent Practices ;

Dan North - deliberate discoveries
Unknown unknowns - 2nd order unknowns

You failed - but we expected to because... and that's OK...

We want to help people improve. Before we go nuts with this, consider what do you like about the person - how about respect them? What is it that anchors the relationship? What is it that  you value them for.

There's the sandwich thing - where you put "the problem" between 2 good things. That's ok - sometimes...

But why - if they screwed up, they probably know it. Let them know that you know, and that they have succeeded in the past and will be awesome again;.

Radical Candor/Care - come out and tell people something is wrong, but come from a place of care. Let them know the person is valuable and valued, and they can make some small corrective action and be awesome. Because SOMETIMES not doing that will lead to much bigger failures.

Failure only "hurts" because of the impact. Safety lines can help - and can help people explore safely. Always work toward buying time - making time - by avoiding the trap of "running out of time" - allow yourself options, just in case.

Remember that "root cause" is misleading. There are usually (often) multiple contributing issues. Sometimes they may not be obvious. What can these things be?

Try scattering around ideas. They MIGHT just land and be able to be acted on immediately. OR, they may lay dormant for a long time, until the context changes. THEN it can take root and grow.

Etsy - home of the blameless review. This is an interesting idea - let's see - often times this gets translated as "the testers screwed up." One thing in the "blameless reviews" - or "learning" reviews - instead of  "who did what..." try "what happened" and "when were decisions made."

Take the blame out and look at the steps and decisions made. That might show opportunities to isolate the issue and act as a fail-safe. You will always miss SOMETHING - so mitigate the risk.

Most organizations struggle with Agile stuff, because stuff is hard. Changing directions can be a challenge. Supporting the ability to change direction is a challenge.

-- The best way to tell someone they failed is to not even mention the failure.
Come from a place of care. Anchor what is valuable. Show what is possible - the bright future ahead. Work on building safety nets.

All of this is part of solid, positive growth... unless England is playing in the football match

**

There are a couple of other items coming up this evening, but first, supper!

**










Wednesday, November 15, 2017

Live from Potsdam, Day 2 - Agile Testing Days 2017

Wednesday, November 15 dawned... well, not really. It was pretty foggy when the sun rose, so I'm not sure there actually WAS a dawn. However, it did get lighter. Conference attendees and participants made their way to the morning activities of Lean Coffee, a morning run and yoga (yeah, I know I did not mention the run OR the yoga in yesterday's post - mostly because I do not do either.)

There is a full slate of activities for today - keynotes, track talks, workshops - AND! Games! TESTING GAMES! Great fun.

Jose is finishing morning announcements and the keynote from Claudio Perrone is about to start - Popcorn Flow.

Interesting title slide - with "Change is hard, make it continuous!"

Here we go!

Operational excellence is not enough, Most ideas for software ideas (and others) are really... crap. Too many times we end up with crap ideas getting delivered really, really fast - because we get told that faster is better. Innovation is key - so you get crap delivered by jet engines or crap on a stick (with spikes)

So - we want to improve, Except Improvement, real improvement, takes change. and CHANGE is a bit like Godzilla. So, don't worry about Godzilla - he's pretty big. Look at single cell organisms, they are changing ALL THE TIME.



Popcorn Flow is an anti-fragile philosophy -

Popcorn Flow Principle 1 - If change is hard, make it continuous;
Popcorn Flow Principle 2 -Everyone is entitled to their own opinion, but a shared opinion is a fact;
Popcorn Flow Principle 3 -

OK - it is too dark in here to really type... so changed seats and now I have a bit more light - maybe I can keep up

Problems & Observations - What is getting in the way of innovation (or anything else)?
To beat inertia, everyone will see problems - when multiple people share the view that something IS a problem, it goes from an opinion to being a FACT. There is a shared view of an issue that is supporting inertia and getting in the way of not delivering crap.

You can form shared observations to create and elicit options (get a bunch of people, like 3 together and explore the ideas around. Experiment - like - "We might try BDD or Code Reviews or Paired worked to maybe make things better..."
    - pull quote! If there is salt in the cake, it is already too late! We can't fix it!

We can talk and agree on experiments, look at their expectation then TRY them! What is the result?

Set an expectation - then see what happens . Not meeting the expectation is not a failure - it is a correction of the anticipation.Be prepared for that form of failure - but - learn from each one.

Fail fast is not the point - LEARN fast is the goal!

The thing about "being agile" vs "doing agile" is not something to worry about -

(I perhaps celebrated a bit too much with my friend Huib... sheesh)

If there is a continuous flow of experiments (the core of agile) then what limitations are there? Some may work really well - KEEP DOING THEM - some may not work - STOP those, try something else.

Try "Here's a problem as I see it and some ideas that might help..." with the manager - then experiment with the experiment model.

DING! Quality is something we worry about - what about the value we get from something? Like, a conference talk? Does a "high quality" talk deliver value to EVERYONE? Or is value to the customer without regard of the "quality" of the presentation - or software (ahem).

Set yourself up for learning - Do something - oberve what the results are (for you and your customers) Is there a correlation between the 2? There may be, but not always - so look for the gaps.

And he slides into an example - His (then) 5 year old son had an idea - he could sell snails!

First few experiments failed - welp - give up? What about trying something else?.So, instead of giving up after the 1st 2 tries - he launched a 3rd idea - a comic book about... snails. Launched - landing page - AND... collected 1 Euro per donation, and send updates with pdfs in the future.

First night - 24 Euro. and it just kept going up - 1 in 6 conversion ratio (that is pretty astounding)

 Don't be afraid to think big - really - like - end world poverty...

Keep thinking - keep innovating.

**
RIGHT - sat down and helped a speaker prepare for her session (SOME one changed all her slides.... last night) I know, I still have not loaded the images from the keynote - I'll get to them a bit later - I promise.

**
And HERE we go!

Wearing Hermione's Hat: Narratology for Testers - with Marianne Duijst.

Narratology is the study of stories - the narrative and other aspects of any story. (warning academic theory stuff coming)

The Fabula is the series of related events - logically or chronologically - the "plot". There there is the Story - the presentation - ANY presentation, the manifestation and colouring of the fabula - the plot.
The TEXT is the medium these are presented to the oberserver or reader -

Building blocks to consider -
The AUTHOR (the one who write the narrative) who is likely NOT the narrator - they are not the same person (usually).

So - J K Rowling presents the story, but the Narrator herself is unnamed.
We experience the actions thru the experience of the characters (Ron, Hermione and Harry) - THEY will experience things based on the different perspectives.

For example - 1st quidditch match - Harry nearly falls off his broom, etc., and we see the story thru Harry's perspectives - however Hermione sees something else.- the change in perspective.

Hermione's Role is to understand and figuring out the fabula / plot /events of the story...
HOW? She gathers and interprets information from MULTIPLE sources - multiple stories. She stores this information and retains it for use.

This is usually not obvious to people who read the story once - usually people read the story once and leave it. Multiple readings will shed more light and reveal more information as we can get past the mechanics.

This works for software as well! Code? how often do we look at code? Maybe once? What are we missing? (We never presume we miss anything - just as we don't believe we miss anything when reading a story - once.

The question is - can we work around this?

Who do we trust?
Rita Skeeter does this weird thing of talking with people - and twisting anything she gets into something else. Hermione, by observation, realizes how Rita works - and finds a way to present an alternate story to the same facts - the same detail.

She - Hermione - listens to all sources - including those that appear to be minor r insignificant - like, House Elves. She realzies that elves such as Doby and Creature have extremely powerful magic - magic stronger than hers. She also is able to retain and present this to others because she gained the understanding needed.

This can set up shifts in understanding from the various perspectives - we change the realities - based on this shifting perspectives. The change in paradigm follows this all - for example - Snape kills Dumbledore in book 6 - and is slagged as irredeemably evil - until we see the reading of Snape's memory.

We are actively NOT looking for all the pieces of the information that can be used - because they do not meet OUR narrative - what things do we hold that we are not considering deeply. Our closely held beliefs/stories /understanding can we wrong. We don't like that. It is uncomfortable.

Context drives everything - Time is part of the context - as is place - The Time Turner allows for shifts in time to allow us to see the actions Place is another context. We know what we know because of where we are.

The challenge to Harry - which Hermione takes on - is to enable others - where Hermione teaches Harry about the information needed - aqnd shows Harry, Ron and others on how they can do the same thing.

The problem is, Hermione does not always have all the information needed to make the best decisions. She builds a model based on incomplete, if not faulty, information. Just as we do with software.

For software people, we need to find ways to think differently - what happens if we do this - what can we infer from THIS. What happens if we change...

**

Lunch and many excellent conversations. More on those later -

Now it time for the 2nd keynote of the day - A Practical Guide to Testing in DevOps, being pesented by Katrina Clokie - the AWESOME Katrina Clokie.

When she was young, she had a teacher named Miss Perfect (really, that was her name) who taught the kids in the class about Atoms - and how they are everywhere and make up everything - including how they could move atoms around by moving their hands in the air - which is made up of... Atoms...

So, when DevOps began to be a thing, many testers looked at that and said "Oh, that is for Devs and Ops folks, we're not in there..." Except... well, Atoms




In Agile, we have everyone helping with testing and helping with quality.  And then we have others added in the mix of Agile to come back as .... Dev Ops - so Infrastructure, Support, Analytics - everything, no?


Citing the equally awesome Dan Ashby. his model - where testing is involved everywhere...

And STORIES!

Once upon a time.... Katrina's org was having a lot of issues. Specifically big red lights going off when builds were made - and things went wrong regularly.  Machines dying and stuff happening and conflicts and - ick.

By splitting the Selenium Grid from the Jenkins Slave, ...
Things got better - - a LOT better.

Now, while this was not a typical "DevOps item, the fact was, that Testers led the drive to make things better. Awesome!

They also test in production. Ummm - yeah - Did I mention they are a bank. Testing in production? Really? Dangerous. Not really.

Monitoring. watching - looking for behaviors that might inform us on what is actually happening.

Because - in a mature DevOps environment, the testing is all around - at every level. Because testing is not looking for bugs - it is looking to see how the organization can be informewd about the bahvior of the software - before and after it goes to production. Really.
 
And a post to the LeanPub site! YEAH!

***

Sliding over to Getting Ready for the Cloud with Sabina Simons. Sabina is a very bright, intelligent young woman from the Kitchner-Waterloo area of Ontario - practically a neighbor of mine! Only around 4 hour drive and an international border crossing away.

And we're underway - OH YEAH!

She is a Development Manager at D2L Corporation. She was given an option of 3 choices for next position in the company - 2 of which she had already done, and the third (the one this story is about "scared the crap" out of her...

What does "the cloud" mean? Welp - their background was to move to an AWS single tenant instance - fine - why?
They wanted to be able to move faster and serve customer needs more quickly and focus on getting product out to people and not the logistics of how to do that. Their customers are educational institutions at all levels, from pre-school to universities.

Can they get this to work? Locally? Globally? How does this actually WORK? What in the world does this look like?>

Change infrastructure - except they have a heap of a mess of legacy... stuff.many, many, many levels of ickiness (my word - family friendly blog)

Some 20 streams of work present, their code base was comparable to... a park with trash everywhere. No test instances, not test cases, not automation of any sort. With a mindset of getting it sorted - their goal was stated as "leaving it a little cleaner" each time they did something. Addressing immediate legacy needs by making incremental improvements.

Experiments! Canary Testing - small actionable instances, small changes to track what is being done. Leave benchmarks and small unit tests documented in the process - and set changes up and run things in the perf lab... things are awesome! Except the code path they were interested was not executed.

Complex scenarios sometimes take unusual solutions - like, how to set things up and how the conditions exist in production. Really.

Benchmarks are fine, making them drive forward. The set the throttle and exercise it.
Things worked great! Except - when they opened it up - stuff hit the fan.

So, now they have a significant problem. Loads of people pointing fingers and loads of "YOU SCREWED UP!" So, deal with the situation as it is - look at what process flow can be tweaked to improve the situation and open conversation. Stepping away from the blame game and being open in communication can actually help calm everyone down and make things better.

Home grown automation tools - (looks like some instrumental graphs she is displaying - and yeah, they were were instrumental stuff - which is awesome - I get to ask a question!)

Tracking behavior through various models - good solid reliable information. Watch what is happening and look for leaks - Really look in the corners, the places that appear to be OK - particularly when things are ... a little odd.

Use the code analyzer to track what is being used - presumptions & beliefs are grreat - BUT - go prove it.

Failure mode testing -

Always sounds great until people realize what that may mean. In short - How do you figure out how the system will behave in different "suboptimal" ways - usually catastrophic!

These start with What If? stuff - and working through it. Sabina's team worked through the issues as a team - (YEAH!) the question is always figuring out the stuff that is likely to happen.

Enter the Master of Disaster - writing alerts to handle with some level of grace things that are likely to go wrong.

Takeaways -
Lead by example. Move boldly and show actual results;
Experiment - What Happens if I Do BLAH!;
No Excuses (really) - everyone has stuff that needs to get done, make things happen;
Share your story - internally, externally - make the world a better place....

**

Right - I snuck off to a corner for a few minutes quite time, then sat with some new speakers and had a lovely chat with them on talking in public and working up the guts to stand up and bare their souls in a way that many people don't realize novice speakers do.

I then did some preparation for the Agile Games night, where I was running a table of mixed small games confusing people and offering puzzles that required them to set aside their preconceptions and immediate beliefs and look at the "call of the question" and the words actually said and written.

Great fun for everyone. I gave away many stickers for prizes and people had fun. There is some socializing going on right now that I intend to return to - with a "tester cabaret" which seems to be great fun. IN the meantime, I've had excellent conversations today and tonight which have given me much food for thought. More on that later.