Sunday, July 13, 2014

On Software Quality and Software Testing

The last week or so I have been deep, very deep, into considering the relationship between Quality of Software and Software Testing.  In this, the conversation has been more at the Meta level, something akin to ASQ's view on quality in general.  (Fair warning disclaimer, along with being a software tester I am also a member of ASQ - American Society for Quality - these folks.)

Interestingly, that relationship helps me when I challenge assertions, usually gratuitous, often fundamentally flawed, on something published by the ASQ or something Deming said or wrote.  Its interesting sometimes to lean into the table and say "Can you explain what that means?  I'm not making the connection between what you are asserting here and my understanding of what {insert quality buzzword} means.  Its possible we have a different understanding of the concept and I'd like to address that to avoid future problems and potential future conflict." 

The response often comes back citing some authority, for example, Six Sigma or some concept championed by ASQ.  Interestingly, that was recently coupled with the ideas of Context Driven Testing and AST - Association for Software Testing (Ummm, for those who don't know me, I'm a member of that, too.)  Oftentimes I will then, when it is clearly an attempt to assert a position by citing authority, say something in as non-threatening a manner as possible along the lines of "I'm a member of ASQ and of AST.  I have read the white papers and books on Six Sigma (or whatever else they are asserting, usually out of the recommended context) and I'm not sure how they align with what your are saying.  I would like to understand what you are saying better.  Can you explain it or would you prefer to have that discussion off-line, maybe over coffee?  I'll buy."

I find people will be much more open to such discussions if I buy the coffee and /or bagel to go with it. 

And yeah, I realize that I am doing my own version of citing authority by making the above statement.  It does serve to get their attention and blow away the smoke screen that is intended to be set up.  Well, maybe not so much removing the smoke screen as is bringing high-powered radar into the mix - I can see their position in spite of the smoke screen.

Where am I going with this? 

Many people I meet use the terms "testing" or "quality assurance" or "QA" interchangeably or in conjunction with each other.  You get statements like "Let me know when this has been QA'd" when they mean "tested."  Then there is "QA Testing."  Do NOT get me started on that.

The idea of "testing improves quality" is often the response to the question "Why do we test?"  The bit that gets left out, possibly because it seems obvious or maybe because people are oblivious to the idea, is that testing improves quality only if someone acts on what is learned from the testing. 

If something is not changed as a result of testing - configuration, code, processes - maybe all three - maybe other stuff as well, then will "quality" be any better?  What is the point of testing?

If people want confirmation that things "work" then by all means - run the happy-path scenarios that possibly were used for unit testing, or build confirmation testing, or maybe in the CI tools - but don't confuse this with "testing."

The point of testing is to learn something about the system or piece of software at hand.  It is usually not to prove anything.  It is rarely done to prove something is "right."  It may be done to check certain behaviors - or to see if specific scenarios behave well enough for a demonstration - or even, in a limited sense, validate some piece of functionality. 

However - if there are any variances found, the testing identifies those variances - nothing more.

Testing does not make anything better.  Testing does not improve quality - ever.

Testing provides information for someone to decide that action needs to be taken. and then someone must act on that decision.  Then the quality may improve. 

It is taking action after testing is completed that improves quality - not testing.

Saturday, July 5, 2014

On the AST and CAST and Conferring about Testing

The Association for Software Testing says this about... well, itself:

The Association for Software Testing (AST)  is an international non-profit professional association with members in over 50 countries. AST is dedicated and strives to build a testing community that views the role of testing as skilled, relevant, and essential to the production of faster, better, and less expensive software products. We value a scientific approach to developing and evaluating techniques, processes, and tools. We believe that a self-aware, self-critical attitude is essential to understanding and assessing the impact of new ideas on the practice of testing.
This is kind of a mouthful. 

I'm not going to write about that.  Well, not directly anyway.

Once upon a time I was a regular participant at SQAForums, and a newly minted "QA Lead."  I was digging for information, ideas and the like to use with my new position.  I remember threads where people posting things like"Really, there are no 'best practices'."  They sometimes went on to say something like "There may be something that works well in some circumstances or may help sometimes, but the idea of 'this is the best thing to always do' is misguided." 

This made a lot of sense to me.  One guy posted something about starting a new group for testers.  This was roughly 10 years ago. I remember thinking "that sounds fantastic, but it is obviously aimed  at people more experienced in testing than I am."  I let the chance pass by. 

Two mistakes in One.  Impressive, Pete.

Fast Forward to 2009.

I was at a conference in Toronto, sitting at breakfast with a group of people I did not know, when I realized the nice lady who sat down next to me and with whom I was speaking was Fiona Charles.  The same Fiona Charles whose articles I'd read and bookmarked. WHOA!  A few minutes later, here comes Michael Bolton - no, not the singer or the guy from Office Space - Michael Bolton the tester guy.  He sits down at the same table!  WHOA^2!

We're digging into a breakfast of eggs and sausage and potatoes and fruit and coffee and tea and we're talking - I'm trying hard to be nonchalant - its not working very well.  Two of my "testing idols" are sitting at breakfast and we are talking.  WHOA^3!

So, the conference day begins - we're doing our thing at a workshop they are conducting and I'm participating in.  Make it through the day.  I grabbe some supper and had a drink and stumble off to bed with my mind quite melted. 

The next day, in between sessions, I find myself chatting with Michael who says "Got anywhere you need to be?  How about we play some games?"  YES!  DICE!  Awesome!  So, we dive in.  Pretty soon, there is a small group standing around the table we're working at - drinking coffee and juice and tea and talking and there are some really smart people there.  A lively discussion around "metrics" and "measurements" and "expectations" and "needs." 

There is Michael, myself, Fiona joined us, as did Lynn Mckee, Nancy Kelln, Paul Carvalho.  I realized that this "hallway track" had some of the best information of the day.  I also realized I was the only participant who was not a speaker.  WHOA^4

At one point, Fiona looked at me and said "What are you doing here?  You don't really fit.  You need to go to CAST."  I responded something like "CAST? What's that?"  And got a chorus of "It's awesome! You'd love it!  Its like this conversation but bigger!"  Then Michael said something like, "You're from Michigan, you said.  CAST is in Grand Rapids next year." 


I LIVE in Grand Rapids.  This way-cool conference is coming to Grand Rapids?  REALLY?  WOW!!!!!!!!!!!!!!

So when I got home, I looked it up.  I found "The Association for Software Testing" and saw the names of some of the people involved - and I said to myself "Self, these are the folks whose writing makes sense to you!  These guys rock!"

I did something I have continued to do since then - I bought myself a birthday present of a membership in AST. I have not regretted it. 

Why?  In AST, I found a community of people who are willing to share ideas and hear you out.  They don't see you as a novice, even when you are.  Instead, most of the people who really get it see you as someone who is on a journey with them to learn about more and better software testing. 

That conversation in the hallway at a conference in Toronto was only the beginning.  When CAST was in Grand Rapids the next August, I swung by the conference site the day before it began and ran into Fiona Charles, who was sitting with Griffin Jones.  I loaded them into my car and dragged them kicking and screaming to my favorite Italian place in Grand Rapids for dinner.  Giving them a mini tour in the process. 

We landed at the restaurant, sat down on the terraza, ordered wine, the lady-wife joined us - and we had an amazing conversation with dinner that covered nearly everything in our heads - architecture, art, the economy, US-Canadian history, software testing.  It was an amazing evening. 

Every CAST since then has been like that for me - Exquisite conversation, learning, enlightenment and challenges. 

Ideas are presented - and it is strongly suggested you be able to explain and defend them - otherwise the results will be "less than ideal" for you.  People selling stuff - from tools to snake oil - are sent packing.  People with challenges are encouraged.  People looking for ideas find them.

Each year is different - and each year there are similarities. Generally, the sessions inspire conversation and discussion.  This leads to thinking and consideration.  Sometimes they result in "Interesting Encounters."

Last year, someone was presenting on failed projects and mentioned the Mars lander - the one that crashed several years ago?  Remember that?  Partway through their story a hand went up and said "That's not quite what happened - I was an engineer on that project..."  Yeah. Really. 

This lead to a series of interesting hallway conversations - and the session she presented was very well attended.

So, what is it that, for me, AST is about? 

It helps me be better at what I do.

Friday, July 4, 2014

On Coffee and Process and Ritual and Testing

I am writing this the morning of July 4th.  In the US, this is a holiday celebrating the original 13 colonies declaring their Independence from Great Britain.  This morning is nearly perfect.  The sun us out, the air is not too hot and not too cold.  There is a gentle breeze.  I'm sitting out in the back garden reading and writing and sipping freshly made coffee. 

I like a really good cup of coffee.

No, I really like a good cup of coffee.

I like a well made cup of tea as well.  Don't get me wrong, a well made cup of tea with a bit of sugar and a dollop of milk, its a wonderful thing.

Still, I really like a good cup of coffee.

A couple of years ago, my lady-wife bought me a coffee press as a Christmas gift.  It was amazing for me to experiment with some of my favorite coffee beans and work out how to get the flavor and balance just right.  When I did, I was a very happy tester who really likes a good cup of coffee.

Things were great, until one crucial part went missing.   The wee tiny bit that held the screen mesh to the plunger that filters the water, now coffee, and separate the coffee grounds from the stuff that you want to drink.  I never proved it, but I strongly suspect that one day after washing the press and waiting for it to dry, our orange tom cat pumpkin found it an irresistible toy.

Needless to say, the choices of the stove top percolator (not bad, but still not as good as the press) or the electric drip coffee maker (ummm, ok, nuff said.)  So, struggling through for what seems an interminable period of "ok" coffee, the arrival of ANOTHER coffee press this past Father's Day was deeply appreciated.

Did I mention that I really like a good cup of coffee?  I do.  A LOT.

So, this was slightly larger than the previous one.  I would need to make some minor changes to my remembered favorite permutations based on which coffee I was making.  Then - I began comparing the coffee I had been drinking the week before to what I was making right then.  I mean, right - then.

Now, let us look at this.  What was different between these?  The coffee itself was the same - same roast, same grind, same water - same... Everything.  Really - that is what coffee is - a mix of ground coffee beans and water and... yeah.  That's about it.

So, what is the difference?  Maybe - the Process of making coffee?

We talk about Process Improvement - how do we make something better - like, Testing?  So, while the lady-wife chuckles at me for "my fussy coffee ritual" I find it makes things... better.  Now, if I use one coffee, like a nice dark roast, or another coffee, like a lighter, maybe a medium or a lighter roast, I may use a slightly different amount of ground coffee.  Or, I may allow the grounds to brew just a tad longer with one than the other.

The difference?  I'm not sure.  Maybe that slight variations will impact the coffee.  The tool I use to make coffee and the method I use to make coffee will definitely make a difference.

What does this mean?  I'm not sure.  Except for one thing. 

I know that if I blindly use the SAME amount of coffee, no matter the roast or the manner of which I plan to make it, and use the tools I plan on using - I will be disappointed. 

Here's what I mean.  If I am camping, which I really like to do, I have a handy percolator that I can make coffee in over a camp stove or over the camp fire.  It does a fine job and makes an enjoyable pot of coffee.  It is less work, well, cleanup work anyway than using a coffee press.  It is also less likely to break than the glass container of the coffee press.

If I am traveling on the road, like flying somewhere instead of driving, I may figure something out with making coffee in my hotel room that is less pleasing to me than my normal "at home" methods or when I am camping.  But, by changing how I make coffee, I get a much better cup than the thin stuff one normally gets from "complementary in-room coffee makers." 

The exception to this, perhaps, is the "complementary in-room coffee" I've had in Germany and Estonia.  (Hey, American hotel folks - go to Europe and check these guys out - they really GET coffee.)  Still, the in-room coffee in Europe is still not as good as I make at home.  Its not bad, just not as good as mine or a really good coffee shop.

What is my rambling point?  Well, I think it is this:  Using the same measures for everything, without looking at the broad circumstances, the context in which you are working, and the tools and means available to the task at hand, is foolish.

It does not matter if you are looking at test practices, management practices or coffee making practices.  Applying something without examination, because it is "the best way" to do something, is folly.

It yields disappointing results in testing, management and coffee.

Sunday, June 22, 2014

On Community and the World Beyond Testing

I live and work in Grand Rapids, Michigan.  Grand Rapids, for those not familiar with the city, was the home of Gerald R Ford and his wife Betty - Yeah, the same Ford who was President of the United States.  They renamed the airport for him - Gerald R Ford International Airport (yeah, the same one of Parkcalc fame).

My colleague Matt Heusser refers to Grand Rapids as the "silicon alley" of software testing.

There are some really outstanding boutique development organizations doing top-flite, world class work.  The first one that pops to mind is Atomic Object.  These guys rock. 

There are also a whole crop of small start-ups in niche markets doing good work and making their world better.

Then there are the really small "shops" - the one and two person outfits doing work on their own or in conjunction with others, or with other development teams that need a little help.  Then there is where they are doing stuff.

Those of us who have spent most of our working careers for companies, large or small, that had a central office where we worked - the sense of community, the water cooler, coffee station, cafeteria , break room, where ever, was part and parcel to what we did.  We'd chat and chill and hang out and bounce ideas off each other.  But when you don't have an office to go to - well, what do you do? 

If you're in (or around) Grand Rapids, you can hang out at a coffee shop, or your home (house, apartment, whatever) OR - you can find a co-working community.  A what?  A Co- working location!  A place to go, work, develop, think, drink coffee, bounce ideas off other folks, hang and learn. 

Like, The Factory.  These folks.

What is special about them?  Well, they do stuff that seems pretty cool to me.  Like, provide work spaces and meeting space and classes and training and coffee and an amazing space to hang out during the day or into the evening. 

They are regular hosts of several meetups, including GRTesters - the meetup I'm involved with.  Its pretty cool.  If you need a place to meet, and they have space available - and you ask them nicely - things can generally be worked out - at least from my experience. 

We've had folks join the discussion, presentation, whatever, simply because they were curious about what we were talking about.  Well, and we had pizza and wings.  That may have helped. 

The thing is, most of the time people are pretty good about things like, learning and sharing ideas.  Most of us like it when people express an interest in what we're interested in.  I expect a fair number of people would be happy if we asked them about what they are interested in.  Places like The Factory kind of make that easier. 

Classes on iOS development, JAVA development, database design, general design - stuff that helps people learn and grow.  People who want to be involved in something new to them is what helps to spread knowledge and ideas.  It is how we invigorate and reinvigorate our practices and our selves.

This is what helps build community and communities of practice. 

And this is what people at The Factory are good at encouraging. 

I don't know if you have something like this where you are.  If you do, that is great.  Get to meet them.  You never know who you might meet.  They might be pretty good folks to meet.  They might be folks who can help you learn and develop and grow.

They might be like the folks at The Factory.

Wednesday, June 18, 2014

On Technical Leadership and Relationships, for Irina

At Nordic Testing Days, this past June, I presented a session on Test Leadership.  It was a fairly light-hearted approach to a complex subject.  Specifically, I was talking about what is sometimes called “non-positional leadership” or, as I prefer to think of it, “technical leadership,” a term I first encountered in Jerry Weinberg’s “Becoming a Technical Leader.” 

It was a good session, I thought.  It was the last track slot of the conference, just before the closing keynote.  Because it was the end of three days of fairly intense work, I agreed with the conference organizers that a lighter approach might be in order.  So, I revived and modified the “Test Leadership Lessons from Harry Potter.”  It seemed a good idea – and passing out chocolates (mini candy bars to be precise) seemed a fun thing to do while people were filtering in.

Somehow I managed to trim a 90 minute presentation into around 35 minutes (well, I did go over – so around 37 minutes) and still had time for questions – around 10 minutes or so.  One question was, I thought fantastic “How do you know if you are ready to be this kind of leader?”

That was an excellent question.  (It also won the prize – they really do have prizes at NTD – for “best question” of at least some session.)  My answer was “Do you have people come up to you and ask for your advice or suggestions on how to do something or how to approach a problem?”  She nodded, with an expression on her face that said she was not quite clear where I was going with my response.  I continued, “If people are coming to you for these things, congratulations! You are becoming a leader!”

What does that mean? 

The idea of people coming to you for help is part and parcel of what I see “leadership” to be.  It is not an appointed thing.  It is not a mandated thing.  It is a mix of mentor, trusted adviser and supporter.   

The hard part is often letting people know help is available, without imposing the help.  When people come to you, on their own, that is recognition that you have developed some level of expertise on the topic at hand and they need guidance.

People who aspire to be leaders might consider that maybe becoming a leader is not really a viable goal in itself, at least through the paths I espouse.  Instead, the type of leader I want to foster and encourage is the type who masters their craft.  They are the type who studies diligently what they are about and why they are doing it.  They will seek out experts and learn from them.  In turn, they will often share what they have learned with others.

A few days after returning home, I had an email from a participant in the session.  She wrote:

The main question that popups is responsibility: does such kind of leader, who you were talking about, have responsibilities?

On the one hand, as long as he is a leader he affects other people, who's decisions may depend on leader's actions or opinion. So as long as he has power to affect - he should have responsibilities. You become responsible, forever, for what you have tamed. (Antoine de Saint-Exupéry )

On the other hand - he doesn't want to be a leader, he is just doing what he like to do. It's other people who put him on a leading position, so he shouldn't be responsible for other people's decision to follow him. Work on yourself first, take responsibility for your own progress. (I Ching)

So the question is: if leader knows that he is a leader, should he evaluate his actions and decisions according to how they might affect on people around him?

I promised that I would consider a response and would send one as soon as I could.

This is my response.

As I read the email, it seems there are three distinct points. 

First, there is the question of “responsibility” in general.  Does it exist for someone not in a traditional leadership/management role? 

To me, being a team lead or having the word “Lead” in your title is stepping into the realm of management.  I suspect this is because most organizations where I have worked, once one became a “lead” of anything the next step on the career ladder was “supervisor” or “manager,” more on that in a moment.

Then there is the statement of generally opposing views on the existence of any such relationship. 
First of these is the statement from de Saint-Exupéry (You know, the guy who wrote The Little Prince? Yeah, him... highlighted above) that states “You become responsible, forever, for what you have tamed.”  This is similar to the concept that if you save someone’s life, you are responsible for that person’s life going forward.

The second viewpoint is that presented in the I Ching (The Classic of Changes or Book of Changes, often attributed to Fu Xi, a legendary ruler of China, but it seems likely to be from much later, roughly 400-200 BCE. Either way, it is old by any measure I know of.)  The point made there, also highlighted above, is that one should “Work on yourself first, take responsibility for your own progress.” 

The third portion of the message is the crucial one, the “call of the question.”  

Let us examine the question. 

First, let me be clear that this form of “leadership” is not the form that most people look at as such.  This is partly because most people look at “leadership” as another term for “management.”  This is not about managers or team leads or senior people overseeing the development and work of more junior people. 

Instead, this is about learning what it is that we, as professional testers and students of our craft, are about.  Instead of leading others by compulsion or any other external means, we share our journey of learning with others and allow them to decide.

In truth, we do not lead anyone.  We enable them to begin on their own journey of learning. 
They may follow the same path we do for a while, or not.  By sharing or offering to share our learning with others , encouraging ourselves and others to share what has been learned thus far and what we want to learn in the future, we are opening the door a little wider than it was open when we arrived.

By helping others see what is possible and encouraging them to discover their own passion, we are leading them – in the sense of empowering them.  When we listen to what others have done, and learn from it to try ourselves, we are allowing them to lead us.

And there is where it gets interesting.

By allowing others to grow and explore their areas if interest, their chosen areas of professional development, we are demonstrating leadership in its most fundamental form: We encourage people and give them the opportunity to grow, then get out of the way until we are asked to assist or contribute. 

In doing so, when they come asking questions or looking for assistance, it is up to us to determine what form that assistance takes.  We can help walk them through their problem.  We can evaluate their solution, as presented to us.  We can pair with them as they do their work. 

We can make them the gift of our time to help them become better at their chosen craft.

This does not mean we do their work for them.  Nor does it mean we take responsibility for them not accomplishing their tasks in a timely manner.  

We can say “No.”

This also does not prevent us from saying “No.”  That may be extended into “No, not right now; can we get together later today?”  Or, it may be “We have done similar things several times; why don’t you try it on your own then, if I can, I can review it with you?”

This is the final point I want to make on the relationship between “technical leaders” and those they are assisting: it must be a relationship that is mutually agreed to.

If would-be leaders find themselves in a position where they are doing other people’s work for them, they are not helping that other person.  Such a relationship exists only when people have the right to opt-out either temporarily or permanently.  From what I have seen, the relationship must benefit both parties to be successful.

The best way I know of to solidify what has been learned is to try and teach or explain it to someone else.  While they are learning a new concept, we, the people explaining the concept, are testing our own understanding of it as we explain it.  The fundamental basis here is that our learning is solidified as we teach others. 

The answer.

In the end, I generally fall in with the concept of the I Ching – we must teach ourselves first.   In doing so, we can open up our learning to others.  As we grow, we can make this available so others may grow.  Whether they chose to act is not up to us.

Having said that, I believe there is a bit of responsibility to others once you step beyond the question of personal professional development.  I believe that we have a responsibility to aid others we help, although not “forever” as Saint-Exupéry would have us do.  Partly this is because we are freeing them, not “taming” them.  We help them discover their own passion to grow and develop.

This is why I take the time to formulate an answer to a question sent by email after a conference. 

Wednesday, June 11, 2014

From Tallinn!! Nordic Testing Days - Retrospective

I had the immense pleasure of participating in Nordic Testing Days in Tallinn, Estonia.

Estonia is a country of 1.3 Million people in Eastern Europe - way Eastern Europe.  The neighbors to the East are Russia - North, beyond the Bay of Finland is... Finland.  There are ferries that run between Tallinn, the capitol city of Estonia, and Helsinki, Finland.  Its a cool city with loads of medieval buildings, castles/city walls and churches.

The city's "Old Town', the medieval heart of the city is a wandering maze of narrow, cobble stone streets.  The town square boasts the oldest apothecary in the world - established in 1422 and continuing to operate today.  Oh, they also have wifi - free wifi.  Wifi EVERYwhere.

Tallinn is far enough North where for those of us from the US - it is pretty amazing.  By 11:00 PM the first week of June, it is still light enough where it would be somewhere between dusk, the gloaming and full night fall.  At 11:30 one evening, whilst sitting with colleagues at a patio bar in the conference center, we were amazed when we realized how late it was - and how none of us had any idea of the time.  (Hint if you'd like to go next year, check out the curtains in your room.  There is a second, heavier curtain that helps cut down at the light coming in the room, so you don't wake up at 3:30 AM - like WIDE awake - as I did a couple of mornings.)

General Conference Impressions

The conference is organized by volunteers - this is the third year - Yeah - 3 years and it is looking really, really good!

Matt Heusser and I agreed to host a Lean Coffee each morning.  And, each morning Lean Coffee had a small but doughty group who met to discuss items on testing and software development in general.

Wednesday, 3 June, Matt and I also hosted the 1 day version of Lean Software Testing.  Essentially, applying Lean concepts to testing.  For us, this was an experiment as we were not entirely certain how a series of exercises that absolutely demanded full participation of everyone in the room would translate across cultures.  In general - with a little effort - it translated pretty well.  We found chocolate helped.

I live blogged my experience from the two conference days.  (Day 1 is here.  Day 2 is here.)  I found the mix of speakers and format to be quite good.  At one point, Huib Schoots tweeted that he was torn between which session to go to.  As the conference went on, others had similar problems/  For me, this is a sign of a great selection of offerings at a conference.

The morning keynotes on Thursday and Friday I found interesting.  There were several valuable ideas that are worth consideration - I blogged them at the time - and chimed in on Twitter a couple of times.  Other participants had other takeaways. 

I must say here, that I got an email from Matt Heusser asking if I'd be interested in presenting the discussion "On Complete Testing" which we have done together a couple of other times.  This is a discussion around a simple question - "When the boss/customer/key stakeholder says 'this has to be completely tested,' what does that mean and how can we do it?"  I said "Sure! I think that would be fun!"  A week or two later I got an email from the organizers about "the keynote."  What?  Ummm - Matt?  We gotta talk.  As it was, we had a lot of fun, and based on the conversations, tweets and other folks blog posts, it looks like other people had fun to.

Bribing them with "chocolate imported from the US" and other items may have helped.


For me, the highlight of any conference is the people I get to interact with and learn from The many people I met or became reacquainted with over the time in Tallinn is astounding.

Let's see.  Who stands out?  Well, among the Organizers - who worked so incredibly hard -Grete Napits  Helena Jeret-Mäe  Raimond Sinivee Participants/speakers/delegates - Huib Shoots, Ruud Cox, Gitte Ottosen, Guna Petrova, Peter Varhol, Gerri Owens, Stephen Janaway, Bill Mathews, Dan Billing, Kristjan Karmo, Irina Ivanova, Rob Lambert, Aleksis Tulonen, Andrei Contan, Rikard Edgren, Martin Nilsson, Aapo Reitsak, Raji Bhamidipati ... the list goes on.  If I stop and think about it - I know I can add more names to this group.  Like the fellows I was chatting with on Thursday during the networking/entertainment evening.  I know you told me your names, but SHEESH guys - bring cards next time to help job my memory!

People referred to with some frequency...In no particular order - Jerry Weinberg, Nicholas Taleb, Michael Bolton, W Edward Deming, Daniel Kahneman, Doug Hoffman.  I know there were others, but these were the names that came up in the sessions I participated in.


I thought Nordic Testing Days was extremely informative.  Unlike some conferences I've been to, there were no sessions where I wished I had made another choice and I did not get something worth considering.  Importantly, there were no sessions where I simply wrote the presentation off and focused on email or something else.

Tallinn is a beautiful city.  The people are wonderful - not in the annoyingly fake "How are you?" way - when what they mean is {Insert generic greeting here} but more in the "We have more than one way to communicate and make you feel welcomed in our wonderful city" way.

One word of caution - when walking in the Old Town, really pay attention to where you are going.  The cobblestone walks and street are - cobblestone, meaning not smooth cement - they are not even and you can trip easily if not vigilant.  

So, WHEN you go next year, keep an open mind, be ready for long nights with fantastic conversation and be ready to think and learn.

Friday, June 6, 2014

LIVE from Tallinn! Nordic Testing Days - Day 2!

Friday, June 6 - Day 2 at Nordic Testing Days.

SOMEone (ahem) overslept and came flying in to Lean Coffee to find Matt Heusser running it (Thanks, man!) with Rob Lambert & Raimond Sinivee and some 6 others participating.  It was a fantastic conversation!  And like every other Lean Coffee I've been to, the topics and conversation flowed too quickly to track what was happening.

And now I'm after a bit to eat and at least one more coffee - Will be back for the keynote - see you then!


First Keynote for the day - Arto Veldre is presenting on Software Errors as the Founding Pillar of Modern Society.  The opening gambit is that there are myths that each of us fall into easily.  Then slides under "what is Estonian" - mentions 99 stereotypes of Estonians.  He observes that Estonia has one of the most connected society in Europe (yeah - that is true from what I have seen - even the forests in the parks and open space have wifi hotspots - no, I'm not making that up!)

So of all the cool technology things going on - are any of them impervious to problems?  Are they safe from accident? (well, no, that is why we have jobs, right?)  The harder question is are they impervious to intentional, not accidental, harm?  How complex are the systems we have built?

Simply put (his words) "Are these computer systems ready for prime time?"

Can we really understand how systems work, let alone their safety from attack - if we don't understand the complexity of the technology under them.  (He had a cool slide showing the landing computer from Apollo 11, along with a binder of source code printed on green-bar paper (flashback to my early career!) 

We could understand the code (at one time) and now many systems have become more and more complex - Now - the fragility of systems increases with each additional layer of technology.

He makes an EXCELLENT point around security - "Information security is the only discipline that has laws before the textbooks are published."  (Yeah - that is related to one of the items discussed in Lean Coffee!)

AND after another reference - he brings up self-fulfilling prophecies - and Nicholas Taleb's Black Swan - where we are blind to the situation if we base our understanding only on a limited set of input - for example - the turkey in the US.  Everything looks like things are awesome and will continue to be awesome until Thanksgiving rolls around - then there is nothing to prepare/warm them.

And references the idea that maybe (maybe?) that the problem lies between the keyboard and the monitor.  (Re: Weinberg's all software problems are people problems)

We are wired (biologically and socially) to expect certain things.  These can fall into the realm of bias when we consider what it is that we are trying to understand - and things don't fit.  We will see something that does not fit and we can either "turn away" or reject it as "not real."  Sometimes it may NOT be real - sometimes it is.  Sometimes we can see things and will say "that can not be it" and we'll translate it into something that fits our view.

He runs thru a series of high profile and less well known - the F22 & Chinook aircraft - both had significant software errors - Heartbleed - Apple XSS - etc.,

How did these happen?  What caused them?  More to the point what allowed them to get through and not be detected/found and removed?

Well, we're human.

Sometimes they may be accidents - then there is stuff that happens that we do not pay attention to, even though we are aware of them at some level.

Why does the Estonian ID card (and other country's for that matter) have a chip embedded in them?  Why do certain "smart TV's" identify and record information about what you are watching - either through cable or broadcast or streamed via the internet?  Do these systems serve a purpose?  Are they really needed?  (reference the NSA)

So, what happens with "big data"?  Who oversees that?  Who guards that and keeps it from improper use?  Who tracks what and why?  How do we know?

OK - he had a stack of slides and ideas - I caught a few of them and that was pretty much it...


Right, so grab another coffee and some fruit and off to a presentation by Raji Bhamadipati.  She's hosting a workshop titled "Where Are the Notes From When You Testing This?"

So, the room is absolutely packed (says the guy who is happy he grabbed a spot near a power outlet)  In the room are Bill Mathews, Rob Lambert, Matt Heusser and a stack of other folks.


Some nice slides of people's notes - from Da Vinci's notes from the British Museum, to diaries of British soldiers from WWI, to images of people's notes from different purposes.  The gist is, notes and note taking tend to serve as memory aides for the person taking the notes.  The "If you are not taking notes you are not learning" is referenced, with the caveat that others may take notes differently.

How these are applied to testing gets pulled in - as in a bug is detected by a customer months after the project releases - How do you remember what you saw when testing?  We might have the scripts, we might have bug reports or session reports or something.  But notes do things that may jog our memory.

EXERCISE!  We have post its - so start with notes!  What do we use for note taking?  LOADS of stuff - see?

Then there are other options...  We have stuff to do - and now we get an exercise! (another?)

Try it: - then select Puzzle 11... and I'm going to try it!

so there's a bunch of us playing - and without hacking the code, we had a bit of fun.  So - click the button on the left - that in turn changes the behavior of the button on the right.  Now... prove it

And folks are walking thru what they did - and Dan Billings (yeah, the guy who talked on Security yesterday) decompiled the code.  (snarl,,,)  Fun.


Right - the point is, notes can help us remember and understand.  Any form of note taking has its own perils.

What makes a good set of notes for testing?

Structure -
Title/charter/test case number -
Time stamp -
Configuration/set-up -
Scenarios -
Detail (just enough information) -
Bugs - what happened you did not expect? -

If we can't identify what and how we are testing, does what we are doing really count as testing?  Really?

Personal notes may require less formality than notes the boss may want to see.  In any case, there need to be enough information for people (like, you) to understand the point of what you did, how, and what you noticed.

So, how do you know if you are taking good notes?  Try opening them up and seeing what they tell you... say in like, 60 days.

Lunch Time!  YUM!!!!!!!!!!!


OK, so I took much of the afternoon off, relaxed and prepped for my track talk.  Sat with my lovely lady-wife, ran into her while she was heading down to lunch - then chatted with folks - it was really a nice hallway track session - relaxed and easy among presenters who were done and others who (like me) were getting ready to go.

We had some interesting views on the world and what fun was - and what to do.

So, in the end - I chilled - until 3:35 - Then I went in and did the talk on Stepping Up to Leadership: Test Leadership Lessons from Harry Potter.

So, things went on and I worked up a sweat and gave away the last of the candy we brought from the US - And generally had a good time.  Changed my shirt and grabbed a beer (yeah, its Estonia - apparently others were jealous when I walked into the closing Keynote with a beer) and grabbed a seat where I could sit and write and ... keep the beer from getting tipped over. (Priorities)

In listening now to Iris Classon on The Undeniable Value in Curiosity, Collaboration and Contribution.  She describes herself as "The world’s happiest .NET developer & Microsoft C# MVP! More energy than a 5 year old, talks more than your grandmother..."

So she's been banging on some of the same ideas I presented in my session -

Ask question, be curious, try stuff - The twitter feed on her  talk (so far) consists of...

Right - so you get the idea.  And - the thing gets cut short because of tech problems?  I'm not sure. 

Anyway - the questions were good - centered around curiosity and generally, they were interesting.  I think Huib Schoots asked the best question "What is curiosity?" 

Now then - The rest of the day is a blur...

It was a fantastic week - I'm exhausted.