Saturday, August 8, 2020

Good Testing When the Audience Calls for Jaja Ding Dong

 Sat down for a night a silly entertainment a little while ago. Found myself watching the movie "Eurovision Song Contest: The Story of FireSaga." Yup, That is the name of the movie. OK, ignore the critics. It is silly. High-brow? This ain't it. (Will Ferrel produced and stars in it. That should tell you about the type of humor in it.) 

To Start, for Americans who have never heard of it, Eurovision is a huge, big deal. In Europe. Each year there are these amazing groups who compete for the honor of representing their country as the national representative at this massive contest. There are fantastic bands playing each year. Sometimes the "bumper" entertainers get loads of attention (Riverdance became a "thing" as a result of one of these performances.)

Like musicians everywhere, I could relate to the dilemma facing Will Ferrel's character early on, and at the end of the film. Each band has stuff they are pretty much expected to play at bars, clubs, concerts, festivals. Whatever. Then there is the stuff they really LIKE to play. The stuff that allows them to shine and show what they are capable of as musicians and as performers.

We ALL want to play the stuff that we like. We want to play the stuff that shows off our talents and leave people in awe. The only problem is, the audience wants us to play the stuff that is, well, not very good. It is not good musically or in any other way. Except, it has a beat and a simple, catchy tag line.

For the fictional band Fire Saga, that song is "Jaja Ding Dong." It is as lame as a song can be. Except it has a simple, obvious tagline that people drinking in a bar can sing along with. Sometimes that is about all you can ask for.

On Stage

I spent a lot of years playing in bands in bars and beer tents where people were there not for the amazing music we played, but to have fun. I absolutely promise that the stuff they found "fun" paid for the night. It also kept them coming back.

This allowed us to experiment and play the music we wanted to play. It allowed us to experiment and tweak it in front of humans who likely would not care. When we had their attention with what we WANTED to play, then we knew we had something that would work anywhere.

If we could not get something to "work" after several tweaks and modifications over a bunch of performances in bars and the like, then that piece was retired. We could use it somewhere, just not there. It stopped being fun for us because it became massive amounts of work. No matter how "good" it was, it wasn't good enough and likely would never be.

The goal was to get enough of what WE wanted to play to the point where audiences wanted to hear it. If you could get the 100 or so folks in a bar to listen, then get several hundred in a beer tent to listen. THEN you could get a couple thousand at a festival to listen. And when you play it at a concert venue, the response will be tumultuous. You have something they really want when not long ago they thought they wanted something very different.

Here's the trick. If you REFUSE to play the stuff they want to hear early on, you will never have the change to play the stuff you want to play. Ever. You won't book that venue again. You likely won't get booked anywhere else again because the owners and operators and reps know each other and will talk.


What does this have to do with testing software?

When you come in as the "new" person to an organization, remember that they often have their own way of doing things. Walking in and telling them they are doing everything "wrong" and you simply refuse to do it that way and you are the only one there who "knows how to test" is likely not going to get you very much.

Even if you are being brought in for some version of a "leadership" position, walking in and telling them to "change everything" is likely going to do more harm than good. Try joining them. See precisely how people are working. Work with them, side by side, as a partner.

That way, you are demonstrating you can do the "testing" that is expected. You can then try other ideas that are similar, but not exactly the same. If these ideas show an improvement in results and still meet or exceed all the control metrics in place, you might be able to tweak them a little. By small changes, you might be able to show more improvements.

In time, you can do the kind of testing you want to do and provide the value the organization needs. If you are patient you might just find yourself helping others do testing that way. That "new" way. 

Then the people demanding you do things their way, once upon a time, will want you to do it this "new" way. Except it likely won't be a new way then. It will be a better way. Then, they might just want everyone to try testing that way.

Then, when someone wants you to do testing like "Jaja Ding Dong" everyone else will laugh at them.


  1. That's good advice. However, I think we should balance that with a few other viewpoints:
    1. Among leaders in music there is no ambiguity about what is (good) music. Testing has a lot of mixed messages. There are also very powerful outside groups who support testing anti-patterns. This reduces the impact of the analogy t omusic.
    2. For various reasons, you can't test in public. As a result, people talk about how to test, without any evidence that they can test. Imagine if that were the case for music. Here is an example of how to test:

    1. If "good music" was universally applicable, I'd agree. People's tastes and what they like or prefer is not universally applicable. Walking into a blues bar playing country music is likely not going to get a repeat booking. That is not what they want.

      Organizations have various reasons for what they want in test practices. Some are good reasons, many are not. Getting them to change from less-good practices to more-good practices rarely works by invoking authority. Teams and organization leadership will be change only when they see the benefit of changing.

      Yes, I know Dr. Kaner. He and I have had many good conversations around testing and teaching and life