……Baby think twice.

Automation in test is wonderful, it can be time-saving, it can find bugs early in development and it can run when you’re not even clocked-in. But, there can be a lot of misconceptions about automation in test, or even whether you consider it testing or checking:

    Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc.

    (A test is an instance of testing.)

    Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product.

    (A check is an instance of checking.)

Ref: James Bach

In my experience, you can have too much of a good thing, I have a metaphor, so bear with me. At home, my children drink a dilutable cordial/squash, to ‘make water taste better’. This comes with a recommended water to squash ratio, let’s say 5:1. If I reduce the ratio of squash to water to be 1:10 , the taste is less acceptable, or conversely, if I increase the proportion of squash to water, it can become too strong and unpalatable.

My end goal is to not see this face when they have a drink:

That is one unsatisfied customer!

The point here is that we need a mixture of testing heuristics, a balance in order to achieve the most coverage of code tested or checked.

James Bach again has a great outline of testing heuristics here.

Sports analogy time…..sort of

I play table football in my lunch break sometimes. I am a one-trick pony with this game, I can hit the ball pretty hard. However, if the opposing player blocks my shot, they may score, and I almost certainly won’t. My point is that if there is only one point of attack, then I’m not testing the defence of my opponent, it’s undynamic and predictable. But, it does feel good when I score. So, I’m not suggesting that I ditch that tactic completely, merely that I diversify.

I am going to blog at a later date on designing an automation project/framework, but for now, here are the key points:

• No single solution will best test your software, please don’t invest in 100% test automation
• The right mixture of heuristics will depend on you as testers, your software, tool availability etc
• Consider what it is that you check and what you test
• Think about where you can get early ‘wins’ with your automation, e.g. time savers
• Don’t automate for the sake of it
• Learn more, explore more, find your balance and don’t get complacent

Blog post title lyrics from: Think twice – by Céline Dion.

Find all the songs from my blog posts at this Spotify playlist.