Three New Testing Heuristics

A lot of what I do is give names to testing behaviors and patterns that have been around a long time but that people are not systematically studying or using. I’m not seeking to create a standard language, but simply by applying some kind of terminology, I want to make these patterns easier to apply and to study.

This is a quick note about three testing heuristics I named this week:

Steeplechase Heuristic (of exploratory boundary testing)

When you are exploring boundaries, think of your data as having to get to the boundary and then having to go other places down the line. Picture it as one big obstacle course with the boundary you are testing right in the middle.

Then consider that very large, long, extreme data that the boundary is designed to stop might founder on some obstacle before it ever gets to the boundary you want to test. In other words, a limit of 1,000 characters on a field might work fine unless you paste 1,000,000 characters in, in which case it may crash the program instantly before the boundary check ever gets a chance to reject the data.

But also look downstream, and consider that extreme data which barely gets by your boundary may get mangled on another boundary down the road. So don’t just stop testing when you see one boundary is handled properly. Take that data all around to the other functions that process it.

Galumphing (style of test execution)

Galumphing means doing something in a deliberately over-elaborate way. I’ve been doing this for a long time in my test execution. I add lots of unnecessary but inert actions that are inexpensive and shouldn’t (in theory) affect the test outcome. The idea is that sometimes– surprise!– they do affect it, and I get a free bug out of it.

An example is how I frequently click on background areas of windows while moving my mouse pointer to the button I intend to push. Clicking on blank space shouldn’t matter, right? Doesn’t hurt, right?

I actually learned the term from the book “Free Play” by Stephen Nachmanovitch, who pointed out that it is justified by the Law of Requisite Variety. But I didn’t connect it with my test execution practice until jogged by a student in my recent Sydney testing class, Ted Morris Dawson.

Creep & Leap (for pattern investigation)

If you think you understand the pattern of how a function works, try performing some tests that just barely violate that pattern (expecting an error or some different behavior), and try some tests that boldly take that behavior to an extreme without violating it. The former I call creeping; the latter is leaping.

The point here is that we are likely to learn a little more from a mildly violating test than from a hugely violating test because the mildly violating test is much more likely to surprise us, and the surprise will be easier to sort out.

Meanwhile, stretching legal input and expectations as far as they can reasonably go also can teach us a lot.

Creep & Leap is useful for investigating boundaries, of course, but works in situations without classic boundaries, too, such as when we creep by trying a different type of data in a function that is supposed to be rejected.

4 thoughts on “Three New Testing Heuristics

  1. Hi James,
    I’m glad to finally have a term to the test execution style you call Galumphing. As soon as I read your description I thought – hey I do that a lot. Now that I’m aware of it and not just doing it instinctively I can make conscious decisions about when to use it.

    It will also be helpful when I am explaining this approach to others – thanks!


    [James’ Reply: I didn’t realize I did galumphing until David Gilbert, a developer, pointed it out to me when I was trying to figure out why I kept accidentally crashing his activity recording tool… I was overflowing his recording buffers with my clicking frenzy!]

  2. Very Interesting !!……. I can relate this to many activity Tom (to catch Jerry)and Jerry (to evade Tom’s traps) do !!

    Can we add a few more to this list ?

    [James’ Reply: Sure, go ahead. Try to think of catchy labels for them. This helps bring them to mind when we need them.]


  3. to add 2 more scenarios to the list

    1. do exactly the opposite – say you have a right handed screw and a left handed space to put it through….Murphy’s law ? something exactly different from what you are expected – say a screen has a name field and it expects you to press enter after you have made an entry but you keep shift+A pressed for more than a min ?

  4. One more scenarios to add:

    There are text boxes where on the search screen where only some characters are allowed. Enter that character and select them using Shift+Home. There are chances that they wont be selected and User will be getting error message as “*** character is not allowed”. I tried it and got in many cases.

Leave a Reply

Your email address will not be published. Required fields are marked *


This site uses Akismet to reduce spam. Learn how your comment data is processed.