Have you ever had a software problem that disappeared even when you did nothing to correct it? Or have you ever fixed a bug by doing something that seems as if it shouldn’t have fixed anything?
Whenever that happens to me, I A) remain wary, and B) remove the fix so that by seeing the problem again I have additional evidence that the “fix” was truly the fix. I call this is the dead bee heuristic, because if there’s a bee in my living room, I don’t want it to mysteriously disappear, I want to see it fly out my window or be dead on my floor.
This applies to testing situations, too. If I change a data file and see that it no longer crashes the application I’m testing, the next thing I do is change it back again so I can see the crash one more time.
“Say, was you ever bit by a dead bee?…You know, you got to be careful of dead bees if you’re goin’ around barefooted, ’cause if you step on them they can sting you just as bad as if they was alive, especially if they was kind of mad when they got killed. I bet I been bit a hundred times that way.” — Walter Brennan as “Eddie” in To Have and Have Not
And always bear in mind that killing the “bee” may not have solved the real problem, or may have created new problems.
In my 9 years of test experience I have whitnessed the ‘dead bee heuristic’.
I have even seen something stranger… once I found a bug, but I only saw it 1 time. After that, I never saw it again. I talked about it with colleagues and friends but none of them could believe the bug. I kept saying and telling what I had seen. But they all said the same thing “show me the bug”, “Can you reproduce it?”, well you know how this story goes.
I never could reproduce it.
Untill on 1 day we had a large meeting with 20 people on a large screen. The person who gave the demo reproduced the bug. It blinked on screen before everyone’s noses. Everybody believed me from that moment. *”you saw the Windows background wallpaper through the application for a split second.”
However, we never found the cause and none of the programmers could reproduce the bug ever since.
We don’t even know if the bug is still in the application or solved because of ongoing development so to speak.
But the thing is: How do you call such type of bug? How should it be named? A mystery? 🙂
[James’ Reply: It’s a hard to reproduce bug.]
Please don’t kill bees. Pollinators are our friends.