Satisfice, Inc. welcomes my sister Erika Good to our company. She’s our new Director of Marketing and Communication.

Erika is two years older than me. When I first got kicked out of my home, in 1981, I was a freshman in high school. Erika and I would meet each morning (on the mornings I didn’t oversleep or otherwise skip school) in the school library to check in. Later, it was Erika who alerted my father that I had started drinking, which is why I have only been drunk twice in my life– both when I was 15 years old– and why I barely ever drink now. Erika and I have always had a bond. Now she and my wife Lenore will share the burden of keeping me productive when I’m on the road and helping me organize and publish all my stuff.

Erika is quiet, polite, and unassuming (the opposite of me, in other words); a dedicated housewife and mother of two grown boys. She’s also an expert marksman (she competed as a member the Vermont National Guard pistol team— wait, she’s saying it was the rifle team. How did I not know that?),  an artist, an author as well as a private pilot and webmaster, who also helps run her husband’s property business… Geez, I didn’t realize how much she’s done until I had to write that sentence!

One immediate effect she will have is on our long neglected website. She’s already put up the announcement of the Rapid Testing Intensive, and added a video page. She’s helping me create a brand new bibliography and resource list. She’s going to add tags to all the blog entries and get a lot more content published that has been sitting on my hard drive.

This poor girl from Ghana inherited many kilograms of gold and can’t get at it because she can’t pay the back due rent on the gold storage unit. She has reached out to me, a total stranger, for help. I use my testing skills to help her understand that she’s being scammed. But you know, sometimes it’s hard to help people see the truth when fantasy seems less painful…

A stranger approaches on Skype…

[12:29:22 AM] Irene Kusi: Hello

[12:29:33 AM] James Bach: hi

[12:29:44 AM] Irene Kusi: How are you doing ?

[12:29:54 AM] James Bach: Why are you contacting me?

[12:30:10 AM] Irene Kusi: I am very sorry about that and sorry again if i disturb you. I am Irene Kusi from the UK but i am now in Ghana and i would like to know more about you.

[12:30:51 AM] James Bach: Why are you contacting ME, though?

[12:31:39 AM] Irene Kusi: Like i just told you .sorry if i disturb you.

[12:32:06 AM] James Bach: You haven’t told me why you came to me instead of someone else. Is there any reason?

[12:32:10 AM] Irene Kusi: Can we get to know each other personally? What do you mean ?

[12:32:27 AM] James Bach: Look, strangers contact me often for business purposes. That’s normal. I’m trying to figure out if this is for business. Usually people tell me why they are contacting me.

[12:33:08 AM] Irene Kusi: Yes is not strictly business. I just would like to get to know you

[12:33:18 AM] James Bach: So, you don’t know who I am.

[12:33:54 AM] Irene Kusi: Yes i really dont know you but i found your profile on skype and decided to get to know you if i can work with you.

[12:34:23 AM] James Bach: You just chose me at random? Seems like a lot of people from Ghana are doing that.

[12:35:13 AM] Irene Kusi: Why have some people from ghana done something like that to you ?

[12:35:18 AM] James Bach: Yes you are not the first person from Ghana to randomly contact me

[12:35:40 AM] Irene Kusi: And what did they want ??

[12:35:51 AM] James Bach: They wouldn’t tell me… Just like you aren’t telling me.

(She cuts the small-talk and tells me her problem)

[12:36:30 AM] Irene Kusi: My main motive here is to find someone who will help me inherit my property thats why i am on skype .

[12:36:40 AM] James Bach: oh, a 419 thing. I understand

[12:38:27 AM] Irene Kusi: Oh who said is a 419 thing

[12:39:03 AM] James Bach: I did, “oh, a 419 thing” See, that’s where I said it.

[12:39:23 AM] Irene Kusi: what shows is a 419 thing

[12:39:41 AM] James Bach: You said “My main motive here is to find someone who will help me inherit my property thats why i am on skype.” If you haven’t heard of 419, that’s okay.

[12:40:54 AM] Irene Kusi: Ok

[12:41:12 AM] James Bach: Usually it’s Nigerians who need this kind of help. I didn’t know it was happening in Ghana, too.

[12:41:34 AM] Irene Kusi: Ok .so are you going to help me or not ?

[12:41:46 AM] James Bach: What do you need?

[12:42:06 AM] Irene Kusi: I will like you to help me pay with the bills so that i can get the gold

[12:43:07 AM] James Bach: Oh, you want money. Why didn’t you say so at the start?

[12:43:20 AM] Irene Kusi: I dont know why

(With my tester skills, I immediately comprehend what’s going on: she’s too innocent; she sees only the good in people)

[12:44:26 AM] James Bach: Okay, but, why would you trust me? You don’t know me. I might be a bad guy.

[12:45:11 AM] Irene Kusi: Nothing i have just gotten to know that you a very giid guy and i will trsut you .

[12:46:39 AM] James Bach: I think you have to be careful, though. There are many people in the world who may try to take advantage of you. Especially if you have inherited gold. Let me ask you this: If I help you, will you share part of the gold with me?

[12:47:51 AM] Irene Kusi: I will do exactly that not just half but i will let you get the whole gold that is i will bring it to the USA and then we will sell it there and share the money. But before we share the money i will give you 5kilograms of the gold for free just for helping me get it . [Editors note: that's about $300,000]

[12:48:30 AM] James Bach: See? You were just about to trust me and then I tried to take your gold! Irene, you are WAY too trusting. People will want to take from you and not give you anything.

[12:49:14 AM] Irene Kusi: Oh so will you do that to me ?

[12:49:28 AM] James Bach: I just proved that you would LET me if I TRIED to. That was a test. I do testing for a living. And you showed that you are too innocent and good. This is a hard world, full of cheaters.

[12:50:07 AM] Irene Kusi: Oh okay thank you so much for telling me this .

[12:50:13 AM] James Bach: They may try to contact you and fool you into giving up your money. So be more careful.

[12:50:26 AM] Irene Kusi: Thank you

(Now I go into consulting mode… First, learn about the problem. Often the real problem is not the one they come to you with.)

[12:50:31 AM] James Bach: Why don’t you keep your money? You will need it.

[12:51:16 AM] Irene Kusi: Ok so are you going to help me pay for the bills so that i get the gold ??

[12:51:46 AM] James Bach: I thought you said the gold belonged to you.

[12:51:52 AM] Irene Kusi: Yes

[12:52:04 AM] James Bach: Then you have much much more money than I have. You can pay them yourself, can’t you?

[12:52:21 AM] Irene Kusi: Oh no not right now. You know my dad left the Gold as a form of inheritance for me and since he could keep all that in his house he had to give it to his lawyer.

[12:53:14 AM] James Bach: Okay, so the lawyer has it.

[12:53:42 AM] Irene Kusi: Yes and what the lawyer is also saying is that he gave it to a security house here in ghana

[12:53:55 AM] James Bach: That’s good. Then you know where it is. What happened when you went to get it?

[12:54:42 AM] Irene Kusi: They told me that i have to pay for the number of years that it has been there so that they can give it to me . And the lawyer told me that i also need to get a man who will stand by me just to help me

[12:55:42 AM] James Bach: Is this place a bank?

[12:56:03 AM] Irene Kusi: No is not bank but a storage house .

[12:56:19 AM] James Bach: Is it licensed to store gold?

[12:56:29 AM] Irene Kusi: Yes. Is a licensed company. It has the sole mandate to store gold for people

(Use my knowledge of local laws and customs to reveal the true culprit. A consultant should have deep knowledge in useful domains.)

[12:56:57 AM] James Bach: Then they are violating Ghanan law by charging you to release it. Your lawyer should have told you that. Do you trust that lawyer? Do you know him well?

[12:57:27 AM] Irene Kusi: Yes he is the family lawyer .

[12:57:51 AM] James Bach: Then why is he telling you something that is against the laws of your own country? I think you need to talk to a different lawyer.

[12:58:59 AM] Irene Kusi: When we went there they told me that is not like they are charging me for the gold which is mine said by my late dad but for the number of years they have kept the gold for safe keeping thats what they mean. The lawyer has got nothing to do with this at all.

[1:01:34 AM] James Bach: That’s called an “advance fee scam” and they are trying to trick you. You need to go to the police. But I think your lawyer may be in on this. He may even have paid off the police.

(My kind of consulting involves tough, direct speech, on occasion. Sometimes that’s hard for clients to hear.)

[1:02:07 AM] Irene Kusi: You cant talk to my lawyer like that

[1:02:12 AM] James Bach: Your lawyer, if he knows Ghanan law, knows that they aren’t ALLOWED to charge you. They can’t charge you! You are being tricked!

[1:02:26 AM] Irene Kusi: Ok

[1:02:39 AM] James Bach: I told you that you have to be careful. Gold is VERY valuable. People want to steal it.

[1:03:03 AM] Irene Kusi: so now are you going to help me pay the bills or not ?

[1:03:46 AM] James Bach: I’m telling you that you don’t owe any bills. It’s a trap. They want you to pay them and then they won’t give you anything. You need to wake up Irene. You are a crime victim.

[1:04:05 AM] Irene Kusi: Stop what you are saying

[1:04:06 AM] James Bach: That gold is yours. You deserve it.

[1:04:08 AM] Irene Kusi: i hate it.  what do you take me for ? you are not ready to pay fpor the bills thats why you are doing this

[1:04:40 AM] James Bach: I think you put too much belief in people who are now trying to hurt you.  Now you are upset and you want to blame me. But I am just the messenger.

[1:05:08 AM] Irene Kusi: do you think the storage house is an orphanage home where they keep things for free?

[1:05:59 AM] James Bach: Stand up for your rights, Irene. Even in Ghana, there are laws and people have to follow them. They can’t just steal your gold and say you can’t have it because it’s been “sitting there for a long time”

[1:06:24 AM] Irene Kusi: Stop what you are saying

[1:06:26 AM] James Bach: Under GHANA LAW a licensed gold storage house CANNOT charge for storage AFTER the fact. They take payment UP FRONT. Your father already paid. Your lawyer knows that. You have to fight for what is right.

[1:06:49 AM] Irene Kusi: no he didnt pay

[1:07:06 AM] James Bach: THAT’S WHAT YOUR LAWYER TOLD YOU? I can’t believe what a liar this guy is. Do you think you can just bring gold to a storage house and not pay them to store it? He must have paid. So now you are asking strangers to help you pay a bill that YOU DON’T EVEN HAVE.

[1:08:23 AM] Irene Kusi: Bye

[1:08:29 AM] James Bach: I hope you get your gold. Think about what I said.

(If the client is not ready, it doesn’t matter what you tell them: they won’t benefit. I like to think Irene is taking my words to heart. But I have to accept that she probably won’t. Meanwhile I can only hope that her lawyer has invested his ill-gotten gains in an American solar company.)

Announcing the Rapid Testing Intensive seminar.

This is something new. From July 24 to 28, my brother and I are going to run a short, extreme testing project over five days. We have a nice big space, good Internet, and we’ll be cut off from all distractions.

The most interesting element, from my point of view, is that students can attend onsite OR online. Onsite attendance is more trouble, more work, more expense, but also a completely vivid immersive testing experience where you will work shoulder to shoulder with two of the guys who created the Rapid Testing methodology that is now taught all over the world. Online participants will also test and attend optional webinars twice a day during the week. We will have a dedicated coordinator and an online forum so that the experience will be exactly like working as a remote tester on a busy project team.

The outcome of the week will be a test report that will become a part of each participant’s professional portfolio.

I’m so glad Jon is doing this with me. We complement each other well. Jon is a day-to-day test manager who has applied Rapid Testing in his work for nearly all his career since the mid-90’s. I’m a consultant who’s been teaching Rapid Testing since I invented it. Besides, it gives me an excuse to use this graphic…

This may look like George Lucas and Vladimir Lenin, but it’s me and Jon…

Hey, I’ll be doing a public class in England, once again. This time in Cambridge. See the details, here.

This year I also have public classes in Estonia, Romania, Australia, and New Zealand.

Michael Bolton is accused of hand-waving in this thread on LinkedIn. (See the comment by Peter).

Michael and I talk a lot about cognition and exploration. We speak in tropes that come from philosophy and various branches of science. Once in a while, some fellow who understands little of what we say assumes that we just made it all up to impress the ladies.

It brings to mind the Large Hadron Collider. Here’s an excerpt from one of their bulletins:

“The very smooth and fast transition to operation with ions was made possible by very good beam instrumentation performance with a relatively low number of charges per bunch, and magnetic behaviour very similar to operation with protons, as expected. These two factors combined allowed the setting-up operations to be completed very quickly, and stable beam operation, with 2 bunches per beam, was achieved in just a few days.”

Gee, if you know nothing about physics, or the LHC project, this might sound very much like hand-waving, too! In this case, however, if it sounds sketchy, it’s probably more about the receiver than the source. After all, there really is a $10 billion device sitting in the ground with 4000 physicists poring over the data it generates.

As ambitious professionals, we need to be able to speak about complex subjects without constantly going back to kindergarten to bring along the people who refuse to study their own craft.

I don’t mind if an earnest seeker, who happens to be ignorant, asks what seems to be a silly question. I will help everyone who wants to learn. And I don’t mind the assertive dissenter who has done the homework and yet has a different style and judgment from mine.

I’m talking about something different: the willfully ignorant blowhard.

Please don’t be like that.

Michael Bolton has gone off like a volcano in Iceland, writing a series about what exploratory testing isn’t:

http://www.developsense.com/blog/2011/12/what-exploratory-testing-is-not-part-1-touring

http://www.developsense.com/blog/2011/12/what-exploratory-testing-is-not-part-2-after-everything-else-testing

http://www.developsense.com/blog/2011/12/what-exploratory-testing-is-not-part-3-tool-free-testing

http://www.developsense.com/blog/2011/12/what-exploratory-testing-is-not-part-4-quick-tests

http://www.developsense.com/blog/2011/12/what-exploratory-testing-isnt-part-5-undocumented-testing

Another thing I would add to this:

Exploratory testing is not defined by any specific example of exploratory testing.

Just as tap dancing does not characterize ballroom dancing, you can’t take any one example of exploratory testing and treat that as representative of the entire concept of ET.

If you were to hear me singing an aria by Mozart, that would be an example of opera singing. It would be an example of BAD opera singing, but it would truly be an example of the style. Similarly, I regularly talk to testers who go “oh yeah I’ve seen that exploratory testing stuff but it’s not structured… not documented… not x… not y… not whatever.” And my reply is “you probably haven’t seen skilled exploratory testing. Would you like to hear me sing an opera now? OR, I could show you a good example of ET in practice.”

Exploratory testing can be done in an unskilled, slapdash, silly way. Just as a unskilled driver behind the wheel of a car is still a driver who is driving a car, a poor tester can still be doing ET– albeit probably not very well.

The cool thing about ET is that, even done badly, it’s still a great way to find some bugs. Michael and I try to help you do much, much better than that.

The core idea of ET remains as it always has been. It’s been expressed in many different ways, but boils down to this: test design and test execution and learning mixed together in a mutually supportive way. Whenever you see that, and to the degree that you see that, you are seeing exploratory testing.

…Unless you want bad testing.

Claire Moss writes:

I am surprised that you say that scripted testing is harder for novice
testers. I would have expected that having so much structure around
the tests would make getting into testing easier for someone with less
experience and that the scripted instructions would make up for a lack
of discipline on the part of the tester.

Structure != “being told what to do”
First, you are misusing the word “structure.” All testing is structured. If what you mean by structure is “externally imposed structure” then say that. But even if you are not aware of a structure in your testing, it is there. When I tell a novice tester to test, and don’t tell him how to test, he will be dominated by certain structures he is largely unaware of– or if aware he cannot verbalize or control them much. For instance: the user interface look and feel is a guiding structure for novice testers. They test what they see.

Cognitive science offer plenty of ideas and insights about the structures that guide our thinking and behavior. See the book Predictably Irrational by Dan Ariely for more on this.

Scripted testing always has at least two distinct parts: test design and test execution. They must be considered independently.

Scripted test execution is quite a bit more difficult than exploratory testing, unless you are assuming that the tester following the script has exactly the same knowledge and skill as the test designer (even then it is a qualitatively different sort of cognitive process than designing). An exploratory tester is following (indeed forming as he goes) his own intentions and ideas. But, a scripted tester, to do well, must apprehend the intent of the one who wrote the script. Moreover, the scripted tester must go beyond the stated intent and honor the tacit intent, as well– otherwise it’s just shallow, bad testing.

Try using a script to guide a 10 year-old to drive a car safely on a busy city street. I don’t believe it can be done. You can’t overcome lack of basic skills with written instructions.

And sure, yeah, there is also the discipline issue, but that’s a minor thing, compared to the other things.

As for scripted test design, that also is a special skill. I can ask my son to put together a computer. He knows how to do that. But if I were to ask him for a comprehensive step-by-step set of instructions to allow me to do it, I doubt the result would help me much. Writing a script requires patience, judgment, and lots of empathy for the person who will execute it. He doesn’t yet have those qualities.

Most people don’t like to write. They aren’t motivated. Now give them a task that requires excellent writing. Bad work generally results.

Both on the design side and the execution side, scripted testing done adequately is harder than exploratory testing done adequately. It’s hard to separate an integrated cognitive activity into two pieces and still make it work.

The reason managers assume it’s simpler and easier is that they have low standards for the quality of testing and yet a strong desire for the appearances of order and productivity.

When I am training a new tester, I begin with highly exploratory testing. Eventually, I will introduce elements of scripting. All skilled testers must feel comfortable with scripted testing, for those rare times when it’s quite important.

Examples

1. Start browser

2. Go to CNN.com

3. Test CNN.com and report any problems you find.

This looks like a script, and it is sort of a script, but the interesting details of the testing are left unspecified. One of the elements of good test scripting is to match the instructions to the level of the tester as well as to the design goal of the test. In this case, no design goal is apparent.

This script does not necessarily represent bad testing– because it doesn’t represent any testing whatsoever.

1. Open Notepad

2. Type “hello”

3. Verify that “hello” appears on the screen.

This script has the opposite problem. It specifies what is completely unnecessary to specify. If the tester follows this script, he is probably dumbing himself down. There may be some real good reason for these steps, but again, the design goal is not apparent. The tester’s mind is therefore not being effectively engaged. Congratulations, designer, you’ve managed to treat a sophisticated miracle of human procreation, gestation, mothering, socializing, educating, etc. as if he were the equivalent of an animated poking stick. That’s like buying an iPad, then using it as a serving tray for a platter of cheese.

Prabhat Nayak is yet another sapient tester recently hired by the rising Indian testing powerhouse, Moolya. Speaking of the ISTQB syllabus, he writes:

One such disagreement of mine is they have put “Exploratory Testing” on purely experienced based testing. James, correct me if I have got ET wrong (and I am always ready to be corrected if I have misunderstood something), a novice tester who has got great cognizance and sapience and can explore things better, can think of different ways the product may fail to perform per requirement can always do a great job in ET than a 5 years experienced tester who has only learned to execute a set of test cases. That is probably one of the beauties of ET. There is of course, always an advantage of having some experience but that alone doesn’t suffice ET to be put under experienced based testing.

You are quite correct Prabhat. Thank you for pointing this out.

The shadowy cabal known as the ISTQB insulates itself from debate and criticism. They make their decisions in secret (under NDA, if you can believe it!) and they don’t invite my opinion, nor anyone’s opinion who has made a dedicated study of exploratory testing. That alone would be a good reason to dismiss whatever they do or claim.

But this case is an especially sad example of incompetent analysis. Let me break it down:

What does “experience-based” mean?

Usually when people in the technical world speak of something as “x-based” they generally mean that it is “organized according to a model of x” or perhaps “dominated by a consideration of x.” The “x”, whatever it is, plays a special role in the method compared to its role in some other “normal” or “typical” method.

What is a normal or typical method of software testing? I’m not aware the the ISTQB explicitly takes a position on that. But by calling ET an experience-based technique, they imply that no other test technique involves the application of experience to a comparable degree. If they have intended that implication– that would be a claim both remarkable and absurd. Why should any test technique not benefit from experience? Do they think that a novice tester and an experienced tester would choose the exact same tests when practicing other test techniques? Do they think there is no value to experience except when using ET? What research have they done to substantiate this opinion? I bet none.

If they have not intended this implication, then by calling ET experience-based it seems to me they are merely making impressive sounds for the sake of it. They might as well have called ET “breathing-based” on the theory that testers will have to breathe while testing, too.

Ah, but maybe there is another interpretation. They may have called ET “experienced-based” not to imply that ET is any more experience-based than other techniques, but rather as a warning that expresses their belief that the ONLY way ET can be valuable is through the personal heroism and mastery of the individual tester. In other words, what they meant to say was that ET is “personal excellence-based” testing, rather than testing whose value derives from an explicit algorithm that is objective and independent of the tester himself.

I suspect that what’s really going on, here: They think the other techniques are concrete and scientific, whereas ET is somehow mystical and perhaps based on the same sort of dodgy magic that you find in Narnia or MiddleEarth. They say “experience-based” to refer to a dark and scary forest that some enter but none ever return therefrom… They say “experienced-based” because they have no understanding of any other basis that ET can possibly have!

Why would it be difficult for Factory School testing thinkers (of which ISTQB is a product) to understand the basis of ET?

It’s difficult for them because Factory School people, by the force of their creed, seek to minimize the role of humanness in any technical activity. They are radical mechanizers. They are looking for algorithms instead of heuristics. They want to focus on artifacts, not thoughts or feelings or activities. They need to deny the role and value of tacit knowledge and skill. Their theory of learning was state of the art in the 18th century: memorization and mimicry. Then, when they encounter ET, they look for something to memorize or mimic, and find nothing.

Those of us who study ET, when we try to share it, talk a lot about cognitive science, epistemology, and modern learning theory. We talk about the importance of practice. This sounds to the Factory Schoolers like incomprehensible new agey incantations in High Elvish. They suspect we are being deliberately obscure just to keep our clients confused and intimidated.

This is also what makes them want to call ET a technique, rather than an approach. I have, since the late nineties, characterized exploratory testing as an approach that applies to any technique. It is a mindset and set of behaviors that occur, to some degree, in ALL testing. To say “Let’s use ET, now” is technically as incoherent as saying “Let’s use knowledge, now.” You are always using knowledge, to some degree, in any work that you do. “Knowledge” is not a technique that you sometimes deploy. However, knowledge plays more a role in some situations and less a role in others. Knowledge is not always and equally applicable, nor is it uniformly applied even when applicable.

For the Factory Schoolers to admit that ET is endemic to all testing, to some degree, would force them to admit that their ignorance of ET is largely ignorance of testing itself! They cannot allow themselves to do that. They have invested everything in the claim that they understand testing.  No, we will have to wait until those very proud and desperately self-inflated personalities retire, dry up, and blow away. The salvation of our craft will come from recruiting smart young testers into a better way of thinking about things like ET. The brain drain will eventually cause the Factory School culture to sink into the sea like a very boring version of Altantis.

Bottom Line: Most testing benefits from experience, but no special experience is necessary to do ET

Exploratory testing is not a technique, so it doesn’t need to be categorized alongside techniques. However, a more appropriate way to characterize ET, if you want to charactize it in some way, is to call it self-managed and self-structured (as opposed to externally managed and externally structured). It is testing wherein the design part of the process and the execution part of the process are parallel and interactive.

You know what else is self-managed and self-structured? Learning how to walk and talk. Does anyone suggest that only “experienced people” should be allowed to do that?

This is the latest version of the commitments I make when I work with a programmer.

Dear Programmer,

My job is to help you look good. My job is to support you as you create quality; to ease that burden instead of adding to it. In that spirit, I make the following commitments to you.

Sincerely,

Tester

1. I provide a service. You are an important client of that service. I am not satisfied unless you are satisfied.
2. I am not the gatekeeper of quality. I don’t “own” quality. Shipping a good product is a goal shared by all of us.
3. I will test your code as soon as I can after you deliver it to me. I know that you need my test results quickly (especially for fixes and new features).
4. I will strive to test in a way that allows you to be fully productive. I will not be a bottleneck.
5. I’ll make every reasonable effort to test, even if I have only partial information about the product.
6. I will learn the product quickly, and make use of that knowledge to test more cleverly.
7. I will test important things first, and try to find important problems. (I will also report things you might consider unimportant, just in case they turn out to be important after all, but I will spend less time on those.)
8. I will strive to test in the interests of everyone whose opinions matter, including you, so that you can make better decisions about the product.
9. I will write clear, concise, thoughtful, and respectful problem reports. (I may make suggestions about design, but I will never presume to be the designer.)
10. I will let you know how I’m testing, and invite your comments. And I will confer with you about little things you can do to make the product much easier to test.
11. I invite your special requests, such as if you need me to spot check something for you, help you document something, or run a special kind of test.
12. I will not carelessly waste your time. Or if I do, I will learn from that mistake.

(This is cool! Yong Goo Yeo has created a Prezi of this.)

An Important Comment

This comment came in from Stuart Taylor. I think it’s important enough that I should add this to the post:

Hi James,

I’m not sure if this is meant to be a little tongue in cheek, but that’s how I read it. That said, I fell at point 1. “I provide a service” really?

Yes, really. This is not a joke. And point #1 is the most important point.

This implies there may be alternative service providers, and if my customer relationship management isn’t up to snuff, I could lose this “client”. Already I feel subservient, I’m an option.

Of course there are alternative service providers. (There are alternative programmers as well, but I’m not concerned with that, here. I’m making commitments that have to do with me and my role, here.) And yes, of COURSE you may lose your client. Actually that’s a lot of what “Agile” has done: fired the testers. In their place, we often find quasi-tester/quasi-programmers who are more concerned with fitting in and poking around tools than doing testing.

Testing is not programming. Programmers are not testers. If you mix that, you do a sorry job of both. In other words: I am a programmer and a tester, but not a good programmer at the same time that I’m a good tester.

Please meditate on the difference between service and subservience. I am a servant and I am proud of that. I am support crew. I spent my time as a production programmer and I’m glad I don’t do that any more. I don’t like that sort of pressure. I like to serve people who will take that pressure on my behalf.

This doesn’t make me a doormat. Nobody wipes their feet on me– I clean their feet. There’s a world of difference. Good mothers know this difference better than anyone.

Personally, when I work on a software  project, I want to feel like I’m part of the team. I need the developer, and the developer wants me. We work together to bake quality in from the start, and not try and sprinkle it on at the end as part of a “service” that I offer.

What strange ideas you have about service. You think people on the same team don’t serve each other? You think services are something “sprinkled” on the end? Please re-think this.

I want to be on the same team, too. The difference, maybe, is that I require my colleagues to choose me. I don’t rap on the door and say “let me in I belong here! I like to be on the team!” (You can’t just “join” a team. The easier it is to join a team, the less likely that it is actually a team, I think. Excellent teams must gel together over time. I want to speed that process– hence my commitments) Instead, I am invited. I insist that it be an invitation. And how I get invited quickly is by immediately dissolving any power struggle that the programmers may worry about, then earning their respect through the fantastic quality of my work.

You can, of course, demand respect. Then you will fail. Then five years later, you will realize true respect cannot be demanded. (That’s what I did. Ask anyone about what a terror I was when I worked at Apple Computer. If you are less combative or more intelligent than I am, you may make this transition in less time.)

I may have actually agreed with your points before I was exposed to Continuous Integration, because that’s how my relationship used to be; hence me interpreting this as a light hearted piece. However I know that this kind of relationship still exists today (its here at this place). When I began working with continuous integration, the traditional role of the tester, with a discrete testing phase became blurred, as the test effort is pushed back up stream towards the source. If the tester and developer share a conversation about “how can we be sure we build the right thing, and how can we ensure we built it right” before the code gets cut, and the resulting tests from that are used to help write the code (TDD, BDD, SBE), then both of us can have a nice warm fuzzy feeling about the code being of good quality. The automation removes the repetition (and reduces the feedback) to maintain that assertion.

First, your experience has nothing to do with “continuous integration.” Continuous integration is a concept. Your experience is actually related to people. Specific people are collaborating with you in specific satisfying ways, probably because it is their pleasure to do so. I have had wonderful and satisfying professional relationships on a number of projects– mostly at Borland, where we were not doing continuous integration, and where testers and programmers cooperated freely, despite being organized into separate administrative entities.

(Besides, I have to say… Testing cannot be automated. Automation doesn’t remove repetition, it changes it. Continuous integration, as I see it, is another way of forcing your customers do the testing, while placing naive trust in the magic of bewildering machinery. I have yet to hear about continuous integration from anyone who seemed to me to be a student of software testing. Perhaps that will happen someday.)

In any case, nothing I’m saying here prevents or discourages collaboration. What it addresses are some of the unhealthy things that discourage collaboration between a tester and a programmer. I’m not sure if in your role you are doing testing. If you want to be a programmer or a designer or manager or cheerleader or hanger-on, then do that. However, testing is inherently about critical thinking, and therefore it always carries a risk (in the same way that being a dentist does) that we may touch a nerve with our criticism. This is the major risk I’m trying to mitigate.

[Note: Michael Bolton pointed out to me the "warm fuzzy" line. I can't believe I let that go by when I wrote this reply, earlier. Dude! Stuart! If you are in the warm fuzzy business, you are NOT in the testing business. My goal is not anything to do with warm fuzzy thinking. My goal is to patrol out in the cold and shine my bright bright torches into the darkness. On some level, I am never satisfied as a tester. I'm never sure of anything. That's what it MEANS to be a professional tester, rather than an amateur or a marketer. Other people think positively. Other people believe. The warmth and fuzziness of belief is not for us, man. Do not pollute your testing mentality with that.]

My confusion over this being a tongue in cheek post is further compounded by point 2. Too many testers do believe that they own quality. They become the quality gate keeper, and i think they enjoy it. The question “can this go live” is answered by the tester. The tester may be unwilling to relinquish that control, because they perceive that as a loss of power.

Just as excellent testers must understand they provide a service, they must also understand that they are not “quality” people. They simply hold the light so that other people can work. Testers do not create quality. If you create quality, you are not being a tester. You are being an amateur programmer/designer/manager. That’s okay, but my commitment list is meant for people who see themselves as testers.

I guess what i’m fumbling about with here, is that some testers will read this, and sign up to it. Maybe even print it out and put it up on the wall.

I hope they do. This list has served me well.

Either way, you have written another thought provoking piece James, but i cant help wondering about the back story. What prompted you to write it?

Stuart - sat in the coffee shop, but with laptop and wifi, not a notepad

This piece was originally prompted when I took over a testing and build team for SmartPatents, in 1997. I was told that I would decide when the software was good enough. I called a meeting of the programmers and testers and distributed the first version of this document in order to clarify to the programmers that I was not some obstacle or buzzing horsefly, but rather a partner with them in a SERVICE role.

That was also the first time I gave up my office and opted to sit in a low cubicle next to the entrance that no one else wanted– the receptionist’s cubicle. It was perfect for my role. Programmers streamed by all day and couldn’t avoid me.

I think this piece is also a reaction to the Tester’s Bill of Rights (the one by Gilb, not the one by Bernie Berger, although I have my concerns about that one, too), which is one of the most arrogant pieces of crap I’ve ever seen written on behalf of testers. I will not link to it.

And now that I think of it, I wrote this a year after I hired a personal assistant, and two months later had her promoted to be my boss, even though she retained almost the same duties. The experience of working for someone whose actual day-to-day role was to serve me– but who felt powerful and grateful as my manager– helped me understand the nature of service and leadership in a new way.

Finally, a current that runs through my life is my experience as a husband and father. I am very powerful. I can make my family miserable if I so choose. I choose to serve them, instead.

I didn’t see you at the coffee shop, today.

Where were you, tester who told me that you need “certification” because, unlike me, you don’t have a public reputation? Where were you, tester who fears the big machine won’t hire you unless you conform, head bowed, to some lowest common standard? How about you, tester who told me that you dare not stop bullshitting your management about test case counts because “he likes those numbers” and, unlike me, you don’t have the credibility (and therefore the power) to say no without being clapped in chains and thrown into the brig?

I was there as usual, with my hardcover Moleskine and my current favorite pens (the Pigma Micron 01 by Sakura Color Products Corporation and the Mitsubishi Pencil Company Unipin Fineline).

I was working on my intellectual property. What were you doing?

Okay I get it: you were busy with your day job. And when the day is over you are tired and you want to go home. Maybe you have children. Children are important. But don’t tell me you are powerless to change things; that you can’t upgrade yourself. You have the power, but it’s not yet a priority for you. If you have reasonable passion and ambition in this field, you will soar above the problems that so many others feel trapped by. You must take time to develop your craft; educate yourself. Punching the time clock is not enough to obtain a better income and better job security.

One place to start is the Konditori (that’s Swedish for coffee shop… sounds cool and kind of mysterious, don’t you think?).

(This photo, above, is one of two Konditori’s I worked at today, in Stockholm.)

I have to keep developing myself and my intellectual property. It’s the engine by which I feed my family. So, today I worked on Rapid Testing Estimation. I want to roll out my new estimation methodology in time for my next test management class, next week.

(Note: I mostly drink green tea, these days… but it’s Stockholm! You gotta drink coffee in Stockholm!)

But who am I kidding? I work in coffee shops, furiously scribbling in my little notebooks, mainly because I love it. It is practical and effective, sure. Mainly, though, it’s peaceful and… hopeful. I am thrilled to feel that, at any moment, I might experience a breakthrough. And you know those breakthroughs, when they happen, can lead to a new hour or two of cutting edge class material, for which people will pay handsome sums. Last year we had the first conference on Session-based Test Management. My brother and I invented that at a coffee shop (well, it was a Denny’s… that’s pretty close).

We can wipe out bad testing. We can dump commercial certification programs into the dustbin. We can create a powerful craft where testers are well paid and respected. And this is how to do it: Meet me at the Konditori. Bring your Moleskine.

(Only if you love the craft, please. If you don’t, geez man, get a different job!)

The difference between Behavior-Driven Development and testing:

This is a BDD scenario (from Dan North, a man I respect and admire):

+Scenario 1: Account is in credit+
Given the account is in credit
And the card is valid
And the dispenser contains cash
When the customer requests cash
Then ensure the account is debited
And ensure cash is dispensed
And ensure the card is returned

This is that BDD scenario turned into testing:

+Scenario 1: Account is in credit+
Given the account is in credit
And the card is valid
And the dispenser contains cash
When the customer requests cash
Then check that the account is debited
And check that cash is dispensed
And check that the card is returned
And check that nothing happens that shouldn’t happen and everything else happens that should happen for all variations of this scenario and all possible states of the ATM and all possible states of the customer’s account and all possible states of the rest of the database and all possible states of the system as a whole, and anything happening in the cloud that should not matter but might matter.

Do I need to spell it out for you more explicitly? This check is impossible to perform. To get close to it, though, we need human testers. Their sapience turns this impossible check into plausible testing. Testing is a quest within a vast, complex, changing space. We seek bugs. It is not the process of  demonstrating that the product CAN work, but exploring if it WILL.

I think Dan understands this. I sometimes worry about other people who promote tools like Cucumber or jBehave.

I’m not opposed to such tools (although I continue to suspect that Cucumber is an elaborate ploy to spend a lot of time on things that don’t matter at all) but in the face of them we must keep a clear head about what testing is.

Most of the technology of “confirmatory” non-qualitative research in both the social and natural sciences is aimed at preventing discovery. When confirmatory research goes smoothly, everything comes out precisely as expected. Received theory is supported by one more example of its usefulness, and requires no change. As in everyday social life, confirmation is exactly the absence of insight.  In science, as in life, dramatic new discoveries must almost by definition be accidental (”serendipitous”). Indeed, they occur only in consequence of some mistake.

Kirk, Jerome, and Miller, Marc L., Reliability and Validity in Qualitative Research (Qualitative Research Methods). Sage Publications, Inc, Thousand Oaks, CA, 1985.

Viva exploratory methods in science! Viva exploratory methods in testing! Viva testers who study philosophy and the social sciences!

(Thank you Michael Bolton for finding this quote.)

I have reported on Estonia before. Now we’re doing a peer conference, here in Tallinn.

Yes, the Context-Driven testers of Estonia (the ones who focus on their skills rather than “best practices” and “certifications”) are beginning to form an active community. <emperor voice>It is as I have foreseen it!</emperor voice>

The theme of the conference is “My worst testing moment and what I learned from it.” We have heard from:

• Kristjan Uba, who spoke about almost missing a bug in a remote control.

• Aire Kurgpõld, who was mildly disappointed in the rate of bug fixing in her company.

• Madis Jullinen, who experienced boredom (while testing).

• Ervin Hülp, who bought QuickTestPro and then didn’t like it.

• Indrek Kõnnussaar, who had trouble fixing a bug.

(Oh, and I read some items from a list of more than 30 mistakes I’ve made in my career. I couldn’t decide if being fired, almost being fired, or resigning were the “worst” moments. I also told a story about mismanaging testing for Borland’s Turbo Profiler.)

All I can say is that testing worst moments in Estonia are apparently hard to come by. But testers here are pretty young. They can look forward to a long career of mistakes and misfortunes yet to come.

Peer Conference Magic

Once again, I’m seeing the wonderful effect of peer conferencing: where everyone is a speaker, everyone is a delegate, and when you stand up to tell your story, you stay there until EVERYONE is finished questioning you. People get to know each other in a deeper way. Reputations are built. Community is created. You don’t get that from exhibition conferences. You don’t get that from just going to work each day.

When you speak at a peer conference, you are taking a mild but important risk. You can’t hide or run. If you say nothing, that is noticed. If you speak badly, that is noticed. You earn the reputation, pretty much, that you deserve.

I can think of nothing better for the testing world than to hold more of these events. It is excellent medicine against the encroachments of creeping certificationism.

Official List of Participants

• Oliver Vilson

• Madis Jullinen

• Kristjan Uba

• Rasmus Koorits

• Indrek Kõnnussaar

• Ervin Hülp

• Aire Kurgpõld

• Helena Jeret-Mäe

• Raimond Sinivee

• Aare Nurm

• James Bach

• Sergei Sergejev

• Harles Paesüld

Rapid Testing Management in Gothenburg, Sweden.

I will be teaching a public Rapid Testing Management class in Sweden on November 10th-11th. I don’t teach this class often, because it requires that you have already taken the Rapid Software Testing course, first. If you have taken Rapid Software Testing (either from me or Michael Bolton), and you want to run a team of rapid testers, you should consider signing up.

Rapid Testing in Warsaw, Poland.

Yes, a public class in Poland, for the first time. It’s official: April 2-4 in Warsaw. It’s being arranged by Krystian Kaczor.

Rapid Testing in Romania.

The schedule and arrangements are not exactly set, but we have an agreement for me to teach a public class in either Bucharest or Cluj-Napoca at the end of March, 2012. Alex Rotaru is arranging that one.

Rapid Software Testing Practicum in the USA.

My wife, Lenore, has asked me to stay closer to home at least ONCE next year. So I will be leading a special public class on my home island of Orcas, next summer. This will be a Rapid Testing Practicum– an opportunity to practice Rapid Testing with me. Like a camp, I guess. (It would be worthwhile to have taken the RST course, but that will not be required.) We will work together to test a product rapidly and professionally, over several days. I’m thinking of running this as a combination class, with a limited number of participants joining me on Orcas, while others participate to a lesser degree, online, at a much reduced cost.

This has not been done before. It could be a wonderful opportunity to gather solid Rapid Testing experience. I’m thinking I will charge $2,500 for live participants, and $250 for online participants (not counting travel and lodging).

And for what it’s worth, Orcas Island is lovely…

UPDATE: The practicum, which we will call the Bach Rapid Testing Intensive, will take place at Rosario Resort, July 23-27, 2012. I’m creating a registration and information page for it, now….

Below is the mindmap from my keynote speech at the CAST conference. I’m posting it by popular request.

It would take too many words to explain it all. But I’ll answer questions if you have any.