I’m on a new project lately, working with a team at QualiTest. We’re testing a class III medical device. This is an exciting project, because for the first time I am aware of, formalized exploratory testing will be used to do such a validation. We will not rely on masses of procedural test scripts. I’ve been called in on this project because I created the first published formalized ET process in 1999 (for Microsoft), and created, with my brother Jon, session-based test management, which is basically a general form of that Microsoft process.

The QualiTest team consists of senior testers hand-picked for this job, who have regulatory testing backgrounds and an enthusiasm to use their brains while they test. On top of testing well, we have to document our testing well, and trace our testing to requirements. Automatic logging is one of the tools that will help us do that.

I am amazed at how crazy nuts some people get over documentation– how they sweat and shiver if they don’t have a script to cling to– and yet they don’t spare a thought for logging. Logging is great for testers, programmers, and technical support. Logging is automatic documentation. Sing the praises of logging.

I’m talking about function-level logging built into the products we test.

If you test a web app, you already have this (the web server and application logs, plus the use of a proxy to log locally, if you want) or would have it with a little tweak here and there by the programmer. For desktop apps, the programmer has to build it in. Here’s why he should do that right away:

1. Instead of following a script written weeks or months ago by some over-literal, function-besotted and data-blind intern, the tester can think, explore, play, and maintain the thread of inquiry without worrying that you won’t know what you tested, later on.
2. Instead of remembering what you tested, the product tells you how you tested it. Process the log with a simple Perl script, and you can potentially have an automatically generated test report.
3. Instead of just wondering how you made that crazy bug happen, the developer can consult the log.
4. Instead of asking the customer what he was doing moments before the crash, he asks for the log.

If logging is built into the base classes of the product, very little coding is involved.

This idea first occurred to me in 1993, after hearing from John Musa about how his telecom systems would “phone home” with data about how they were being used, but I couldn’t get a programmer to put logging into anything I tested until I was at SmartPatents in 1997. Since then I’ve helped several projects, including a couple of medical device projects, get going with it.

On this most recent project I was asked to create requirements to specify the logging. Here is the generic version of what I came up with:

1. Each significant action that the user takes shall be logged. (pressing buttons, touching screen objects, turning knobs, startup and shutdown, etc.) This provides critical information needed to demonstrate test coverage during validation, and improves our ability to meet and exceed regulatory requirements.

2. The results of any diagnostic self-tests or assert failures shall be logged.

3. Any function should be logged, regardless of user action, that causes a change to data, screen display, system configuration, modes or settings, communicates with other equipment, or produces an error or information message.

4. Everything that could be interesting and useful for testing, support, and system debugging should be logged UNLESS the event occurs so frequently (many times a second) that it poses a performance or reliability risk.

5. Each log event shall include at least the following information:
- Time stamp: For instantaneous events, time stamp (millisecond resolution). For events over time log the start and stop times by logging it as two separate events (e.g. “Event START”, “Event END”). Events that set a persistent mode or state can be logged as one event (”high security mode ON”) but the state of any such modes shall be automatically logged at startup and shutdown so that a complete record of that setting can be maintained over time.
- Event type ID: always unique to event type; IDs not re-used if an event is retired and a new event is created.
- Event type description: short, unique human readable label
- Event information: any data associated with the event that may be useful for customer service or assessing test coverage, this data may be formatted in ways specific to that event type.

6. At startup and shutdown, the current settings, modes, and confuguration shall be recorded to the log.

7. Any errors shall be recorded to the log, including the actual text of the error message.

8. Every type of loggable event shall be stored in one table in the source code or in a data structure accessible on the system itself, such as a header file, enum, array or resource file. This facilitates providing the validation and customer service teams with a complete list of all possible events.

9. The log format shall be in text form, structured and delimited consistently such that it can be parsed automatically by a third party tool. The data for each event should be on one line, or else be marked with standard start and end markers.

10. The log format should be structured and delimited such that it is reasonably human readable (such as tab delimited).

11. The level of detail included in the log file should be configurable in terms of preset levels: 1- error and service events only, 2- Functional events, error events, service events, 3- All events including diagnostic information messages about internal states and parameters.

12. The log should behave as a ring buffer with a maximum of X events (where X is configurable within some limit that would not be exceeded in 7 days of heaviest anticipated use). If the size of the log exceeds available space, the oldest events shall be discarded first.

13. When the log is exported, it should have a header that identifies the software version (and serial number of the HW, if applicable) and current configuration.

My brother and I are experimenting with short podcasts. Here are the first two 15-minute segments:

• Testers say the darndest things. One issue in coaching testers is getting them to speak carefully about evidence and about what they can and can’t do. For instance, we talk about how we react when someone tells us that “a tester ensures the quality of the product.” Jon’s strategy is to point out the incompleteness of testing. Testing can’t be perfect, therefore testers can’t ensure quality. My strategy is to focus on the control aspect: testers aren’t in control of the project, and therefore can’t ensure anything. We think that part of the reason testers make over-the-top statements is that it’s comforting to cling to simple fairy stories about testing.

• The trap of not asking questions. Jon and I notice that testers presented with challenges and problems often jump right into test execution instead of asking questions. We question this phenomenon.

One of the things I like to do is give testing lessons over Skype. I offer this as a service for pay, and I also do it for free under certain conditions.

Here’s how it works:

1. You contact me over Skype. (ID: SATISFICE)

2. If I’m not charging you than it’s on an “as available” basis: How do you know I’m available? Well, if I’m online, then Skype me and ask if I’m available. For free lessons, I’d rather not schedule anything in advance.

3. My Motivation: I am motivated by people who entertain me with their passion for learning. Besides that, I want learn how to coach better. Finally, I’m developing material for another book. But here’s my business reason: I want to publish the edited transcripts of my coaching sessions. So, I need from you permission to include the session in my writing (with your name attached or removed as you wish).

4. The process is Socratic dialog: That means I ask you questions and pose problems, then you develop your answers. I also instruct directly as needed. My goal is to help you develop into a tester worthy of respect according to the standards of the Context-Driven School of Testing.

5. We work on things that matter to you: If you have a goal to learn a specific kind of thing, we work on that. Otherwise, I’m happy to walk you through my own syllabus of testing.

6. I may require you to do homework as a condition of further coaching. I may ask for permission to publish your homework as a training example.

7. I may want to watch you test over Skype screen sharing.

8. If you are a beginner, I may require you to work with another coach first (one of my students or colleagues) as a condition of working with me.

9. Our sessions are not secret unless you want them to be. You are free to tell others that you are working with me.

10. If you ask me a specific question, I will probably ask you to tell me your answer, first. I will expect you to have googled around and thought about it some before we talk.

You should bear in mind that I’m a demanding tutor. I want to spread clarity and excellence in software testing. Don’t approach me unless you have an ambition to be excellent in your craft.

My offer is open to anyone in the world, as long as I believe I can help you.

I recently got back from the Oredev developer’s conference, in Malmö, Sweden. I enjoyed it, though I wish I’d had more time to actually go to the various developer-oriented talks. The closest I got to that was being on this panel:

This is probably the most fun panel I’ve been on. I loved the enthusiasm and wit of my fellow panelists.

(Editor’s note: oops, I originally  put the beginning of another blog post down here. I’m taking it out to put it into its own entry.)

My work is greatly influenced by fellow students of the craft. My friends are critical to my education.They help me focus my attention and they stir up my ideas.

Here are some substantive examples from just the last two days in my life as a student of software testing:

• David Gilbert Skyped me with a new version of his TestExplorer tool. I’m using it to develop examples for my next book, tentatively titled “Real Exploratory Software Testing” to contrast with the shallow knock-off of an ET book out there that has a similar name.
• Michael Bolton roused me out of bed to talk about a change he wants to make to the Rapid Testing Class.
• Pradeep Soundararajan Skyped me to talk about Weekend Testers.
• Ajay Balamurugadas posed me a math problem, and I wrote him about the solution.
• Parimala Shankaraiah emailed me asking for a testing challenge to pique the skills of the Weekend Testers.
• Liz Marley blogged wonderfully about learning challenge I gave her, and I remembered that I have to follow up on my part of that challenge. She also offered a good example of using emotions and a test oracle.
  
• Karen Johnson Skyped me about the protocol of collegial critique between us. She reminded me that I am suppose to critique her work. I promised to.
  
• Matt Heusser Skyped me about writing an article for him. We’re doing it on his wiki. First time I’ve written an article that way.
• Sohail Hasware Skyped me to complain about the stupid testing curriculum he’s forced to go through to get his IT certificate. I once again tried and failed to get him to quit that program and stop wasting his precious time and life consorting with fools.
• Buddy Parker, a 14 year-old home schooled kid also living on Orcas Island, emailed me about how to learn HTML, so I responded with links and some examples.
• Adam Roy, a 7 year-old kid who likes algebra, son of author Jennifer Roy, conversed with me about learning Python.
  
• Ravisuriya continued an email conversation with me about context-driven testing and priority-severity distinctions.
• Adrian Segar and I conversed about the etymology of the term “Peer Conference.” Apparently he’s written a book about it.
• Rodney Thompson emailed me to sound me out about a new term: “situational testing.” I’m dubious about it, but we’re still talking.
• Jonathan Bach alerted me to a new article attacking “best practices”, so I commented on that. He also wants a new article from me.
• Elisabeth Hendrickson offered an apt and testerly reply to my puzzle “guess the next term in the series: tom, dick, ?” She wrote “Bill, if the sequence is from Thomas Richard Williams, pioneer of stereoscopy.” I was thinking “harry”, based on a common cultural pattern, but she demonstrates that other answers are plausible, as a good tester should.
• Henrik Andersson Skyped me about how he might be coming to the U.S.A. at the special invitation of the Association for Software Testing. Henrik is a rising star as a Context-Driven guy in Sweden.
  
• Jerry Gao, a tester from China, wrote to ask how to advocate for exploratory testing over there. I’m replying to him, shortly.
  

These are all happy examples. I’m not leaving anything out. I just don’t happen to be in a flame war with anyone, at the moment.

And thank you Kathy…

I want to mention the most recent useful criticism I’ve received from a colleague. It was from Kathy Iberle, at the PNSQC conference. I had just delivered a presentation I felt good about, on the “myths of rigor.” Kathy said she was expecting me to provide more details and analysis about the alternative approach to rigorous engineering that I had proposed. She said, “I expect to hear something new when I hear a talk from you.” Well, actually, I thought I had presented something new, but as I thought over it again, I realized that I’d left some big fat loose ends in the talk. So, just as I was about to argue with her, agreement stepped in and threw a blanket over my cage.

Kathy is an interesting example of someone who has gained her reputation in my community entirely without– I want to say trying, but that’s not right– Kathy doesn’t push. It’s actually frustrating to me, who does push. I like turning over bee hives to get the honey, but with colleagues like Kathy, I have to wait patiently for the bees to fill tiny thimble jars and fly by with them. What honey, though!

The main reason I know about Kathy is from seeing her at peer conferences. That’s the single best mechanism I can recommend for building mass collegiality.

(BTW, my Skype ID is “satisfice”)

There’s something important going on in India. It’s called Weekend Testers. It seems to be the beginning of a truly Indian version of the Context-Driven School of Testing. It’s a sapient testing insurgency!

If you are a tester in India (or you aspire to be) and you want to be a GOOD tester, then you should get involved with this or start something like it yourself.

I taught in India, years ago, planting seeds at Wipro and Mindtree. Michael Bolton has taught there a few times, since. I’ve been working with Pradeep Soundararajan and Shrini Kulkarni for a while. I just discovered that Matthew Heusser has also been mentoring some of them, too. I’ve blogged about other Indians who test. But this is the first organized community of skill-bearing software testers I have seen in India.

I hope it’s the beginning of something big. You deserve it. Plus… when there’s enough of you, and some of you get into management, perhaps you’ll invite me to come out and teach again. I’d like to go back for more.

Michael Bolton, my brother Jon, and I have produced a new version of our Exploratory Testing Dynamics document. We unveiled it last week at STPCON.

This document describes the elements of exploratory testing as we currently understand them. This new version has not yet been reviewed by Cem Kaner or any of our colleagues who normally have opinions on this, but I don’t anticipate that it will be revised much when they do. It should be fairly stable, for now.

ET cannot be summed up fairly as “unstructured playing around” and if you look at this document you will see why. If nothing else, count the elements in it. Crikey.

The document consists of four lists:

1. Evolving Work Products

Exploratory testing is an approach that emphasizes evolution. We start with something and make it better and better. Often testers can be overly focused on finding bugs, but look at all the things that get better. For instance, you can run your automated regression tests a thousand times and the tests don’t learn and neither do the button pushing testers. But you learn a lot by sapiently retesting– you evolve. One of the products of ET is a better tester.

Some evolution also happens in highly scripted testing, but in ET it’s a primary goal. It’s the point.

2. Skills and Tactics

What does an exploratory tester do? Well, feast your eyes on the Skills and Tactics list. Each of those items are things that seem to us to be reasonably independent, teachable, and observable. Some of them are also skills that a highly scripted tester would need, but they work a little differently for explorers. For instance, the skills of reading and analyzing documents means doing that to prepare for or aid exploration of the product, rather than to prepare detailed instructions for scripted test execution.

3. ET Polarities

The Polarities list is unique to ET. An exploratory tester must manage his own mental process, and that process is about developing ideas and using those ideas to search the product for trouble. We’ve found over years of experience doing this that a tester loses steam if he spends too long in one train or type of thought. So, what can he do?

One answer is alternation. We alternate among complementary activities. Two activities are complementary if performing one of them rejuvenates the other. A great example is reading about a product and directly interacting with it.

4. Test Strategy

The Heuristic Test Strategy model is a set of guideword heuristics that we publish as part of the Rapid Software Testing class. In addition to the general ideas of exploration, evolution, and alternation, the guidewords represent the specific subject matter of testing– the conditions we need to succeed, the things we look at, the things we look for, and the specific test techniques we use to produce tests.

With these lists, I can systematically evaluate a tester and identify strengths and weaknesses. We in the Context-Driven School of testing need models like this, because we focus on skills, rather than techniques and tools. This document allows us to focus our efforts and develop specific exercises for tester training.

An open letter to James Whittaker:

You wrote: “I had an amicable hallway conversation with James Bach. His blogger angst at my use of the title ‘Exploratory Testing’ didn’t spill over to a face-to-face discussion. Frankly, I am not surprised. I’ve never claimed the term as my own, I simply took it and made it work in the hands of real testers on real software under real ship pressure. Consultants can coin all the terms they want, but when us practitioners add meat to their pie, why cry foul? Is it not a better reaction to feel happy that there are people actually doing something with the idea?”

None of that is true.

I would not describe our conversation as amicable. Perhaps you thought it was amicable because we didn’t talk about anything important, and during that moment, I didn’t raise my voice. Or punch you.

My criticism of you is not “blogger angst”, it’s my opinion based on studying for 20 years something you’ve hardly studied at all. Every substantive conversation we’ve had has consisted of you denying whatever I happen to say, without offering evidence and in most cases without offering an argument. You have a zeal for dismissing my work that is truly extraordinary– you once even denied, again without evidence, that I knew how to run a file compare tool. Wow.

Now you say you made ET work? Well, first, you don’t know what ET is. Second, you’re an academic. You stayed in school and studied formal methods (that no one uses) while I was cutting my teeth in Silicon Valley. I have taught and demonstrated ET all over the world. I’m not alone, but work with a community of like-minded testers and thinkers, comparing notes with them, and deepening our understanding of exploratory learning applied to testing. You have not been a part of that.

I don’t think you’re adding meat, I think you’re serving thin gruel.

ET does work. My community repeatedly shows that it does. We will patiently continue to teach and develop it.

Subtlety hasn’t worked with you. So I’m saying this publicly: You’re a good speaker, but as a practitioner, if your prose is any indication, you don’t know much about what you are doing. If you applied yourself, you could become a good tester. But I’ve seen no evidence of that, yet.

When Cem Kaner introduced the term “exploratory testing” in the mid-80’s, everyone ignored it. When I picked up the term and ran with it, I was mostly ignored. But slowly, it spread through the little community that would become the Context-Driven School. I began talking about it in 1990, and created the first ET class in 1996. It wasn’t until 1999 that Cem and I looked around and noticed that people who were not part of our school had begun to speak and write about it, too.

When we looked at what some of those people were saying, yikes! There was a lot of misunderstanding out there. So, we just kept plugging along and running our peer conferences and hoping that the good would outweigh the bad. I still think that will happen in the long run.

But sometimes it’s hard to stomach how the idea gets twisted. Case in point: James Whittaker, an academic who has not been part of the ET leadership group, and also has little or no experience on an industrial software project as a tester or test manager, has published a book called Exploratory Software Testing.

Whatever Whittaker means when he talks about exploratory testing is NOT what those of us mean who’ve been working on nurturing and developing ET for the last 20 years. As far as I can tell, he has not made more than a shallow study of it. I will probably not write a detailed review (though his publisher asked me to look at it before it was published), because I get too angry when I talk about it, and I would rather not be angry. But Adam Goucher has published his review here.

Another guy who shows up at the conferences, B.J. Rollison, also gets ET wrong. He’s done what he calls “empirical research” into ET, at Microsoft. Since he, again, has not engaged the community that first developed the concept and practices of ET, it’s not altogether surprising that his “research” is based on a poor understanding of ET (for instance, he insists that it’s a technique, not an approach. This is similar to confusing the institution of democracy with the mechanics of voting), and apparently were carried out with untrained subjects, since Rollison himself is not trained in what I recognize as exploratory testing skills.

Experimental research into ET can be done, but of course any such work is in the realm of social science, not computer science, because ET is a social and psychological phenomenon. (see the book Exploring Science, for an example of what such research looks like).

Now even within the group of us who’ve been sharing notes, debating, and discovering the roots of professional exploratory thinking in the fields of epistemology and cognitive psychology and the philosophy and study of scientific practice, there are strong differences of opinion. There are people I disagree with (or who just dislike me) whom I still recognize as thoughtful leaders in the realm of exploratory testing (James Lyndsay and Elisabeth Hendrickson are two examples). Perhaps Whittaker and Rollison will become rivals who make interesting discoveries and contributions, at some point. Time will tell. Right now, in my opinion, they are just skating on the surface of this subject.

I told James Whittaker that I don’t use the term “manual testing” and that I prefer the term “sapient testing” because it’s more to the point. This is evident in the first definition of the word “manual” in the Oxford English Dictionary: ” 1. a. Of work, an action, a skill, etc.: of or relating to the hand or hands; done or performed with the hands; involving physical rather than mental exertion. Freq. in manual labour.“ Sapient, on the other hand, means “wise.”

Whittaker laughed and said “Bach, you are always making words up.” And then told me that in his opinion manual testing did not evoke the concept of unskilled manual labor. Now, other than establishing that James Whittaker doesn’t have an online account to the O.E.D. (definition of “sweeeeet!” is “an online O.E.D. account.”), or perhaps doesn’t consider dictionaries to be useful sources of information about what words mean, I see something else in his reaction: I blew his mind. What I said doesn’t intersect with anything in his education.

To understand me, Whittaker will have to use his sapience, rather than responding manually (i.e. with his hands).

In other words, I notice that some of my rivals in the testing industry don’t merely disagree with me, they apparently don’t comprehend what I’m saying. Example: after some ten hours of solid debate with me, over several sittings, Stuart Reid (who is working on a software testing standard of all preposterous things), told a colleague of mine that he believed I don’t truly mean the things I said in those debates, but merely said them to “be provocative.” Huh. That’s some serious cognitive dissonance you got going, Stu, when the only way you can process what I’m saying is to declare, essentially, that it was all just a dream.

Of course, I don’t think this is an intelligence problem. I think this is a lack-of-effort-to-use-intelligence problem. It’s not convenient for certain consultants to tear up their out-of-date books and classes and respond to the challenge of, um, the last 30 years of development of the craft. So they continue to teach and preach ideas from the seventies (or create testing standards based on them, because they believe not enough people appreciate testing disco).

Anyway, in the Context-Driven community’s latest attempt to explain the ins and outs and vital subtleties of testing, Michael Bolton has come up with a promising tack. Maybe this will help. He’s drawing a distinction between testing and checking.

Brace yourselves for insight. A lot of what people call testing is actually mere checking. But even checking requires testing intelligence to design and do well. This gives more specifics to my concept of sapient testing. Here are Michael’s seminal posts on the subject:

1. http://www.developsense.com/2009/08/testing-vs-checking.html
2. http://www.developsense.com/2009/09/transpection-and-three-elements-of.html
3. http://www.developsense.com/2009/09/pass-vs-fail-vs-is-there-problem-here.html
4. http://www.developsense.com/2009/09/elements-of-testing-and-checking.html

When Michael first made the distinction between testing and checking, I was annoyed. Truly. It blew my mind in that bad way. I thought he was manufacturing a distinction that we didn’t need. I decided to ignore it. Then he called me and asked “So what do you think of my checking vs. testing article?” I had to say I didn’t like it at all. We argued…

…and he convinced me that it was a good idea. Thank you dialectical learning! Debate has saved me again!

I now agree that it’s a practical distinction that can be used as a lens to focus on the quality of a test process. I do have to get used to the words, though. I now see a difference between automated testing and automated checking, for instance: automated testing means testing supported by tools; automated checking means specific operations, observations, and verifications that are carried out entirely with tools. Automated testing may INCLUDE automated checking as one component, however automated checking does NOT include testing.

Making this distinction is exactly like distinguishing between a programmer and a compiler. We do not speak of a compiler “writing a program” in assembly language when it compiles C++ code. We do not think that we can fire the programmers because the compiler provides “automated programming.” The same thing goes for testing. Or… does that blow your mind?

Context-driven testing is apparently very difficult to understand. I wouldn’t have thought that it’s a difficult concept, except for the last decade I’ve watched hundreds of people in terrible confusion. Testers outside of the United States have an especially hard time with it, for some reason that escapes me.

I do have some working theory of the misunderstanding. I think it has a lot to do with how so many people are looking for first-order formulaic answers to every problem. Context-driven thinking rejects that. We say you can’t do quality work unless you develop skill. There’s no secret phrase or recipe book that will grant you skill. You must develop skill by struggling with testing problems.

India has a lot of testers, but so very few have shown publicly that they can think outside the “best practices” and Factory School kiddie pool. But, I’m happy to say, Parimala Shankaraiah is one of them. I’m adding her to my blogroll. Her blog is called Curious Tester. (It’s nice to be able to honor a female tester for a change.)

Bear in mind I have no idea if she thinks of herself as context-driven. She hasn’t mentioned it directly. I believe she is, based on how she writes.

Several things strike me about Pari. One is her sheer enthusiasm. Her love of testing is infectious. Another is the breadth of her imagination. In one post she related the book The Secret to her experience as a tester. That was impressive analogizing! Almost acrobatic. She believes in peer conferences and she opposes certification, too. Those are important indicators of a context-driven sensibility.

I have often said that I expect the Indian testing industry to wake up someday (at least a few thousand of them) and realize that they can do a lot better than writing dimwitted scripted test cases. Pari is hopefully just the beginning of the Indian testing renaissance.

When my publisher decided to release my Secrets book for “free” as an ebook, I thought it meant FREE. I just found it meant free-for-a-little-while-and-then-gone. Apparently it has some sort of DRM expiration date on it.

[Update: One of my tester friends found that he could defeat the expiration mechanism by playing with the system date, but that's not very practical, I guess.]

They didn’t mention this to me when they told me about the promotion. I suppose they thought “Well OBVIOUSLY it’s only free for a period of time… we’re in the publishing business!” But it wasn’t obvious to me or I would have told you when I announced it.

Anyway, you can still download the book until the 24th of July, and you can read it for some period of time after that before it goes POOF. Not bad, considering that it didn’t cost you anything except the annoyance of downloading it, but still…

Naturally, I hope everyone goes out and buys the hardcover with real money. That way I’ll be encouraged to write more books. Otherwise, it’s an expensive hobby.

Side note: It’s interesting how glitchy the process is, too. Several people found that the special Adobe reader software complained and moaned and wouldn’t install on their systems. QUALITY IS DEAD!

Secrets of a Buccaneer-Scholar is now available in electronic form. You can get it from Simon & Schuster, or less expensively on the Kindle from Amazon.com.

The book is about learning. It’s about alternatives to schooling and certification. But it also describes how I became a tester, and why I progressed as a tester.

The hardcover version, which will be quite nice and small enough to smack people with and not do permanent damage, will be available in September.

The CAST testing conference is happening in Colorado Springs, July 13-16. I mention this for two reasons:

1. I will be teaching a testing tutorial there. I also will be wandering around with my various testing games and challenges hoping to do them with anyone who wants to see what I mean by “testing skills.”

2. CAST is the only conference that is organized in the true spirit of the Context-Driven School of Testing. It’s kind of our home conference. If you find my thoughts interesting, or those of Cem Kaner, Michael Bolton, Ben Simo, etc. Then you’ll want to be there, if you can.

This comes from an ISTQB advertisement they spammed me with, today:

“To ensure the quality of any software system, testers and QA professionals must thoroughly test the product. But how do you know that these tests are effective? If your team is conducting ad hoc, informal tests with little guidance or planning, the quality of the end product can be severely jeopardized—negatively affecting your bottom line.”

I don’t like to say things like this, nor am I comfortable supporting people who do. It’s not that it’s untrue– it is not necessarily untrue. But it is the kind of statement that fans the flames of a certain sort of Factory School bigotry in our industry. “Oh, you can’t trust testing unless it is pre-planned, pre-packaged, pre-approved, formalized, etc.”

Notice they say nothing about skill. It’s all about methodology, here, not skill. This kind of setup suggests that the next statement will be about the importance of factory-like test methodology. But that’s not what happens.

“The best way to be certain that you are providing customers with quality software is to make sure your team of testers is certified.”

Friends, I’m aware of no one in the industry– not even my worst enemies, not even Rex Black or Stuart Reid– who would publicly assert this or defend it. In fact, in debates against those of us who think certification is consumer fraud, the most typical move is for certificationists to say that certification isn’t even about skill, but rather about basic knowledge. “It’s a start” they say. “It’s a foundation.” (I reply that it’s a bad start and a bad foundation. Much worse than what it tries to replace.)

But then they allow this sort of advertising to go out! Completely undercutting their innocent-sounding plea! And they wonder why I complain that they don’t have the best interests of the testing craft at heart.

Notice that the “ad hoc and unplanned” stuff doesn’t even logically connect to certification. In fact, wouldn’t a highly skilled tester be far more likely to succeed with an ad hoc testing regime? When Roger Federer plays ad hoc tennis, I bet he still wins.

I think the reasons they start talking about methodology and end up talking about certification is A) their potential customers don’t understand the difference between skill and method, B) method is more concrete than skill, thus easier to evoke, and C) they know that what they say doesn’t have to be true or even logical, as long as it evokes horror and promises hope.

Oh, but there’s more…

“By taking the Software Tester Certification course and earning an internationally recognized certification in software testing, your team will gain the expertise needed to handle your greatest testing challenges; earn credibility and recognition as competent quality assurance professionals; and provide greater value to your organization.”

It’s internationally recognized? By whom? Some people who don’t study testing and some people who study testing and financially benefit from certification. Okay, but it is also internationally ridiculed by serious testers of many nations who wish to raise themselves to a level of skill that can’t be obtained in just a couple of days of training.

I recently encountered Dot Graham, now semi-retired, who told me that it hurts her feelings when people like me suggest that certificationists are only in it for the money. Dot is a sweet person. I don’t want to hurt her feelings. But I point her to advertising like this and I challenge her to explain it in any other terms. If not greed, then what, Dot? Stupidity? Pride?

Dot doesn’t want to argue with me about this. Of course she doesn’t. Rex Black doesn’t want to argue, either. Naturally. What answer could there be? Lois Koslowski once told me that “big dogs” don’t need to debate (in fairness, Lois Koslowski claims not to be a tester. I agree that she showed no testing competence or knowledge in the conversation we had. I just mention her because she did claim to be in charge of the ASTQB organization. Yikes!) This is capitalism in its ugly form– harvesting the ignorance and fear of others. Debate has no place here.

Is there no one in that self-declared professional community who reviews the advertising and stands up for professional temperance and humility?