What follows is a somewhat grumpy argument against tester certification programs. I have mixed feelings about writing this, because I know a lot of otherwise friendly people who are involved in certification. I know there are a good many organizations committed to certification. I will probably lose some business because I’m going on record opposing it. What I hope, of course, is that I will gain as much business as I lose. My ideal client is someone who wants straight talk, rather than only happy talk.
Please keep this in mind, as you read: my opinion is based on experience, yes, and it is based on my feelings and priorities and reasons. But it is also formed through the dialectic process of considering what other people have to say. I invite you to respond.
Please do not support bad tester certification programs. If you are already certified, please don’t take it seriously or expect other people to take you seriously because of it. Thank you.
I know something about this. I was involved in producing and approving the original body of knowledge for the Certified Software Quality Engineer program for the American Society of Quality, in 1995. By the end of that process, I could not endorse the program.
I have reviewed the syllabi for ISEB and ISTQB as well as the SWEBOK. I have debated these syllabi with some of the people who helped create them. Attempts have been made to recruit me into two other certification attempts, both of which I turned down.
I’m a big fan of merit. I do not have a high school diploma. (For those of you into etymology, “diploma” refers to a folded piece of paper.) I left school because I could not put up with the certification mentality of academic life. It’s an empty-headed way of trying to fill heads. It’s soured by petty politics. I am in this industry because it rewards primarily merit. I think the current crop of tester certification programs threaten that, in much the same way that the Capability Maturity Model has stalled and withered discussion about real software process improvement in the government and military sectors of the software industry.
Because I keep getting asked about this, I’ve laid out my arguments, below.
“No Single Community” Argument
- Certification is a community phenomenon. Certification is simply a clarification of community membership. Nothing wrong with that, except when the certifying agency does not actually represent the community.
- I imagine, for some occupations, there are well-established and internationally recognized organizations that speak for those occupations. Not so with software testing.
- There are many communities within the testing industry. These communities have different ideas about testing; differing values and vocabularies. Some people say there is a consensus about “good practice” in testing. There isn’t. There is no process to determine consensus. There has never even been a serious attempt to form such a consensus. (A few friends getting together to agree on practices hardly counts as an industry consensus.)
- Most people who do testing for a living don’t take classes or read books. They don’t go to conferences. They are not community activists. (This impression is based on the informal polls I take in my corporate onsite testing classes and by my conversations with people who create certification programs.) Yet they may well be able to test software effectively. There is little outreach to such testers by the testing activists. The experience and creativity of most testers is therefore not being harnessed in any systematic way by people making up certification programs.
- Sometimes people tell me that there is no real controversy among testing thinkers about the true basics of testing. Then I argue with them for an hour and see what happens. I am a living existence proof of controversy, since my rapid testing methodology rejects much of traditional testing folklore. How people react to me reveals how they define their community: those who dismiss my ideas are telling me I’m not a Citizen of Testing, and thus they preserve their consensus by banishing those who do not give consent. Through the liberal banishment of anyone who disagrees, consensus can be achieved on any topic whatsoever.
- Although certifying agencies can speak only for their own organizers, their ideas are too often taken seriously by people who don’t know any better. This distorts the great conversation and debate about what testing is and should be. People who are not testing afficiandos don’t know that the testing industry is fragmented. They don’t know that certification programs don’t represent consensus. Because they don’t know, they tend to assume that all the tester certification programs are pretty much the same, and that the certifying agencies are authoritative, and that people who are not certified must not know much about testing.
- An excellent certification program would have to be based on a comprehensive study (not just a survey or opinion poll) of testers in the field and in a variety of technology sectors.
- I cannot support a tester certification program unless it identifies its community, studies its community, and acknowledges the existence of other communities. That’s why I call myself a member of the Context-Driven School of testing and that’s why I give names like Factory School or Quality School to other factions in the testing world who have refused to name themselves.
“Bad Test” Argument
- A certification process is a testing process. I believe in good testing. Therefore, I look for a certification program that effectively tests a candidate tester for relevant qualities.
- I am interested in the ability of a tester to test; testing competence. The ability to remember word definitions and pat answers about oversimpified testing situations is not the same as testing competence.
- An exam that focuses on the way words are used is therefore a poor test of testing competence. Yet, there are no skill-based tester certification programs.
- Some people who agree with me that certification exams are a poor test of competence believe that it is at least a test for interest and commitment level. That’s fine, but there are also many other ways to demonstrate interest and commitment, and better ways too, since testing certification requires so little skill to acquire.
- I am aware of no tester certification program that actually guarantees or even indicates the quality of the tester. It has not been my experience that certified testers, of any stripe, perform any better in my testing classes (which include hands-on testing exercises) than non-certified testers.
- There are better tests available: a university degree in computer science, philosophy, psychology, law, math, electrical engineering or even music. Pretty much any university degree deserves far more credibility as a certification for testers than the pathetic quicky classes that prepare people for tester certification.
- Certification exams do not measure the quality of a tester. Until they do, they merely facilitate discriminatory hiring practices.
“Chilling Effect” Argument
- Tester certification programs of ANY kind (even really good ones) necessitate a narrowing of views about what constitutes testing. The certification program designer must make choices about what’s in and what’s out.
- The mere existence of a testing “Body of Knowledge” or syllabus has a chilling effect on the development of new testing ideas that enhance or transform that notion of knowledge. This is because people will just use the syllabus instead of rethinking it, on the assumption that it is bad to rethink an answer when the answer is already known. But in an immature craft like testing, the mere existence of a syllabus does not mean that the problem of defining the craft or the skills of the craft has been solved– even for the community it purports to serve. We need experience with a variety of models of the craft and we need to test those models.
- A second aspect of the chilling effect is that committee politics interferes with innovation.
- A third aspect of the chilling effect is that vested interests, such as consultants who develop courses, have want to keep the status quo. Rejecting change lowers course development costs.
- I believe that any attempt at certification, especially if it is done by a community steeped in traditional testing folklore, risks retarding our progress toward a better future as a respectable discipline.
“Folklore is a Bad Foundation” Argument
- Every syllabus I have seen is just a collection of folklore; paved cow-paths of popular testing mythology. Where is the critical thinking about this folklore? Why should we be satisfied with hearsay as our primary research method? I won’t take the space here to go into detail on specific examples, such as the boneheaded way boundary testing is taught, or the moronic principle that all tests must have a prespecified expected result, or that finding a bug late is more expensive than finding it early. There is just too much faith and not enough critical thinking in our craft.
- I believe that true excellence in testing is not about memorization or promotion of testing folklore, but rather about general systems thinking, epistemology, and the philosophy and methodologies of empirical research.
- The testing communities I have encountered, other than my own, almost without exception, express either indifference or contempt for the cognitive processes of excellent testing. The exceptions I have encountered are only among testing communities that don’t realize that they are testing communities (such as decision theorists or artificial intelligence researchers).
- I believe that even my own community has only just begun the kind of scholarship required to develop a robust idea of how to train and assess excellent testers. We are lurching awkwardly forth, but we are making progress.
- Testing folklore has some value, but it is no basis upon which to declare ourselves a mature and functional craft.
“Exploitation is Bad” Argument
- I want my actions to contribute maximally toward the better future of testing. I want to help other people contribute creatively to the craft.
- I observe that the motivations driving certification programs are mainly economic, rather than being rooted in a desire to improve the testing field. Consultants find that they can easily sell classes that are tied to certification requirements. I’ve been in conversations with such people, before they knew I was against certification, as I listened to them tell me what a gold mine certification is for them.
- Besides economics, some people who push certification are motivated by a desire for greater influence and respect than they would otherwise receive from their peers if they had to stand intellectually naked and alone and justify their thinking. I know a good many people– even people who speak at testing conferences and write books– who in my opinion have little knowledge or competence as testers.
- There are a lot of people out there ripe to be exploited, including novices who want an easy way into the craft, or managers who want an easy formula for hiring testers.
- I feel bad about these things because I have gone through a difficult and long process of working out my own ideas about testing and grounding them in the history and traditions of organized thinking. When the get-rich-quick personalities set up shop as testing experts without any grounding other than the kind regard of foolish friends, I get depressed.
- I am against certification when certification requires exploiting the ignorant.
“No accountability” argument
- You can’t sue a tester for malpractice, because no testing certification, under the law, establishes a true profession.
- What happens to the certified tester if he does a bad job? Nothing. Does he lose his certification? No. What skill must a tester demonstrate to maintain certification? None.
- Tester certification has no teeth. It has the same legal footing as World’s Greatest Boss or World’s Greatest Dad.
“Yes There is an Alternative to Bad Certification” argument
- I do support certification programs that are designed to promote personal responsibility and protect an activity from restrictive regulation. For instance, if people were dying too often from scuba diving accidents, scuba diving would eventually be banned.
- I generally support certification programs that provide reasonable protection for consumers in an inefficient market, without posing an unreasonable burden to trade and innovation. But in the case of testing, employers don’t need protection from bad testers (because they already have protection, see below), and even if they did need it, they aren’t getting it from any of the certification programs that currently exist.
- I support any organization’s right to decide who can be a member, within the law. There is a certification process already in the testing field. It’s called a job interview. This protects employers. This can be followed up with an assessment process called watching people do their jobs. This protects employers. If you don’t know how to tell if someone is doing their job, then you aren’t qualified to be supervising that person. If you are an employer who knows nothing about testing or software development, and yet you want to have testers and software developers working for you, consider hiring a project manager who does understand these things. Alternatively, you can hire an outside company to manage things. At least they are accountable. But who is accountable if a certified tester does bad testing?
- One alternative to certification is taking the free online Black Box Software Testing course. It’s a full bore testing course, with hours of video lectures and lots of other materials.
- Another alternative is to get friends to say good things about you. Become “colleague certified.” My business as an independent consultant, trainer, and expert witness is based almost entirely on my reputation. Reputation = opportunity = money.
- I could support a tester certification program only if I thought it was honest and useful both to the tester and people dealing with the tester. Once our craft matures a lot more, I suspect I will support some form of industry-wide certification. Meanwhile, I can support certification programs that are A) identified with a particular community rather with the industry at large and B) skill-based, C) have undergone some kind of field testing process, and D) celebrate self-critical practice.
- There are no certification programs for testers that meet all of these criteria– not even BCRIT (Bach Certified Rapid Tester) which is my own program. BCRIT is still in development. I am not promoting it, yet. I may never promote it. I’ll promote only when I’m prepared to support it with cogent reasoning and evidence. I call upon other would be certificationists to do the same.