No Preservatives, No Pretensitives
Cem Kaner recently posted an announcement of a class he’s giving. In the announcement he said “This course is not affiliated with a university. It is not accredited, not approved by any professional society and not associated with any certification effort.”
I smiled when I read that. Then I realized why I was smiling. Our little craft is so overrun with artificiality and pretense, it’s unusual and refreshing to see someone going the other direction.
For an example that does not make me smile, look at what Villanova is doing. They are offering a “Master Certificate in Software Testing”, their advertisement cites the importance and value of ISTQB certification (without mentioning, of course, that certain prominent people in the world of testing, such as but not limited to myself, have longstanding, deep, and principled objections to the ISTQB. I think it’s a system designed to take your money, then cauterize your education as a tester with a lot of propaganda about best practices). To scare you into action, they cite an official-looking study by NIST that impressed at least several testing experts (such as but not limited to myself) as being poorly researched and badly reasoned. Then they cite Rex Black as being a world-renowned expert in software testing, in case you might otherwise have thought to question his authority.
Let’s add it up: Famous University offers you a “Master” certificate from Great Authority Teacher who is President of International Authority Organization so you can be certified by said Authority because Official Government Report says that the industry is in bad shape.
I wonder, after all that, whether the guy who teaches it appears in the classroom as a ball of flame. Where’s Toto when we need him?
Now, I have no better claim to the status of testing expert than does Rex. If you’ve taken a look at my video about testing expertise, you know that. But that’s not my point. My point is that here’s a testing class selling itself for big money with all the makeup, glitz and pomp of a royal self-coronation…
…and then over here Cem Kaner is offering a new version of his own longstanding testing class for free, taking care to tell you it’s just plain ideas– no additives, no preservatives, no baloney. It’s a different style.
I don’t know much about Rex’s class, but I do know that the only way to emerge as a master tester after taking a few courses is to be a master before you start!
That’s why I feel safe around people like Cem, Matt Heusser, Jonathan Kohl, Michael Bolton, Karen Johnson, or David Gilbert. Read their blogs. Good wholesome fare. No panaceas, no pesticides. Don’t be fooled by fast food.
December 26th, 2006 at 5:04 pm
Good stuff James.
A couple of years ago I attended a conference in Richardson, Texas.
That was the first time I had a chance to talk to you in person.
During one of the breaks I was approached by someone from Germany (I do remember his name very well). The guy was there to promote ISTQB. He was telling me how poorly our testing industry was doing and our big need for a certification recognized worldwide.
When I asked for his background he pulled some intelligent sounding certifications and diploma’s out of his hat…. most of them related to finance and marketing, none of them related to testing.
Like you wrote a while ago, I believe in self-certification.
I get a lot of satisfaction just by communicating to people like David Gilbert, Jon Bach, James Lyndsay, Esther Derby, James Bach, Marta Gonzales, …
You all seem to be interested to hear about my specific context when I contact you… as opposed to other well-known people in the testing industry who seem to find a pleasure to tell me that I am wrong. (sometimes even before I have told them anything)
Erwin.
December 27th, 2006 at 7:17 am
First of all, where is your video on expertise in software testing? I’d like to see it.
Second, Erwin wrote: “You all seem to be interested to hear about my specific context when I contact you… as opposed to other well-known people in the testing industry who seem to find a pleasure to tell me that I am wrong. (sometimes even before I have told them anything)”
Me Too! My ears are still burning after one particular gentlemen kept asking me “Why don’t you use Mercury X?” for about half an hour. It turned out that I would have an explanation, then then he would immediately move to a different mercury product. Eventually, I realized that he wasn’t really asking me “Why”, but instead he was making the _Assertion_ that should use mercury products. I looked at my watch, said “Gee, look at the time”, and carefully excused myself.
[James’ Reply: For the video, see: http://www.satisfice.com/blog/archives/50]
December 27th, 2006 at 11:12 am
This reminds me of my PMP certifcation preparation program I took a few months back. At the begining of the program the director urged the students to go back to our work places and advertise the importace of project management and the the PMP certification, giving it more “visibility” so to speak. And then in several classes students brought up real life PM issues and the standard response from the instructor was “This won’t be on your certifcation test, you do not need to know it.”
Interesting, isn’t it? If PMP certification is not aiming to help you solve real life problems, what good is it? If PMP certified professionals cannot go back to their workplaces and demonstrate real practical skills, how exactly do they convince the executives that project management is important? If what they learn cannot help them finish the project before deadline, below budget, with quality, what value does it have besides that piece of paper?
Back in the day I remember I was so bored at work one year I took 10 certification tests, just blew through them, how? because they are nothing more than memorization tests. Real combined value? Very little.
It is up to the institutions to rethink the value of the education they are offering.
I see this not just in certification programs, but many universities as well. They tend
to sell superficial values and shove students through them without really invest time
and effort to figure out how to provide real value.
In a lot of ways, it is a reflection of our instant gratification based culture.
[James’ Reply: Before I came out publicly against bad certification programs, I had the occasion to listen in on a conversation among a group of consultants in England, who were sitting around a lunch table discussing how certification was such an economic boost to their teaching. People were willing to pay so much more for classes that result in certification.
They are willing to pay more. I respect those teachers and consultants who won’t bow to the temptation to sell magic beans to naive and frightened testers, because it sure is easy to do.]
December 27th, 2006 at 5:35 pm
I don’t know how I missed this before but you are the son of Richard Bach?!!
Your dad’s beautiful writing has been an inspiration for me for years. “Illusions”
is one of my favorite books. I love his writing and his philosophy, in fact come
to think of it it is a good time for another read. Wow, how interesting!
[James’ Reply: Thank you!]
December 27th, 2006 at 6:13 pm
I definitely agree with you about the hocus-pocus involved with most certifications, but I see a silver lining to this story: the appearance of more certifications means that testing is gaining more legitimacy in our industry. Certifications are cottage industries that build around popular subjects or positions, as we’ve seen with software development, security, project management, etc. Despite the fact that this certification is probably a waste of time and money, simply seeing the inceased interest in testing is nice.
December 28th, 2006 at 3:01 am
Some time ago I have initiate and led workshops (continuous through a few month) in my organization to discuss ISTQB Syllabi. It was worthwhile experience, indeed! Syllabi covers the process and the general terms pretty good. Even foundation level includes a lot of activities and terms that we don’t practice/use in our company. Person who has learned this will be able to pretend knowing everything about testing… at least during conversation, not during real job. I’ve seen good testers not able to stand their ground while talking to management – that what they could probably learn from such training. Possible ways to extend your process may also be a learning objective. Not testing however.
In order to train The Testing I demand new testers in my organization to do the video lecture course by Cam Kaner that you refer to (I believe). So I recommend doing everyone else. This is the training that should become informally recognized by test experts. I dream of that informal recognition to compete with formally approved certifications at least in testing community if not by management.
December 28th, 2006 at 3:32 am
>Despite the fact that this certification is probably a waste of time and money, simply seeing the increased interest in testing is nice.
Wow… that sounds like a non-sequitur to me. Wait… it’s worse than a non-sequitur. That is: do we really want increased interest in our field based on a waste of time and money? I think I’d rather dwell in obscurity than suffer guilt by association.
—Michael B.
December 28th, 2006 at 8:01 am
I’m from Colombia South America, and here are two or three companies that sell testing as an outsourcing service, when you came to them and ask what kind of testing do you do, they answer “Functional testing”, that’s ok but do you ask a little more to know about the service and the ¿what’s that? Questions emerge and then the silence came into the room. And even they are ISTQB certified!!! They didn’t now any technique besides equivalence partitioning.
The “The Ongoing Revolution in Software Testing” paper of Cem shows the excuses for do bad testing, and then you keep listen to them in the meetings. But you know they are in the business for about 5 years, so they had to know… NO WAY! The certifications don’t make the experience, or have a lot of years making testing, that only proves that you can do the same crap for several years in a row.
Thanks for share your experience with all of us, and let me learn more about it. I love your blog, the one from Jonathan Kohl and the black box course of Cem Kaner, it’s really interesting to argue with you about testing.
December 28th, 2006 at 12:22 pm
I think you may be misinterpreting my comment. Anytime you have something increase in popularity, you will have cottage industries trying to feed off of that popularity. Look at the iPod, for instance. As it increased in popularity, we saw tons of accessories and services attempt to feed off of that popularity - I would say that most of them are a waste of money. Do you really need 4 different ways to connect your iPod to a speaker system or headphones? No. This doesn’t mean that the iPod is a terrible product because of the rise of companies trying to feed off of the iPod.
I’m making the same comparison with testing. The rise of cottage industries surrounding testing that are of questionable usefulness are a sign to me that testing is becoming a more popular topic; something I’m happy to see.
December 28th, 2006 at 11:28 pm
James — after having just rewritten my home page for about the hundredth time, I will first thing tomorrow, as earlier promised, write a blog on a conversation I had this weekend on this topic. But here is the upshot, condensed… not all certifications are evil. In fact, I am not certain any of them are. Guns don’t kill people, people kill people. What you *BELIEVE* about a certification…whether you cooked it up yourself, or whether your bought the snake oil someone else was peddling…that is what will get you in trouble. Certifications are like learners permits…they can be a good starting point, a quasi official blessing from ONE oracle, not necessarily respected by all, that can show someone what they DON’T know more than what they do. But if taken as such, they can have value. Where you get into trouble is when you think the learners permit makes you Tony Stewart. Now really, when was the last time you saw someone actually FAIL the learners permit test? Now, think about all those fine folks on the freeway with you every day, but driving 200 mph. Suddenly, the difference becomes crystal clear. And telling those idiots they have no business driving that fast, if at all…that’s admirable stuff, and that’s why I respect what you and Cem and everyone else who is waving this flag is doing.
[James’ Reply: Please, David. I have never attacked the idea of certification. I’m attacking the kind of certification programs that exploit fear and ignorance for profit. These include all of the publicly available certification programs I know about in the testing field, as of today. These bad certification programs are not like learners permits. They are merely a combination of self-serving propaganda and magic bean shammery.]
January 1st, 2007 at 2:07 pm
Thanks James. I’m new to blogging so I appreciate the feedback. I sincerely want to post material that contributes and not just post bloated blogs. Comments and feedback help.
January 2nd, 2007 at 1:40 pm
Let me try to be context-driven here, does the ISTQB-certification add any value at all for any people at all at any time at all?
Despite the obvious downsides with the current certification - it costs money, some of the theory is really baloney, a few chosen people decide what is the best for everyone etc - I see a place for the certification program. I have been both working with and teaching other testers for the last twelve years. The ignorance I have seen in testing is immense also within the testing community! It has been really hard for me to get the message through to those people that should be the most receptible or rather already enlightened. The message here beeing knowledge about testing. For these people the foundation certification opens their eyes a bit. At least they see that there are things to learn that have worked for other people (other than me that is). They understand that there are accepted test design techniques, some tools, some principles, processes etc. That means that they are now hopefully open to learn some more. So in my view it is a first step towards enlightment. For most people that read or write this blog I doubt that there is much value in an ISTQB certification class. But for the beginners it is a step in the right direction that their managers are often willing to pay for.
[James’ Reply: Toby, this argument only works if the certification is better than alternative solutions and does not actually do damage to a tester’s education. I think the answer is no on both counts. Have you considered the available alternatives? Have you thought much about the damage? In any case, you may not agree with me, but at least engage my arguments. I have not criticized ISTQB based on its lack of perfection. I’ve criticized it because it is worse than what already exists, and its primary redeeming value is that it lines the pockets of a handful of consultants.]
Unfortunately the certification test itself, which very few fail, is too often seen as a proof of important skills rather than basic knowledge about testing. Something that really upsets me is when I see an ad for a job where certification is seen as a merit but a college degree is not mentioned! That implies that a three day basic class in testing is worth more than my five years getting a Masters degree in Computer Science! Nevertheless - if someone wants to hire a person with a certificate…well a testing guru only needs his reputation, a less famous person might need a paper saying that (someone got paid to say) they know someting. So I don´t think that people are afraid of not having a certificate, they just don´t have the energy to fight the system. Cynical, yes I think so.
Having said that I believe that real skill comes from self-study and practical work combined, which no certification program can replace.
[James’ Reply: Anyone can plausibly self-certify in testing. You did, for instance. And Pradeep Soundararajan attracted my attention and has joined the context-driven community as an active member, not because of his encyclopedic knowledge and long experience of testing, but because of his determination and passion to learn. If I had been a hiring manager at Hewlett Packard, instead of a testing consultant, I might have hired him as a test manager based on the work he’s showed me, and his writings in testing.]
March 8th, 2007 at 2:18 pm
As a new entrant to the testing field, I found the lack of knowledge in general in the industry to be appalling. I was even lucky enough to work with some who thought it was okay to test a report status filter by ensuring that it exists and different statuses can be selected from it. These consultants were the same group of people you speak of, who like to boast certifications instead of producing quality work. I have been aware of your arguments against testing certifications for a few years now, though I couldn’t completely agree until I obtained the CSTE. The cost is low enough to take the test, however, the amount of time spent pouring over the 600+ page book of ‘Knowledge’ is what really makes me feel as if I had wasted my time. Factor in how many people I’ve met who have proven their weight in gold as testers who couldn’t pass the exam due to silly memorization and ambiguously phrased questions, and I can’t help but wonder why I didn’t listen to you in the first place. We all have to make our own mistakes though, and I’m happy to look at it as a learning experience.
Having been certified for almost a year now, I can honestly say there was nothing of value from having it aside from being able to slap it on my resume. A bigger problem than the lack of worth for becoming certified, though, is the scare tactic mentality you speak of which may affect not only the testers, but companies which may require one, or choose one candidate over another based on the presence of that piece of paper. A comment from Toby [above] states “That [requirement of certification w/out mention of degree] implies that a three day basic class in testing is worth more than my five years getting a Masters degree in Computer Science!” I can understand his frustration, but also have to point out that he’s comparing one piece of paper to another. What’s to say that a person who has a master’s degree is actually a better tester than somebody who has obtained the ISTQB? I’m not trying to insult Toby here at all, simply trying to point out that the worth of a tester is difficult to assess in any case. Do I weigh a Masters in CS higher than the ISTQB? Of course I do, takes much more discipline to obtain, and is more socially acceptable (for the most part) as a form of professional development. But what is a degree aside from a certification that takes a lot longer to complete? Yes I understand there are several factors to account for [reputable school, great teachers/professors, curriculum which tests more than simply memorization] and that’s my point. If one piece of paper can be weighed more heavily against another, so too can a piece of paper be weighed against the absence of said paper but with a strong foundation of knowledge/skills. There are always factors which help to determine whether somebody is capable in the field and those are what need to be assessed by potential employers and peers.
[James’ Reply: I’m no fan of CS degrees, or any degrees. But it does seem to me that a great deal of real work goes in to getting a university degree, generally speaking. Whereas very little work is required to achieve one of those certifications.]