Becoming a Software Testing Expert
I was recently invited to speak at Google on the subject of “Becoming a Testing Expert”.
You may view the video of the talk at Google Video. Sometimes the screen is hard to read, so you might want to read the slides in PDF form.
This talk is basically about how you can become recognized as a expert in software testing. I explain how to do it by explaining how I did it. My goal is to de-mystify the field for you. This matters because we need original thinkers in the testing field, and I’m worried that you may delay writing articles or doing talks because you think that the rest of us who contribute publicly to the field are on some inscrutable higher plane of thinking.
Bear in mind, I have many professional rivals. Some of them are good honest people who have a different vision of the craft than I do. Others, I think you will find, are charlatans who are either out to grab your money or out to get ego validation by impressing you. Of course, from your point of view, I may be one of those charlatans. I understand. I simply advise you to keep an open, yet skeptical mind. Taste everything, swallow nothing.
By the way, my brother Jonathan took my advice, and now he’s become recognized as a testing expert in his own right. He even got to do his Google TechTalk before me! He did an excellent talk about exploratory testing which you can find here.
June 18th, 2006 at 7:51 am
Nice video.
The PDF file is not available as of yet but I’m sure that will be rectified once you see this comment.
I don’t have many comments except that it was refreshing seing you be so involved into the subject matter.
June 18th, 2006 at 9:37 am
PDF problem is fixed. Thanks for reporting that.
June 18th, 2006 at 1:23 pm
James - Great to see another post from you. Enjoyed the video.
June 18th, 2006 at 5:39 pm
This is great stuff, thanks James!
Chris
http://amateureconblog.blogspot.com
June 19th, 2006 at 10:00 pm
That video sounded great, so I clicked on over there to take a look. It offers to let me download it, so I do. It tells me I need thier video player to watch it. I dropped it onto VLC just to check, but sure enough: their video only through their player. Whatever. They lock me into their proprietary player which also isn’t good for anything except their video. (I hear a mocking voice in my head, “Don’t be evil”).
I download their precious video player and crank it up with the film. It flickers a lot, for quite a while, then settles down (some) and tells me it will be between 3 and 4 hours before it starts unbuffered playing. On my broadband connection, it will take between 3 and 4 hours.
Not your fault, James, nothing to do with you, and I’m sure you don’t have any choice about any of that. But this is exactly the kind of “customer contempt” that I think good SQA practice needs to assassinate.
Believe it or not, I do have a few other things to do with my time, connection and computer than wait four hours to watch a short video.
(Yeah, yeah, server problems, network congestion, using (gasp!) my network for other things.) This misses the point. As a customer, I’m angry. Not disappointed, angry. They think I have four hours to just wait for them? They’re wrong. If I’d bought it, I’d be demanding my money back. As it is, I’ll just avoid .gvp files forever after, and urge people to translate them to some other format if possible. Hell, I’d even settle for .ram. Obnoxious as it is, RealPlayer doesn’t expect me to wait for hours for it.
This is the kind of thing I believe we, as a craft, are charged with detecting and stopping before ever a customer sees them. If, due to unavoidable problems, it’s going to take hours, then FIND ANOTHER WAY. Say, “I’m sorry”, say “Try again later”, say “You may want to reduce your bandwidth usage while it loads”, say just about anything except “f*** you, just sit there and wait, we’ll show it to you when we’re good and ready.”
I do appreciate the irony of finding that particular example on a link to this particular blog, though.
June 19th, 2006 at 11:36 pm
May be you are one of those charlatans you have said, but I confirm that you must be a man filled with much humor:)
June 20th, 2006 at 12:25 am
Jim,
I was skeptical when I saw the title of your talk. I hold that “Expert” is a title that is bestowed, not one that is claimed, but as I watched, you took the wind out of my sails on that attack when you specified “perceived as an expert” and “what it means to me to feel like an expert”.
Since I was already being skeptical, I then went to your personal testing syllabus, bound and determined to find *something* important that you missed… I eventually surrendered after coming up with a huge list of items that all seemed to be sub-categories of what you had listed and no list of items that were really missing (as opposed to categorized differently than I might have).
After that, I started digging, disecting, applying epistimology to find hidden meanings…
only to realize that it’s a dog-gone powerpoint slide deck, and a 60 minute talk that you could have spend days talking about and that I have plenty of testing problems that are more important for me to be putting this kind of effort into at this moment.
All that is a very long-winded way of saying…
“Cool presentation, Jim. There is some stuff in there that I think I’d like to use in some of my talks/classes. Thanks!”
Cheers,
Scott
June 20th, 2006 at 1:57 am
Hello Insectivorous,
Years ago, I made the prediction that we would all, someday, come to speak of quality the way we speak about the weather: in terms of general prediction, tinged with resignation. “80% chance of satisfactory performance today, as server availability moves through the area, but look for locally heavy buffering in the suburbs with a chance of crashing toward the afternoon. Now, here’s Sally with the upgrade outlook…”
– James
June 20th, 2006 at 3:28 pm
insectivorous,
did it say it need 3 or 4 hours to buffer the whole time? was the player able to download a portion of the video? because the bufferring time should keep decreasing quickly.
also, are you using a proxy server?
The video can be played with the DivX player and you can download the .avi using this url.
http://vp.video.google.com/videodownload?version=0&secureurl=xwAAAG7ggqAHSiJjpW0D3w4aYTXsXKQ-sKZ48PaGIi18uTMupjQ6MvyosbH5P11LL9rhCdmH68YNDvTpEY3fCttbAet7qitbvrFMdM8yCmqAGjODeNQ6dlxL-UqJ8AuZYQuzkpmYIUyV3-MJedtlmjE8xfLMXwwlJSnTTjRryMWxtB4YkhdBnkXkvNuz4K_hU9JhDQeS1c9Nnnv6M03-52UAX4cLOcr6uOiOuH2kAarDXLQKFXLx-dNEAUDXUH32GbOqx6Pr7V4PeDLccSODLE8sP3Q&sigh=duClULmZp3vkmDDpsTZw3pVTTuI&begin=0&len=3450498&docid=6852841264192883219
June 21st, 2006 at 7:53 pm
[…] Hoy me he tragado un par de conferencias de las propuestas por Google en su Google Video. Se trata de unas charlas llamadas Google TechTalks, que os podéis imaginar son de lo más entretenidas (para el que le guste esto, claro). Estas dos en concreto las he encontrado en el blog de James Bach, un “experto en pruebas” según se autodenomina él, y al que debería observar con escepticismo según su propia “doctrina”. Las técnicas y enfoques que presenta son a la vez algo radicales, y mundanalmente realistas. Radicales en el sentido de arrojar por tierra las “buenas maneras” propuestas por mucho otros expertos y “escuelas” de probadores de software. Realistas si te encuentras permanentemente en “el ojo de la tormenta” que el mismo describe con tanto énfasis. No me había dado cuenta de lo cercanas que eran hasta que he puesto el vídeo de la charla. Alguien ya me había pasado su frase: “Testing is the infinite process of comparing the invisible to the ambiguous so as to avoid the unthinkable happening to the anonymous.“ […]
June 22nd, 2006 at 8:28 am
Vincent, no, no proxy server, but this is far from my first experience with streamed video. I gave it quite a while to settle down and do something, but got nothing except the time estimate that went down to 3 hours, the started climbing back upwards. Yikes!
I really do appreciate the avi link though! Thank you!
You mentioned the DivX player, so I wanted to point you to the free VLC player from videolan.org, because I think you’ll really like it. It will play just about any non-proprietary format, audio or video, and streaming. And it doesn’t use the system codecs, so no fighting over filters or having codec packs fall all over each other.
It’s a shame the docs are nonexistent and the support’s lousy, but having one player that handles it all, and I include DVD’s and VCD’s, and even disk images, really rocks. It even includes a stream server and stream capture.
I have no connection with it, just a happy customer.
James, it does seem to me that documentation and active support are two areas that regularly get short shrift in software, but I’m not entirely sure this belongs under the SQA umbrella. On the other hand, if it doesn’t, where does it belong, and who will do it if not us? On the gripping hand, this is a completely different skill set, especially in the area of support.
So I’m wondering just what the SQA role ought to be in this, and what pointers to material addressing it you may know of? I’ve been doing voluntary support work for bittorrent, and the human elements of this are truly maddening.
Planacity, shouldn’t that be “ungrateful anonymous”? Or maybe “illiterate anonymous”? Or have I been doing too much support?
June 23rd, 2006 at 11:10 am
[…] Speaking of being an expert, check out James Bach’s video on becoming an expert. He talks about how it happened to him. Among other things, he shows his startling lack of expected credentials. This is part of his approach of providing first-hand insights that both demistify the notion of an expert and raise a high bar for attaining expert status. […]
June 27th, 2006 at 5:55 pm
An informative and interesting talk, from an interesting perspective. I generally find your blog thought provoking, but it was very good to hear you talking for a good length of time. Cheers.
June 27th, 2006 at 7:21 pm
James,
nice video - almost as good as your live presentations! Aside from testing profession, if you still practice paragliding, I will be very happy to host you for the 33th edition of “Coupe Icare” in Grenoble/France, September 2006 (http://www.coupe-icare.org/GB_home.html) - world’s largest free flight festival. And, as a pilot myself (not an expert!), make a flight or two with you in the area. In addition, why not have both pleasure and pain: give a talk at my company
June 28th, 2006 at 12:26 pm
James said “This matters because we need original thinkers in the testing field, and I’m worried that you may delay writing articles or doing talks because you think that the rest of us who contribute publicly to the field are on some inscrutable higher plane of thinking.”
This is a difficult problem to solve. During a break in the middle of your talk at the CAST conference recently, I overhead someone talking about being in awe of the speakers. I encouraged him to speak up - ask questions and make intelligent comments, and that would help him earn respect, even if he was disagreeing. This seemed to give him a new insight, and he thanked me. But I still didn’t hear him chime in. He’ll need continuing encouragement.
I’m not generally hesitant to chime in on any discussion. But every time I have a face to face conversation with you, James, I come away feeling like an idiot. I feel like I’ve wasted your time, and there’s too much information for me to try to absorb in order to learn from it. When I post something on a mailing list you’re subscribed to, I know there’s a chance that I’ll be subjected to your scrutiny, and that I’ll be proven wrong on several points. If I have a thought that’s half-formed, I may hesitate to send it. And as an MBTI extrovert, most of my thoughts are half-formed until I’ve vocalized them or typed them for public consumption.
Imagine how someone who’s more shy than me feels. Perhaps you’re not inscrutable, but you often are on a higher level of thinking than the people you’re communicating with. Yet, as you imply, any of us can and should learn from each other.
I’m not ready yet to suggest how to fix this. I don’t generally mind being corrected, and having others help to fully bake my half-baked thoughts. I haven’t yet figured out why it’s different with you. One thing I started doing when I realized that I tended to intimidate people I was talking to is to reduce the “firehose effect” - not to give people more than they can absorb.
On one of the mailing lists I maintain, we even came up with a name for the phenomenon you’re talking about or something similar - the “luminary effect.” When a recognized expert answered someone’s question, the discussion tended to die right then. Few people were willing to challenge the expert. The only solution we could think of, and it wasn’t a very good solution, was for the experts to hold off on joining the discussion until others had a chance to voice their opinion.
-Danny
June 28th, 2006 at 6:35 pm
To Insectivorous:
I gotta like someone who uses Motie figures of speech!
I briefly ran tech support for SmartPatents, once. One thing I learned from it is that it has a different rhythm and skill set than testing. Same with documentation, from what I gather. What makes testing special is that we are paid to believe in failure. Everyone else believes in success; almost everyone else on the project creates something. From that, I think most of our skills flow.
To Danny:
I can turn on the charm and be sympathetic and encouraging. When I’m doing that, I am not fully engaging the ideas in the conversation. To me, engagement means dissection and analysis and finding problems. Intellectual engagement is a testing process, for me. Also, there’s a certain amount of stress when I hold back, so I get tired.
I hold back with my wife and son, and with other people (such as most of my students) whom I don’t want to intimidate and who may be susceptible to intimidation. I have *not* been holding back with you, Danny, because I thought you wanted me to treat you like a colleague. You seem to jump into conversations with confidence. So, that’s what you are seeing from me. With people I see as full-blown colleagues, I just say whatever I feel is productive, assuming that they will take care of themselves just as I do.
– James
June 29th, 2006 at 12:36 am
James, I’ll agree as far as that goes, the activities have a different skill set, but…
Well, yes, we are negative, or yang, or the other side. Between the time when the oceans drank Atlantis, and the rise of Microsoft, when Ken Olsen could scoff at the idea of a home computer and everyone believed him, I was a real-estate examiner. In a property deal, the buyer wants to buy, the seller wants to sell, the mortgate company wants to lend the money, the closer wants to insure it…everybody in the whole parade is in favor of going forward, except the poor schmuck who’s examining the title. He’s the only one saying “no”. (Made me a good fit for QA, I guess)
But that was my job, and it was reasonably well paid and respected. I mean, nobody thought that because I was a title officer, it was because I wasn’t “good enough” to be a closer, or anything remotely like that.
But if you’re a tester, you’re still fighting the lurking impression that testers are just failed programmers. The battle to establish testing as a profession in its own right, with its own skills and abilities, is far from over. Just consider how many shops are still stuck, at this late date, in the “code/test/fix” mentality, and have never even considered any kind of serious QA effort — or could even conceive of what that might look like.
If we’re ever to make serious progress on that front, it will have to be with obviously superior products — because of the QA effort that went into them. A lot like justifying a bonus at review time, we have to be able to point to definite contributions that are clearly QA’s. And doing that is a problem.
We’ve all learned that marketing trumps quality, hands down. If it didn’t, we’d all still be using dBase/FoxPro instead of Access. Or Quattro instead of Excel. We can’t do much about that.
But support and documentation also trump quality. The product might be great, and blissfully bug-free, but if nobody can figure out how to use it or hates to have to, then the quality of it otherwise won’t matter at all. And this is an area where I think we CAN make a difference.
We can bring our QA skills to the documentation effort, and to the support effort. We can find, test and reject docs that just re-iterate the help screens (and if they were adequate, why have docs in the first place?) We can discern support strategies and methodologies that are and are not effective, that do and do not lead to real closure and customer satisfaction. We can and do field-test applications, so why aren’t we looking at field-testing docs and help systems? We create use-cases, but why aren’t we also creating support-cases?
I take pride in finding ways to break applications, but I also take pride in finding obvious uses that aren’t documented at all, or in technical terms never explained. (Ex: When you first started using Access, remember how long it took you to figure out what the hell a “pivot table” was?)
And the next time you have an issue with cell-phone billing, after you’ve calmed down and can stop cursing, look at the whole experience with a professional eye: is this process broken? Can we instrument it to count and illustrate defects? Can we find and fix potential problems before we put the customer on hold for fourty-five minutes? Couldn’t we do support-testing, and do it as a fairly simple extension of what we’re already doing, too?
The way we do documentation and support is just as stuck in the dark ages as the old “let the customer find the bugs” thinking that we’re trying to stamp out. We’re the people who have the skills to show how it can be done better, in the same ways we can make software better, only we haven’t focused on it.
We can’t, and shouldn’t, start to actually DO documentation or support (though that helps, just as programming experience helps testing). But we can start to run quality control and quality assurance on both with the same techniques and skills we’ve already got.
But surely I can’t be the first one to think of this?
June 29th, 2006 at 10:41 am
James,
Let me add to Danny’s comment in a way that may help you digest his main point (which as I understand it is NOT that he wishes you to “hold back”).
If you cast your mind back to WOPR2 you will recall a certain “dominating” personality at the workshop. It was not the general consensus at the meeting that he was “the expert” - although everyone would admit that he had a lot of experience, knowledge, and presented himself as an expert.
If you recall, there were many threads that had a lot of energy among the group which tended to die out rapidly after this person spoke. I feel (strictly my thoughts here) that the reason for the lack of discussion after his comments was not in what he said but in how he said it. He talked to the other attendees as if he was the teacher and we were his students - he spoke in a way that suggested that we better not disagree with him or we would be wrong.
There was only one person at that meeting who, in my opinion, could have “stood up” to him - which was you - yet unfortunately you were facilitating and could not directly participate in the discussions.
I think the point that Danny makes is that when you engage people in talks, you DEFINITELY invite them to scrutinize you and to challenge any of your comments - but you also attack (sometimes mercilessly) differing opinions in a way that is similar to our WOPR2 participant - you are right and they are wrong. I have noticed that when you present your opinion in the manner (or, more accurately, tone) that you are the expert then the discussion shrivels - when you present the opposing view point with a less superior attitude, then the other person is more readily engaged into the conversation.
I once had a direct boss who would continuously question and challenge every idea I presented to him without any indication that he thought the idea was potentially good. After a year, I stopped presenting him my ideas as I was tired of being shot down. I would implement my ideas without discussion with him - or even informing him. I later found out that he was intentionally questioning me in many ways about my ideas. He was proud of his behavior because he felt that it would make me think that he was listening to me and that he was ensuring that I was thinking the idea through. His strategy would have worked very well IF he had given me a little bit of hope at the beginning by saying: “That sounds like a good idea. Have you thought about…”. Without that acknowledgment I ended up feeling that I was being attacked along with my all of my ideas.
I am in no way asking you to change what you say or how you try and engage people, but perhaps you could start out a bit more gently and receptive when initially engaging someone in a “conversation”.
BTW - Feel free to keep the gloves off in our discussions - it is hard to find someone as passionate and intelligent as you to have a good conversation with.
- Paul
June 29th, 2006 at 12:00 pm
In my talk, I spoke of developing a colleague network. I spoke of hanging out with demanding minds. Paul Holland, Danny Faught, and Scott Barber are three of those minds I have had a lot of conversations with. You see their comments, above.
I ask my colleagues to speak frankly to me. I try to make it worth their while, mainly by acknowledging their contributions to work that otherwise might be mistaken as purely mine.
I mull their comments and try to get better at what I do, because I want to be fabulous at this testing craft. Sometimes its hard for me to process what they are saying, but it’s important to do it. Paul’s post illustrates the power of shared experiences in helping your colleagues and getting help. The fellow he’s referring to is one of the most opinionated people I have met in my professional life. It’s a vivid image.
These are colleagues I would recommend to any of you, out there, to make part of your network, too.
Also, although I don’t know who you are, Insectivorous, I’m interested in reading more of your writing.
– James
July 2nd, 2006 at 11:05 am
Paul, thank you for reframing my argument. I think it comes down to compassion. If I can show someone that I have compassion for them and understand their point of view, and then prove them absolutely wrong, I’ve not only won the argument but also perhaps changed some people’s minds. Being logically correct is not often sufficient to change someone’s mind - it’s a hollow victory at best.
The discussion about support is interesting. I handle practically all of the tech support and user documentation for a startup, as well as all of the testing - the support is not a part of the testing function, but in some contexts it’s appropriate for the same person to do more than one of them. There are two points of view I take on support - 1) Helping to get the product working (which is positive thinking), and 2) Making the product fail for me in the same way it fails for the customer (which is negative thinking, calling on my testing skills in a similar way as when I try to reproduce an intermittent failure). I’m personally not satisfied when I do #1 without #2, because then it’s likely that this customer and maybe many others will encounter the problem again.
July 6th, 2006 at 6:59 am
Hi Vincent and James,
I tried downloading the *.avi version through the link given above and had a different experience.
I do not have an internet connection at home as of now and also we are not allowed to download media files in our office PC’s, so tried downloading the same from an internet cafe which is supposed to be a broadband. It went on for 6 hours, 175 MB and still was going on… I had to cancel the download paying around 150 INR. I came back home disappointed and it was Sir Thomas Alva Edison’s quote “People back off without being aware that they were close to success” that kept me sit, wait and see it getting downloaded for 6 hours.
I am exploring, why while downloading certain stuff alone we get to see the file size?.
While downloading the avi, I did not see any file size or is it that I am not aware of viewing it.
I also did a google search to find a tool to estimate the file size for the 57 minute avi video but perhaps could not find any such tool. It would be useful if it is available, isnt it?
Regards,
Pradeep Soundararajan
http://testertested.blogspot.com
September 20th, 2006 at 3:12 am
One thing that I feel contributes to your (James’) unassailability is that most of your published test approach is heuristic-based. So there’s not really any arguing with that. Heuristics can be helpful to a degree, dangerous to a degree, but coming from you, are unlikely to be flat-out wrong (ie. Not actually a rule which helps solve a problem).
- I might have a heuristic which works better for me, but that doesn’t invalidate yours. No point arguing (but worth mentioning).
- I might not be able to think of a better heuristic on the spot, so no arguing.
- I might be able to think of instances in which the heuristic is fallible, but that simply enriches it. No arguing.
[James’ Reply: Naturally, if we don’t have to agree on anything, we don’t need to argue about anything. But if we did have to collaborate on something, we might experience a clash of heuristics. This is not a problem if we are using heuristics properly. Proper use of heuristics, I think, is not to surrender yourself to them. They are merely aids. Therefore, if our heuristics oppose each other, we put them aside and rethink the situation together.
People who use heuristics without understanding them may not be able to operate without them. They won’t be able to deal with exceptional situations, or even a fresh perspective on a normal situation. This is a frequent complaint I have with the Agilists (note the capital “A”). They have many aphorisms, but it’s rare to hear them talk about conditions under which their little sayings hurt instead of help. Heuristics that are not questioned quickly become unquestionable, and a creative art that produced them becomes a dead art, like astrology or numerology.
When I share a heuristic with you, I am not sharing wisdom. I am sharing what I hope is a reference to wisdom. I hope the heuristic resonates with your own wisdom, so that it helps you exercise your skill with greater confidence. I share all my heuristics in the spirit that they are improvable and always subject to and cleansed by criticism.]
October 6th, 2006 at 5:52 am
Nice blog about software testing - I would appreciate if you can post more podcasts and webcasts on QA and testing aspects. Its good to see so many comments - your blog is a popular frequent visited site - please post more multimedia stuff.
Here is a blog on Storage Area Networking - How to get a job in SAN , how are the pay structure & how you can build your career in SAN . This one if filled with lots of Cool tech videos & makes very interesting reading http://storage-jobs.blogspot.com
SAN tutorials
February 8th, 2007 at 6:21 pm
There is an article on software testing on my blog:
http://riseagain.wordpress.com/2007/02/09/software-testing/
I’m also mentionning you in my previous article, please tell me if it’s not accurate.
February 17th, 2008 at 3:47 am
Thanks James for sharing this video with us.
I have to admit that this was the first time I took the time for looking at presentation video till the end on the web. And it was worth to watch. Not only what you are saying about becoming a Software tester. It also so made me think how I came here, why I watched it and how it impacted me on a new quest for searching for information.
Normally I search for presentation hand-outs, articles or whitepapers. Only those didn’t present body language. Or support the feeling how you “actually” think about it at that moment. Perhaps that is one of the strongest things of your presentation. Make it available to the community how an expert is doing it and learn from it to become a better tester my selves.
Thanks again!
With kind regards,
March 12th, 2008 at 7:44 pm
Would love to hear your comments/views on a new company called uTest.com (http://www.utest.com). Your post directly relates to their idea as they are attempting to create a “community testing” business model. Clearly, a whole host of issues arise when attempting to train an entire community in software testing.
Please let us know your thoughts! Thanks!
Best,
Jonathan
[James’ Reply: How do you train them? How do you know they have skills?]