Two Short Dialogs on Test Coaching
My brother and I are experimenting with short podcasts. Here are the first two 15-minute segments:
- Testers say the darndest things. One issue in coaching testers is getting them to speak carefully about evidence and about what they can and can’t do. For instance, we talk about how we react when someone tells us that “a tester ensures the quality of the product.” Jon’s strategy is to point out the incompleteness of testing. Testing can’t be perfect, therefore testers can’t ensure quality. My strategy is to focus on the control aspect: testers aren’t in control of the project, and therefore can’t ensure anything. We think that part of the reason testers make over-the-top statements is that it’s comforting to cling to simple fairy stories about testing.
- The trap of not asking questions. Jon and I notice that testers presented with challenges and problems often jump right into test execution instead of asking questions. We question this phenomenon.
January 8th, 2010 at 1:19 am
I “liked” this on Facebook–I only wish I could “like” it twice! I hope these can — not “go viral” — do service as inoculations, as little mind-sized chunks of ANTIvirus exposure. There are a lot of viruses out there in the memetic pool of testing.
January 8th, 2010 at 6:52 am
A note on “The Trap of Not Asking Questions”. At around 13:17, James says “…you need to have a model in your mind of what a test project is. I have a model in my mind, and it has five elements. The five elements are test lab, test team, development process, requirements, and my mission. And so one model I might use, one principle I might use is, Do I know enough about each of those five things to jump in and be reasonably confident that I’m not completely wasting my time?” James here is referring to the Context Model. You can find that at http://www.satisfice.com/tools/satisfice-cm.pdf.
—Michael B.
January 10th, 2010 at 1:23 pm
The trap of not asking questions -
I don’t consider this to be a trap that is limited to test coaching only.
It is my experience that developers very often fall in that same trap. They start their development work based on very ambiguous requirements and mix in their own interpretation and a lot of assumptions. Once problems are being reported they claim that ‘the requirements are bad’.
I am planning to use the Context Model that Michael is referring to in my coaching of developers as well.
The developers coaching assignment that has been recently given to me was to compose a list of questions that every developer should ask. I can come up with some sample questions, but like James suggested, it may be appropriate not to ask any question at all.
I think that it should not be too difficult to create a context model for development planning. I expect that it will look very similar to the context model for test planning.
Many engineers prefer to work with a fixed list of pref-defined questions. So far I have always considered this as an indication of lack of engineering skill.
January 10th, 2010 at 1:30 pm
I consider the requirements documents, test documents, and design documents to be components of the product.
With that point of view, I feel like I am interacting with the product if I am asking questions about the requirements.
Jon said that test execution = interacting with the production. Based on that I consider the questioning to be a part of the test execution. Also because I keep on questioning (requirements
throughout the completion of my testing process)
I wonder if you consider this ‘questioning’ to be part of a different ‘test phase’ (test preparation?)
[James' Reply: This is why I don't make a strong distinction between testing and test preparation. The distinction is sometimes useful, but it all kind of blurs together in the big picture.]
January 11th, 2010 at 4:03 am
After these two podcasts I was left with the question: How can we have good testing if testers start without knowing about their job description AND if they are left alone once they have a job?
The background for this is my own. After getting out of university I got my first job, was send to a partner company and was told “test this”! No one ever cared if I had any idea about testing. No one cared to explain to me what a good test is. No one cared to tell me how I was supposed to handle bugs … .
For me it is a miracle that companies accept such conditions because “it’s just testing”.
To get back to the podcasts, I think the reason you, James and Jon, see testers with missing attitude and knowledge about their job is due to situations like the one described above.
Someone who worked in test for a year is laid off, looks for a new job and suddenly meets a “real” tester. How likely is he to get another job in that field?
January 13th, 2010 at 8:20 am
I’d like to propose another point why the questions are not asked during coaching sessions when presented with challenges and problems.
The problem is in the way people are trained to think in schools. Teacher gives you assignmet to do and you do it. For 12 years. And usually also in universities. That kind of thinking of teaching is then stuck. Teacher gives stuff to do - you do it. No thinking about it.
It is not that people don’t think in real-life situations. They do and they question their assignments also. But another problem
here is from working culture. In most places the boss hands out the assignment and it people follow it, even if it feels a bit strange.
Both of these are nothing that anyone actively does, it is just they way they are used to do stuff. And it takes time and practice to change this habit.
January 13th, 2010 at 6:03 pm
I liked these a lot. Much richer than say, reading similar content in an article.
There also seemed to be some spontaneity that happened during the conversation that you also might not get in an article that is revised and reworked a number of times.
January 18th, 2010 at 12:20 pm
I think you two should have named the second podcast “The trap of unvocalized expectations…”, since that seems to be the gist of the exercise. If you have expectations that the applicant under test, if good, will ask questions before testing, you should allow for that option in the initial directions; throwing a floppy at someone [especially someone who may need a job to feed their family] and stating “Test this!” is merely a demand to test an application without requirements, verbal or otherwise.
I’m willing to bet you’re going to get exactly what YOU requested.
If I discovered that my interviewer had played me in such a way, without even saying, “Feel free to ask as many questions as you need”, I would turn down any proffered position; clearly, your organization has some primal issues with communications, and I’m wagering that this is only the tip of the dysfunctional iceberg.
You may not be wanting your audience to think like you, but it smacks of wanting your audience/employee/applicant to read your mind. And that’s an expectation that will NEVER be met, no matter how much you vocalize it.
[James' Reply: I agree that mind reading is not what we can expect. So, I have two reactions to your comment. One partial agreement, and the other partial disagreement.
First the agreement. What I do-- I think Jon does this, too-- is not to flunk people who don't ask questions. There may be lots of good reasons someone doesn't ask questions. But what I will do if I don't hear questions is to say "I'm concerned that you haven't asked me any questions. I would expect a tester to have many questions." If the candidate still doesn't ask anything, then I feel that's more of a problem. I once asked a candidate NINE TIMES if he had any questions. Each time he responded with a comment instead of a question. He finally leveled with me that he was "brought up that it was impolite to ask questions." Sorry, that doesn't work for me.
I expect confusion in an interview. I expect mistakes. I assess a candidate by how he recovers and reacts.
Now a little disagreement. I expect, if someone is seeking an engineering position, that he is able to perform basic arithmetic. I shouldn't have to put that in the job requirements! It's a fundamental professional skill. I consider questioning to be such a skill and habit for testers. If you feel differently, by all means do NOT apply for a job on my team. I'm looking for a different kind of tester than you. But really, don't you think questioning is an important practice?]
January 19th, 2010 at 8:01 am
I most say I enjoyed this too and look forward to seeing/hearing more from James. To the above comment, I think merely what James is trying to do is to get people thinking differently about testing (and for the better) and outside the box. This takes alot of re programming depending on the individual and I suppsoe the testing pratcices they have adopted up until that point.
As far as the questions go, Asking Questions has to be in the first the top 3 things that are important that a Tester be capable of doing. My opionion would be if you don’t have the curosity to be asking questions then you don’t care and therefore are not testing effectively.