saw this announcement on James website,Congrats Man.I have few years of experience in testing but i saw your blog and feel that i am new to testing.Your zeal for testing is infectious.Its a great site..keep blogging..

And congrats again for Satisfice India

Congratulations! I love your passion for testing. Passions like yours
are rare in this world and should be greatly valued. Go for it Pradeep,
sky is the limit!

yan

I have been reading your blog regularly and glad to read James’ announcement about Satisfice India.

All the best bro!!

[James' Reply: Ainars, I don't agree that they have rote answers. I think anyone who attempts a rote answer to them will be exposed as a fool.

Also, I don't think it's fair to say that his answers "only" raise more questions. They provide useful information while raising more questions. An important quality of a good answer is that it provokes useful follow-up questions. And sometimes, the better answer is to challenge the question itself.

Knowing when to stop testing is an interesting and complex social and technical issue. I'm not surprised that Pradeep did not launch a full explanation of that point! But if you'd like one, I've already written several articles about that.] 

[James Reply: People rarely say "three minutes". But that's about the attention they will give to the answer, so you should know how to do it. Variations of this question come to testing consultants all the time. What's special about how you test? What is your test methdology? I don't understand how you test, can you explain it? Etc. The most interesting situation where I was asked this was during a court case, in a deposition. The next most interesting was when I was trying to convince investors that the company I worked for had a decent test process.

It also come up when you are training a new tester.]

Congrats man in the first place. You have more challenges that are ahead. Wish you all the best.

Venkat.

Maria

Regards,
Amol Agare

[James' Reply: I will reply, and then I will show you Pradeep's reply, which was written with independently of mine (and therefore not dictated to him by rote). You may then make your own judgment.

Rote answers may work when you are taking the ISTQB or ISEB exam, but the kind of skeptical clients we seek expect an interactive conversation. Any rote answer tends to fall apart in a conversation.

Rote answers may be feasible when you know the questions in advance, but we don't know them. I was just imagining a few for you. But even if we know some of the questions in advance, the answers to those questions are generally context-dependent, not fixed.

Most of the questions I gave are not terminology questions. In fact, only one of them seems to be:

How do you know this is good work, Pradeep? This requires the skill of doing a "good enough" analysis on a test project.

What methodology did you use? To answer this requires a description and explanation.

What alternative methodologies could you have used? To answer this requires knowledge of more than one kind of solution to testing problems. It has to do with breadth of knowledge of testing, and respect for controversy in the craft.

What is your coverage and what are your oracles? This is not requesting a definition, this is requesting a context-specific description of specifically what was tested and specifically how did the tester recognize bugs.

State your test strategy on one piece of paper. This is also context-specific. Test strategy changes over time and from project to project. Writing a test strategy is a challenging process, if you want to do it well on a single piece of paper.

Please explain testing in three minutes or less. This is not merely terminology, this is presentation. The three minute goal forces the tester to leave a lot out of the eplanation, but he must choose what to leave out and how to express the critical ideas.

What is rapid testing and how does that differ from context-driven testing? This is the only terminology question, here. I think it's possible to learn this by rote, but if you don't understand it, you won't be able to deal with follow-up questions.

As for how you recognize exceptional testers. I've written about that: to evaluate a tester, test him like a product. One thing I do is get the tester to test for me. Over and over again. And also to talk about testing.

Now, here is Pradeep's unedited reply. You can tell that English is not his first language. I'm working with him to improve his writing skill for an American audience. But focus for a moment on the underlying message. Is this someone who seems to have ideas? Ideas are the mainspring of testing...

The biggest challenge everyone faces is to explain "what is *testing*, according to them?". Its a philosophy that comes out when someone explains testing. If you agree to someone else, you should also be able to tell, "Why you agree with them?" "What experiments you did to agree to someone else's definition/explanation of a terminology". I feel, the questions are really challenging. Answering to questions about terminology and or testing definitions might look simple to someone who holds a certification because there was only one definition he/she came across about a specific terminology, but as an explorer, I see a lot of dimensions to it.

For example - “How do you know this is good work, Pradeep?
Isn't this a challenging question for a tester to answer?

These are the ways, I identify that I did a good work - ( I better say "good enough" instead of "good" )

A sample of questions that I'd ask myself are -

1. Did I understand the mission that the client set?
2. Did I use all possible resources that were available ( hidden or obvious )?
3. Did I use the right heuristics based on the mission?
4. Did I keep thinking about the cost v/s value and did I take the necassary actions to keep on track?
5. Did I state my assumptions, well before I could go wrong anywhere with it?
6. Did I recover from my mistakes, rapidly?
7. Did I get more ideas from people around me?
8. Did I apply my past learning?
9. Did I diversify my ideas?
10. Did I stop testing at the right time?
11. Did I learn the program?
12. Did I report effectively?
13. Did I do something new than I did not before, with similar exercises/mission?
14. Did I give a report that contained quality related information about the product I am testing, to help the management take informed decisions?

Actually, there is a huge list, which I might run quickly in my mind before I say, "I conjecture that I did a good enough job". Wait, I might also ask a question "Did I miss asking an important question to myself?"

This list is something that I prepared to suit me and not given by James Bach, as a handbook for joining Satisfice. It might not work for you nor the things that has been working for you would work for me. I might change questions, add new ones, chuck out some based on my future experiments with it. I am willing to learn from you, too.

Also, the ways I feel that might help in identifying exceptional testers are by 1) the kind of problems they solve 2) the methods they use to solve a problem 3) the rapidness in solving such problems 4) the ability to help someone solve such a problem 5) the ability to inspire people by solving such problems 6) the ability to explain, "how to solve such a problem?" 7) the ability to learn from anything and everything around 8 ) the ability nurture someone who wants to solve such problems in the future 9) the ability to accept that, "there could be more dimensions to becoming an exceptional tester".

Thanks for putting up a question that made me list something that I can refer to, anytime I want to check - "Am I practicing the things I listed here?". I go by this saying that I heard from one of my guru - "A saint is someone who practices "prayers". The moment he stops practicing, he is no more a saint". I have remained a tester all this while and will die as a tester.

-- Pradeep Soundararajan ]

Wish you best.

Niteen Yemul