Accountability for What You Say is Dangerous and That’s Okay

[Note: I offered Maaret Pyhäjärvi the right to review this post and suggest edits to it before I published it. She declined.]

A few days ago I was keynoting at the New Testing Conference, in New York City, and I used a slide that has offended some people on Twitter. This blog post is intended to explore that and hopefully improve the chances that if you think I’m a bad guy, you are thinking that for the right reasons and not making a mistake. It’s never fun for me to be a part of something that brings pain to other people. I believe my actions were correct, yet still I am sorry that I caused Maaret hurt, and I will try to think of ways to confer better in the future.

Here’s the theme of this post: Getting up in front of the world to speak your mind is a dangerous process. You will be misunderstood, and that will feel icky. Whether or not you think of yourself as a leader, speaking at a conference IS an act of leadership, and leadership carries certain responsibilities.

I long ago learned to let go of the outcome when I speak in public. I throw the ideas out there, and I do that as an American Aging Overweight Left-Handed Atheist Married Father-And-Father-Figure Rough-Mannered Bearded Male Combative Aggressive Assertive High School Dropout Self-Confident Freedom-Loving Sometimes-Unpleasant-To-People-On-Twitter Intellectual. I know that my ideas will not be considered in a neutral context, but rather in the context of how people feel about all that. I accept that.  But, I have been popular and successful as a speaker in the testing world, so maybe, despite all the difficulties, enough of my message and intent gets through, overall.

What I can’t let go of is my responsibility to my audience and the community at large to speak the truth and to do so in a compassionate and reasonable way. Regardless of what anyone else does with our words, I believe we speakers need to think about how our actions help or harm others. I think a lot about this.

Let me clarify. I’m not saying it’s wrong to upset people or to have disagreement. We have several different culture wars (my reviewers said “do you have to say wars?”) going on in the software development and testing worlds right now, and they must continue or be resolved organically in the marketplace of ideas. What I’m saying is that anyone who speaks out publicly must try to be cognizant of what words do and accept the right of others to react.

Although I’m surprised and certainly annoyed by the dark interpretations some people are making of what I did, the burden of such feelings is what I took on when I first put myself forward as a public scold about testing and software engineering, a quarter century ago. My annoyance about being darkly interpreted is not your problem. Your problem, assuming you are reading this and are interested in the state of the testing craft, is to feel what you feel and think what you think, then react as best fits your conscience. Then I listen and try to debug the situation, including helping you debug yourself while I debug myself. This process drives the evolution of our communities. Jay Philips, Ash Coleman, Mike Talks, Ilari Henrik Aegerter, Keith Klain, Anna Royzman, Anne-Marie Charrett, David Greenlees, Aaron Hodder, Michael Bolton, and my own wife all approached me with reactions that helped me write this post. Some others approached me with reactions that weren’t as helpful, and that’s okay, too.

Leadership and The Right of Responding to Leaders

In my code of conduct, I don’t get to say “I’m not a leader.” I can say no one works for me and no one has elected me, but there is more to leadership than that. People with strong voices and ideas gain a certain amount of influence simply by virtue of being interesting. I made myself interesting, and some people want to hear what I have to say. But that comes with an implied condition that I behave reasonably. The community, over time negotiates what “reasonable” means. I am both a participant and a subject of those negotiations. I recommend that we hold each other accountable for our public, professional words. I accept accountability for mine. I insist that this is true for everyone else. Please join me in that insistence.

People who speak at conferences are tacitly asserting that they are thought leaders– that they deserve to influence the community. If that influence comes with a rule that “you can’t talk about me without my permission” it would have a chilling effect on progress. You can keep to yourself, of course; but if you exercise your power of speech in a public forum you cannot cry foul when someone responds to you. Please join me in my affirmation that we all have the right of response when a speaker takes the microphone to keynote at a conference.

Some people have pointed out that it’s not okay to talk back to performers in a comedy show or Broadway play. Okay. So is that what a conference is to you? I guess I believe that conferences should not be for show. Conferences are places for conferring. However, I can accept that some parts of a conference might be run like infomercials or circus acts. There could be a place for that.

The Slide

Here is the slide I used the other day:

maaret

Before I explain this slide, try to think what it might mean. What might its purposes be? That’s going to be difficult, without more information about the conference and the talks that happened there. Here are some things I imagine may be going through your mind:

  • There is someone whose name is Maaret who James thinks he’s different from.
  • He doesn’t trust nice people. Nice people are false. Is Maaret nice and therefore he doesn’t trust her, or does Maaret trust nice people and therefore James worries that she’s putting herself at risk?
  • Is James saying that niceness is always false? That’s seems wrong. I have been nice to people whom I genuinely adore.
  • Is he saying that it is sometimes false? I have smiled and shook hands with people I don’t respect, so, yes, niceness can be false. But not necessarily. Why didn’t he put qualifying language there?
  • He likes debate and he thinks that Maaret doesn’t? Maybe she just doesn’t like bad debate. Did she actually say she doesn’t like debate?
  • What if I don’t like debate, does that mean I’m not part of this community?
  • He thinks excellence requires attention and energy and she doesn’t?
  • Why is James picking on Maaret?

Look, if all I saw was this slide, I might be upset, too. So, whatever your impression is, I will explain the slide.

Like I said I was speaking at a conference in NYC. Also keynoting was Maaret Pyhäjärvi. We were both speaking about the testing role. I have some strong disagreements with Maaret about the social situation of testers. But as I watched her talk, I was a little surprised at how I agreed with the text and basic concepts of most of Maaret’s actual slides, and a lot of what she said. (I was surprised because Maaret and I have a history. We have clashed in person and on Twitter.) I was a bit worried that some of what I was going to say would seem like a rehash of what she just did, and I didn’t want to seem like I was papering over the serious differences between us. That’s why I decided to add a contrast slide to make sure our differences weren’t lost in the noise. This means a slide that highlights differences, instead of points of connection. There were already too many points of connection.

The slide was designed specifically:

  • for people to see who were in a specific room at a specific time.
  • for people who had just seen a talk by Maaret which established the basis of the contrast I was making.
  • about differences between two people who are both in the spotlight of public discourse.
  • to express views related to technical culture, not general social culture.
  • to highlight the difference between two talks for people who were about to see the second talk that might seem similar to the first talk.
  • for a situation where both I and Maaret were present in the room during the only time that this slide would ever be seen (unless someone tweeted it to people who would certainly not understand the context).
  • as talking points to accompany my live explanation (which is on video and I assume will be public, someday).
  • for a situation where I had invited anyone in the audience, including Maaret, to ask me questions or make challenges.

These people had just seen Maaret’s talk and were about to see mine. In the room, I explained the slide and took questions about it. Maaret herself spoke up about it, for which I publicly thanked her for doing so. It wasn’t something I was posting with no explanation or context. Nor was it part of the normal slides of my keynote.

Now I will address some specific issues that came up on Twitter:

1. On Naming Maaret

Maaret has expressed the belief that no one should name another person in their talk without getting their permission first. I vigorously oppose that notion. It’s completely contrary to the workings of a healthy society. If that principle is acceptable, then you must agree that there should be no free press. Instead, I would say if you stand up and speak in the guise of an expert, then you must be personally accountable for what you say. You are fair game to be named and critiqued. And the weird thing is that Maaret herself, regardless of what she claims to believe, behaves according to my principle of freedom to call people out. She, herself, tweeted my slide and talked about me on Twitter without my permission. Of course, I think that is perfectly acceptable behavior, so I’m not complaining. But it does seem to illustrate that community discourse is more complicated than “be nice” or “never cause someone else trouble with your speech” or “don’t talk about people publicly unless they gave you permission.”

2. On Being Nice

Maaret had a slide in her talk about how we can be kind to each other even though we disagree. I remember her saying the word “nice” but she may have said “kind” and I translated that into “nice” because I believed that’s what she meant. I react to that because, as a person who believes in the importance of integrity and debate over getting along for the sake of appearances, I observe that exhortations to “be nice” or even to “be kind” are often used when people want to quash disturbing ideas and quash the people who offer them. “Be nice” is often code for “stop arguing.” If I stop arguing, much of my voice goes away. I’m not okay with that. No one who believes there is trouble in the world should be okay with that. Each of us gets to have a voice.

I make protests about things that matter to me, you make protests about things that matter to you.

I think we need a way of working together that encourages debate while fostering compassion for each other. I use the word compassion because I want to get away from ritualized command phrases like “be nice.” Compassion is a feeling that you cultivate, rather than a behavior that you conform to or simulate. Compassion is an antithesis of “Rules of Order” and other lists of commandments about courtesy. Compassion is real. Throughout my entire body of work you will find that I promote real craftsmanship over just following instructions. My concern about “niceness” is the same kind of thing.

Look at what I wrote: I said “I don’t trust nice people.” That’s a statement about my feelings and it is generally true, all things being equal. I said “I’m not nice.” Yet, I often behave in pleasant ways, so what did I mean? I meant I seek to behave authentically and compassionately, which looks like “nice” or “kind”, rather than to imagine what behavior would trick people into thinking I am “nice” when indeed I don’t like them. I’m saying people over process, folks.

I was actually not claiming that Maaret is untrustworthy because she is nice, and my words don’t say that. Rather, I was complaining about the implications of following Maaret’s dictum. I was offering an alternative: be authentic and compassionate, then “niceness” and acts of kindness will follow organically. Yes, I do have a worry that Maaret might say something nice to me and I’ll have to wonder “what does that mean? is she serious or just pretending?” Since I don’t want people to worry about whether I am being real, I just tell them “I’m not nice.” If I behave nicely it’s either because I feel genuine good will toward you or because I’m falling down on my responsibility to be honest with you. That second thing happens, but it’s a lapse. (I do try to stay out of rooms with people I don’t respect so that I am not forced to give them opinions they aren’t willing or able to process.)

I now see that my sentence “I want to be authentic and compassionate” could be seen as an independent statement connected to “how I differ from Maaret,” implying that I, unlike her, am authentic and compassionate. That was an errant construction and does not express my intent. The orange text on that line indicated my proposed policy, in the hope that I could persuade her to see it my way. It was not an attack on her. I apologize for that confusion.

3. Debate vs. Dialogue

Maaret had earlier said she doesn’t want debate, but rather dialogue. I have heard this from other Agilists and I find it disturbing. I believe this is code for “I want the freedom to push my ideas on other people without the burden of explaining or defending those ideas.” That’s appropriate for a brainstorming session, but at some point, the brainstorming is done and the judging begins. I believe debate is absolutely required for a healthy professional community. I’m guided in this by dialectical philosophy, the history of scientific progress, the history of civil rights (in fact, all of politics), and the modern adversarial justice system. Look around you. The world is full of heartfelt disagreement. Let’s deal with it. I helped create the culture of small invitational peer conferences in our industry which foster debate. We need those more than ever.

But if you don’t want to deal with it, that’s okay. All that means is that you accept that there is a wall between your friends and those other people whom you refuse to debate with. I will accept the walls if necessary but I would rather resolve the walls. That’s why I open myself and my ideas for debate in public forums.

Debate is not a process of sticking figurative needles into other people. Debate is the exchange of views with the goal of resolving our differences while being accountable for our words and actions. Debate is a learning process. I have occasionally heard from people I think are doing harm to the craft that they believe I debate for the purposes of hurting people instead of trying to find resolution. This is deeply insulting to me, and to anyone who takes his vocation seriously. What’s more, considering that these same people express the view that it’s important to be “nice,” it’s not even nice. Thus, they reveal themselves to be unable to follow their own values. I worry that “Dialogue not debate” is a slogan for just another power group trying to suppress its rivals. Beware the Niceness Gang.

I understand that debating with colleagues may not be fun. But I’m not doing it for fun. I’m doing it because it is my responsibility to build a respectable craft. All testing professionals share this responsibility. Debate serves another purpose, too, managing the boundaries between rival value systems. Through debate we may discover that we occupy completely different paradigms; schools of thought. Debate can’t bridge gaps between entirely different world views, and yet I have a right to my world view just as you have a right to yours.

Jay Philips said on Twitter:

I admire Jay. I called her and we had a satisfying conversation. I filled her in on the context and she advised me to write this post.

One thing that came up is something very important about debate: the status of ideas is not the only thing that gets modified when you debate someone; what also happens is an evolution of feelings.

Yes I think “I’m right.” I acted according to principles I think are eternal and essential to intellectual progress in society. I’m happy with those principles. But I also have compassion for the feelings of others, and those feelings may hold sway even though I may be technically right. For instance, Maaret tweeted my slide without my permission. That is copyright violation. She’s objectively “wrong” to have done that. But that is irrelevant.

[Note: Maaret points out that this is legal under the fair use doctrine. Of course, that is correct. I forgot about fair use. Of course, that doesn’t change the fact that though I may feel annoyed by her selective publishing of my work, that is irrelevant, because I support her option to do that. I don’t think it was wise or helpful for her to do that, but I wouldn’t seek to bar her from doing so. I believe in freedom to communicate, and I would like her to believe in that freedom, too]

I accept that she felt strongly about doing that, so I [would] choose to waive my rights. I feel that people who tweet my slides, in general, are doing a service for the community. So while I appreciate copyright law, I usually feel okay about my stuff getting tweeted.

I hope that Jay got the sense that I care about her feelings. If Maaret were willing to engage with me she would find that I care about her feelings, too. This does not mean she gets whatever she wants, but it’s a factor that influences my behavior. I did offer her the chance to help me edit this post, but again, she refused.

4. Focus and Energy

Maaret said that eliminating the testing role is a good thing. I worry it will lead to the collapse of craftsmanship. She has a slide that says “from tester to team member” which is a sentiment she has expressed on Twitter that led me to say that I no longer consider her a tester. She confirmed to me that I hurt her feelings by saying that, and indeed I felt bad saying it, except that it is an extremely relevant point. What does it mean to be a tester? This is important to debate. Maaret has confirmed publicly (when I asked a question about this during her talk) that she didn’t mean to denigrate testing by dismissing the value of a testing role on projects. But I don’t agree that we can have it both ways. The testing role, I believe, is a necessary prerequisite for maintaining a healthy testing craft. My key concern is the dilution of focus and energy that would otherwise go to improving the testing craft. This is lost when the role is lost.

This is not an attack on Maaret’s morality. I am worried she is promoting too much generalism for the good of the craft, and she is worried I am promoting too much specialism. This is a matter of professional judgment and perspective. It cannot be settled, I think, but it must be aired.

The Slide Should Not Have Been Tweeted But It’s Okay That It Was

I don’t know what Maaret was trying to accomplish by tweeting my slide out of context. Suffice it to say what is right there on my slide: I believe in authenticity and compassion. If she was acting out of authenticity and compassion then more power to her. But the slide cannot be understood in isolation. People who don’t know me, or who have any axe to grind about what I do, are going to cry “what a cruel man!” My friends contacted me to find out more information.

I want you to know that the slide was one part of a bigger picture that depicts my principled objection to several matters involving another thought leader. That bigger picture is: two talks, one room, all people present for it, a lot of oratory by me explaining the slide, as well as back and forth discussion with the audience. Yes, there were people in the room who didn’t like hearing what I had to say, but “don’t offend anyone, ever” is not a rule I can live by, and neither can you. After all, I’m offended by most of the talks I attend.

Although the slide should not have been tweeted, I accept that it was, and that doing so was within the bounds of acceptable behavior. As I announced at the beginning of my talk, I don’t need anyone to make a safe space for me. Just follow your conscience.

What About My Conscience?

  • My conscience is clean. I acted out of true conviction to discuss important matters. I used a style familiar to anyone who has ever seen a public debate, or read an opinion piece in the New York Times. I didn’t set out to hurt Maaret’s feelings and I don’t want her feelings to be hurt. I want her to engage in the debate about the future of the craft and be accountable for her ideas. I don’t agree that I was presuming too much in doing so.
  • Maaret tells me that my slide was “stupid and hurtful.” I believe she and I do not share certain fundamental values about conferring. I will no longer be conferring with her, until and unless those differences are resolved.
  • Compassion is important to me. I will continue to examine whether I am feeling and showing the compassion for my fellow humans that they are due. These conversations and debates I have with colleagues help me do that.
  • I agree that making a safe space for students is important. But industry consultants and pundits should be able to cope with the full spectrum, authentic, principled reactions by their peers. Leaders are held to a higher standard, and must be ready and willing to defend their ideas in public forums.
  • The reaction on Twitter gave me good information about a possible trend toward fragility in the Twitter-facing part of the testing world. There seems to be a significant group of people who prize complete safety over the value that comes from confrontation. In the next conference I help arrange, I will set more explicit ground rules, rather than assuming people share something close to my own sense of what is reasonable to do and expect.
  • I will also start thinking, for each slide in my presentation: “What if this gets tweeted out of context?”

(Oh, and to those who compared me to Donald Trump… Can you even imagine him writing a post like this in response to criticism? BELIEVE ME, he wouldn’t.)

Competent People and Conference Keynotes

My colleague and friend Anne-Marie Charrett has a thing about women. A) She is one. B) She feels that not enough of them are speaking at testing conferences. (See also Fiona Charles’ post on this subject.) I support Anne-Marie’s cause partly because I support the woman herself and it would make her happy. This is how humanity works: we are tribal creatures. Don’t deny it, you will just sound silly.

There is another reason I support their cause, though. It’s related to the fact that we people are not only tribal creatures. We are also creatures of myth, story, and principle. Each of us lives inside a story, and we want that story to “win,” whatever that may mean to us. Apart from tribal struggles, there is a larger meta-tribal struggle over what constitutes the “correct” or “good” or “moral” story.

In other words, it isn’t only whom we like that motivates us, but also what seems right. I’m not religious, so I won’t bother to talk about that aspect of things. But in the West, the professional status of women is a big part of the story of good and proper society; about what seems right.

The story I’m living, these days, is about competence. And I think most people speaking at testing conferences are not competent enough. A lot of what’s talked about at testing conferences is the muttering of idiots. By idiot, I mean functionally stupid people: people who choose not to use their minds to find excellent solutions to important problems, but instead speak ritualistically and uncritically about monsters and angels and mathematically invalid metrics and fraudulent standards and other useless or sinister tools that are designed to amaze and confuse the ignorant.

I want to see at least 50% of people speaking at conferences to be competent. That’s my goal. I think it is achievable, but it will take a lot of work. We are up against an entrenched and powerful interest: the promoters-of-ineptness (few of whom realize the damage they do) who run the world and impose themselves on our craft.

Why are there so many idiots and why do they run the world? The roots and dynamics of idiocracy are deep. It’s a good question but I don’t want to go into it here and now.

What I want to say is that Anne-Marie and Fiona, along with some others, can help me, and I can help them. I want to encourage new voices to take a place in the Great Conversation of testing because I do believe there is an under-tapped pool of talent among the women of testing. I am absolutely opposed to quotas or anything that simply forces smaller people with higher voices onto the stage for the sake of it. Instead let’s find and develop talent that leads us into a better future. This is what SpeakEasy is all about.

Maybe, if we can get more women speaking and writing in the craft, we will be able to imagine a world where more than 50% of keynote speakers are not spouting empty quotes from great thinkers and generic hearsay about projects and incoherent terminology and false dichotomies and ungrounded opinions and unworkable heuristics presented in the form of “best practices.”

I am not a feminist. I’m not going to be one. This is why I have work to do. I am not naturally biased in favor of considering women, and even if I were, can I be so sure that I’m not biased in favor of the attractive ones? Or against them? Research suggests no one can be complacent about overcoming our biology and gender identity. So, it’s a struggle. Any honest man will tell you that. And, I must engage that struggle while maintaining my implacable hostility to charlatans and quacks. The story I am living tells me this is what I must do. Also, Anne-Marie has asked for my help.

Here’s my call to action: To bring new beautiful minds forth to stand against mediocrity, we need to make the world a better, friendlier place especially for the women among us. I’m asking all you other non-feminists out there to consider working with me on this.

Integrity #3: A Testimonial

Oliver Erlewein is an automation specialist. He’s respected in the Context-Driven Testing community of New Zealand and has been an agitator pushing back against the ISTQB. After some years of frustration with bad management he finally went independent. Now he’s back to full-time work. He posted the following as a comment:

Starting 2014, I have given up my self-employment and joined a (sort of) start up. I didn’t think I was ever going back to being employed but this was worth it. I have found a company that respects my professionalism and listens to what I say, where I am responsible for what I produce and get the full control of how to go about it. I and the task I do are respected. The word integrity doesn’t get used here but it is a place that actually has oodles of it.

Every now and again I hear the sentence “you are the expert so what do you suggest we do?” or “do what you think is right, you are the expert” ….and they mean it exactly like that. It makes for a completely different working environment. It motivates, it invigorates and it makes working fun. It puts heaps more pressure and responsibility on me but I am happy as taking that on board because I am convinced that I can do it (even if I still don’t know how right at this moment).

Although this shop is not agile (but more agile than a lot of the shops out there that call themselves agile!) they do something that is one of the main success factors for agile: They re-introduce back the idea of responsibility, professionalism and craftsmanship into (IT) work. And that motivates. I feel like I can call bull**** if it is appropriate to do so or get traction on subjects I think are important.

So although it meant I made a career change away from my original trajectory I made it consciously towards a more ethical work life, where integrity and being the best you can actually counts for something.

Thank you for sharing that with us, Oliver. It goes to show that there are good managers out there who understand craftsmanship and leadership.

 

Mr. Langella Never Does it the Same Way Twice

This is from the New York Times:

Its other hallmark is that Mr. Langella never does the part the same way twice. This is partly because he’s still in the process of discovering the character and partly because it’s almost a point of honor. “The Brit approach is very different from mine,” he said. “There’s a tendency to value consistency over creativity. You get it, you nail it, you repeat it. I’d rather hang myself. To me, every night within a certain framework — the framework of integrity — you must forget what you did the night before and create it anew every single time you walk out on the stage.”

I love that phrase the framework of integrity. It ties in to what I’ve been saying about integrity and also what is true about informal testing: if you are well prepared, and you are true to yourself, then whatever you do is going to be spontaneously and rather effortlessly okay, even if it changes over time.

I often hear anxiety in testers and managers about how terrible it is to do something once, some particular way, and then to forget it. What a waste, they say. Let’s write it all down and not deviate from our past behavior, they say. Well I don’t think it’s waste, I think it’s mental hygiene. Testing is a performance, and I want to be free to perform better. So, I make notes, sure. But I am properly reluctant about formalizing what I do.

Doing your best work includes having the courage to let go of pretty good work that you’ve done before.

Integrity 2: On Being Under the Radar [REVISED]

I have taken down the original text of this post at the request of my colleague who had the courage and audacity to let me post his detailed comment about how he works “under the radar” to change things in his company.

I had posted his comment originally with his permission, of course. But, apparently, in his country, “it’s illegal to harm [one’s] employer’s business” and it can reasonably be considered doing harm to express a low opinion of your own company’s behavior, even if you are dedicated to improving that behavior. Dirty laundry in public is arguably bad for business, if your business involves telling people that you’re a trustworthy expert, and your laundry says otherwise.

Of course this is understandable. Working under the radar generally means not being public about what you are doing. Therefore, as much as I prefer the clean feeling of working ON the radar, I wish him good luck with his mode of influencing a big, commercial, ceremonial system.

 

 

 

 

Finding Your Own Integrity

I have a belief that I’m not going to justify– I’m simply going to say it and challenge you to look into your own experience and your own heart and see the truth of it for yourself: Your sense of identity, as a human among humans, is the most powerful force that animates and directs your choices. It is more important than sex or food or religion. It lurks behind every neurosis (including those involving sex or food or religion). As I read history and experience life, answers to the questions “Who am I? Am I a good example of what I should be?” are the prime movers of human choice throughout all of history, and the proximal cause of every war.

There are certainly exceptions to this rule: drug addiction, mental illness, or panic over a sudden, surprising, physical threat. Maybe those things have little to do with identity. Granted. I’m talking about normal daily life (and every Shakespeare play).

“I am an American. I am a human. I am a father. I am a husband. I am lovable. I am helpful. I am a tester. I am a skeptic. I am an outsider. I am dangerous. I am safe. I am honorable. I am fallible. I am truthful. I am intellectual…”  Each of these statements, for me, are reflective shards that tumble in a kaleidoscope of my identity. The models of personhood they represent comprise my moral compass. Although the pattern formed in that kaleidoscope may seem to shift with the situation, the underlying logic of my adult identity changes little with time.

That is the context for integrity.

Integrity means wholeness; the harmony and completeness of one’s identity. Practically speaking, a person with integrity is a person to lives consistently according to their avowed moral code, as opposed to someone who has no moral code, or who changes it as a matter of convenience. A person of integrity therefore creates continuity across the events of his life, and other people feel they know who they are dealing with.

The Challenge of Finding Your Integrity

Recently, in a discussion about what is reasonable for an employer to ask of a tester, a colleague felt I was trying to impose my own values onto potential employers of my students and wrote that as teachers of new testers “employment [for the testers] should be our first priority.” I disagreed sharply, writing that “our first priority is integrity.” My correspondent seemed to take offense to that.

Now, the employment-first position might be construed to imply that we should advocate robbing banks, because it is the quickest way to get money, or perhaps we should train prostitutes, because prostitution is an old and reliable industry with lots of job security for the right people. That would be absurd, but it’s also a straw man argument. I am certain no one intends to argue that any job is better than no job. Safety, legality and morality do enter into the equation.

Conversely, the integrity-first position might be cast as requiring a tester to immediately protest or resign in the face of any ethical dilemma or systemic ethical lapse, no matter how seemingly minor. This would turn most testers into insufferable, dour lawyers on their projects. We would get very little done. Who would hire such people?

These extreme positions are not very interesting, except as tools for meditating on what might be reasonable behavior. Therefore, I’d like to describe a less extreme position that I think is more defensible and workable. It goes like this:

1. Integrity is a vital and important matter. We suffer as people and society suffers when we treat it too lightly.

2. As testers and technical people, our integrity is routinely threatened by well-meaning clients and colleagues who want us to portray ourselves and the world to be a certain way, even if that isn’t strictly the truth.

3. If we never think directly about integrity, and simply trust in the innate goodness of ourselves and others, we are definitely taking this matter too lightly.

4. Integrity is not like a vase that shatters easily, and that once shattered is irretrievable. Integrity is more like an ongoing public artwork, exposed to and buffeted by the elements, sometimes damaged but always ultimately repairable (although our reputation may be another matter). Integrity is a work in progress for all of us.

5. Integrity, like education, is both personal and social. Your society judges you. It is reasonable that it does. But it is also reasonable to negotiate limits on that judgment. We spend our lives negotiating those lines, one way or another.

6. Forgiveness, although perhaps difficult and expensive to obtain, should always be available to us. (I test this by occasionally imagining my most “depraved” enemies in testing, and then imagining what they could do that would allow me to forgive them and even collaborate with them.)

7. Although integrity is our highest priority, in general, it is not the only thing that matters. We must apply wisdom and judgment so that the maintenance of integrity does not unreasonably affect our ability to survive. There is no set formula for how to do that.

8. Therefore, our practical priority must be: to learn how to think through and solve problems of survival while maintaining reasonable integrity. This itself is an ongoing project, requiring temperance and self-forgiveness.

9. New testers need to realize that they are not necessarily responsible for the quality of their work. Sometimes you will be asked to do things you don’t understand the value of, even though there may be value. In those situations, it’s okay to be compliant, as long as you are under supervision and someone competent is taking responsibility for what you do. It’s okay to watch and learn and not necessarily to make trouble. (Although, I usually did, even as a newbie.)

10. Experienced testers? Well, much is expected of you. Your clients (your non-tester colleagues and bosses) don’t know how to test, but you are supposed to. You can’t just do what you are told. That would be like letting a drunk friend drive home. Remember, someday your clients may sober up and wonder why you agreed to their stupid plan when you were supposed to be the expert.

Having laid this hopefully reasonable and workable strategy before you… I actually think the dispute between me and my correspondent, above, was not about the importance of integrity or employment at all, but rather about the specifics of the case we were debating. I should tell you what that was: whether it is reasonable for an employer to expect an entry-level tester to “write test cases.”

From a context-driven testing perspective, no practice can be declared unreasonable outside all contexts. But I do know a lot about the typical contexts of testing. I have seen profound waste, all around the industry, due to reckless and thoughtless documenting and counting of things called “test cases.” So, I don’t think that it is reasonable, generally speaking, to require young testers to write test cases. First, because “writing test cases” is what people who don’t know how to test think testers do– so, it’s usually an indicator of incompetent management. Second, because entry-level testers do not have the skills to write test cases in such a way that won’t result in a near complete waste of their client’s time and money. And third, because there are such obviously better things to do, in most cases, including learning about the product and actually testing the product.

Many people disagree with me. But I believe their attitude on this is the single most direct and vital cause of the perpetual infancy and impotency that we see in the testing industry. In other words, it’s not just a disagreement about style, it’s something that I think threatens our integrity as sincere and thoughtful testers. Casual shrugging about test case writing must be stamped out the way transfats are being outlawed in fast food. Yes, that also took years to accomplish.

Speaking of fast food…

Here’s a metaphor that might help: eating at McDonalds.

Eating at McDonalds will not kill you (well, not outright). But what if you were forced to eat at McDonalds for your work? Every day, breakfast, lunch and dinner. Nothing but McDonalds. What if it were obvious to you that eating at McDonalds was not helping you actually succeed in your work? What if instead it was clear to you that such a diet was harming your ability to get your work done? For instance, perhaps you are a restaurant reviewer, except you are almost always full of McDonalds food so you can’t ever enjoy a proper meal at a restaurant you are supposed to review? And yet your manager, who knows nothing about restaurant reviewing, insists that you maintain a McDonalds-dominated dietary regimen.

Couldn’t someone say, hey, it’s a job and you should do what you are told? Yes, they could say that. And it might be true enough at first. But over time, that diet would hurt you; over time, you would have to cope with how poorly you were doing what you believed to be your real job. You might even be criticized for missing bugs– I mean– failing to review restaurants fully, even though it’s largely due to your employer’s own unreasonable process prescriptions.

At some point you might say “enough!!” You might refuse to eat another Big Mac. From the point of view of your management and colleagues, it might look like you were risking your job just because you didn’t want to eat a hamburger. It might look crazy to them. But from your point of view, the issue isn’t the one burger, but rather the unhealthy system, slowly killing you. This breakdown comes more quickly if you happen to have a gluten allergy.

Ethics and integrity in testing is not just about following prissy little rules that many other people flout– it’s about not making yourself sick even if other people are willing to live a sickly life. This requires that you be able to judge what health and sickness means to you. Integrity is about identity health.

A Story of Quitting Even Though I Needed the Work

In 1998, I was hired by a consulting company outside of Washington D.C. I negotiated for a $30,000 sign-on bonus, and bought a house in Virginia. I was the sole breadwinner in my family, with a wife and son to support. I bought a new car, too. In short, I was “all in.”

Six months later, I quit. I had no other job to go to. I had bills due. It took me seven years to pay back my sign-on bonus, with interest (I forfeited it because I did not stay for two years). But with the help of colleagues and family over the following weeks, I made the transition to running my own business. I am most thankful for my wife’s response when I came home that night and told her I walked out on our only source of income. She shrugged and said it was surely for the best, and that something good would come of it. (I can only recommend, gentlemen, that you marry an optimist if you can.) I am also thankful to Cem Kaner, who bought me a laptop (my only computer was then owned by my employer) and said “times like these are when you discover who your true friends are.” This was fitting because it was partly because of Cem that I had previously decided never to sacrifice my professional integrity.

This illustrates one lesson about ethics: community support helps us find and maintain our integrity.

I quit because my company was insisting that I bill hours on a project that, in my opinion, was absolutely certain not to benefit from my work. The client wanted me to create fake test cases. They didn’t call them fake test cases, of course. They claimed to want real test cases; and good ones. But no product had been designed at that time! All I had access to was a general description of requirements, which in this case were literally statements of problems the product was intended to solve, with no information on how they would be solved. It was a safety-critical factory process control system, and no one could show me what it would look like or provide any examples of what specifically it might do. The only test cases I could possibly design would therefore be vague and imaginary, based on layers of soft, fluffy assumptions. The customer told me they would be happy if I delivered a document that consisted of the text of each requirement preceded by the phrase “verify that…” I told them they didn’t need a tester for that. They needed a macro.

The integrity picture was clouded, in that case, because the client believed they had to follow the “V-Model” process, which they had interpreted as demanding that I submit a test case specification document. It was a clash between the integrity of a heuristic (the V-Model) vs. the integrity of solving the problem for which the heuristic was designed. My client might have said that I was the one violating the integrity of the process. Whereas I would have said that my client was not competent to make that judgment.

I’m not saying I won’t do bad work… I’m just saying I won’t do bad work for money. If I do bad work, I want it to be for fun or for learning, but not to anyone’s expense or detriment. Hence a line I use once in a while “I could do that for you, except that you pay me too much.” This is one reason I like being independent. I control what I bill for, and if I think a portion of my work is not acceptable, I don’t charge for it– like a chef who refuses to serve an overcooked steak.

It wasn’t as sudden as it looked…

I didn’t just lose my temper at the first sign of trouble. Things had been coming to a boil for a while. On my very first day I reviewed the RFP for that project and concluded it was doomed, but management bid on it anyway, telling me I needed to “be practical” and that surely “we could be helpful to them if they hired us.” I needed the job, so I relented against my better judgment.

During my first staff meeting, my first week on the job, I challenged the consulting staff about what they did to study testing on their own time. My challenge was met with an awkward silence, after which one of the consultants, sounding soul-wounded, told me he was offended that I would suggest that they weren’t already good enough as testers, “These are the best people I’ve ever worked with” said the twenty-something tester with little experience and no public reputation. “But how do you know they are good?” I asked, knowing that our company had just issued a press release about having hired me (a “distinguished industry pioneer” to quote it exactly). There were other murmurs of annoyance around the table, and the manager stepped in to change the subject. I could have pushed the issue, but I didn’t. I needed the job, so I relented against my better judgment.

I was later told that despite my company’s public position, the other consultants felt that I was a mere armchair expert, whereas they were practical men. I don’t know what evidence they had for that. They never showed me what they could do that I supposedly could not. Management tolerated this attitude. That means they were lying directly to their customers about me– claiming I was an expert when clearly they did not believe I was one. I could have insisted they behave in accordance with their public statements about me. But… I needed the job, so I relented against my better judgment.

I knew the day had come when I must quit because I found myself fantasizing about throwing chairs through windows. That triggered a sort of circuit-breaker of judgment: change your life now, now, now.

So what happened after that?

I suffered for this decision. First came the panic attack. I felt dizzy and it was hard to breathe for a few hours. This was followed by a few years of patching together a project here and a project there, never more than 8 weeks from completely running out of money and credit. We were twice within a week of filing for bankruptcy in the early days. During that time I walked away from a few more projects. I resigned from a dysfunctional government project, hopefully saving valuable taxpayer dollars by not writing a completely unnecessary Software Configuration Management plan that no one on the team wanted. I got myself fired from a project at Texas Instruments after about 90 minutes, because I told them a few things they didn’t want to hear (but that I felt were both true and important).

It’s not all suffering, of course. I once was fired from a project (along with the rest of the test team) and then was the only one hired back– partly because the client realized that my high standards meant that I billed far fewer hours than other consultants. In other words, saying no and being a troublemaker earned me another 500 hours of work, while the yes-sayers lost their situations. I also got some great gigs, including my very first one as an independent, specifically because I am a rabble-rousing kind of thinker.

These days, I cultivate as many clients as I can, so that I don’t rely too much on any one of them. And I have established a reputation for being honest and blunt that generally prevents the wrong sort of people from trying to hire me. It’s not easy, but it can be done: I have made integrity my top priority.

What about before I was well known?

Well, I’ve always had this attitude. It’s not some luxury to me. It’s fundamental. That’s why I had to leave high school. I’ve never been able to “play the game” at the expense of feeling like a good honest man. Like I said, I suffered for it. I wanted to go try myself at MIT, where my much more pliable best friend from high school eventually graduated. I am born to be an academic, but since I can’t stand the compliance-to-ceremony culture of the academic world, I must be an independent scholar, without access to expensive journals and fantastic libraries.

Before anybody heard of me, I treated getting work like dating: be a slightly exaggerated version of myself so that I will be rejected quickly if the relationship is not a fit (a stress testing strategy, you might say). My big break came at Apple, where I worked for a man of vision and good humor who seemed to relish being the mentor I needed. The environment was open and supportive. There was an element of luck in that my first ten years in testing I worked for people who didn’t ask me to tell lies or do bad work on purpose.

So I know it’s possible to find such people. They are out there. You don’t have to work for bozos, and if you currently do, there is yet hope.

A person who does not live true to himself feels sick and weak inside. My identity as “excellent software tester” demands that I take my craft seriously. I hope you will take this craft seriously, too.

P.S. What if my sense of identity doesn’t require me to be good at my job?

Then, technically, none of this applies to you. Your ethical code can include doing bad work. But… why are you reading my blog? How did you get in? Guards! Seize him!

 

Benjamin Mitchell and the Trap of False Hypocrisy

One of the puzzles of intellectual life is how to criticize something you admire without sounding like you don’t admire it. Benjamin Mitchell has given an insightful talk about social dynamics in Agile projects. You should see it. I enjoyed it, but I also felt pricked by several missed opportunities where he could have done an even deeper analysis. This post is about one example of that.

Benjamin offers an example of feedback he got about feedback he gave to a member of his team:

“Your feedback to the team member was poor because:
it did not focus on any positive actions, and
it didn’t use any examples”

Benjamin immediately noticed that this statement appears to violate itself. Obviously, it doesn’t focus on positive actions and it doesn’t use any examples. To Benjamin this demonstrates hypocrisy and a sort of incompetence and he got his reviewer (who uttered the statement) to agree with him about that. “It’s incompetent in the sense that it has a theory of effectiveness that it violates,” Benjamin says. From his tone, he clearly doesn’t see this as the product of anything sinister, but more as an indicator of how hard it is to deeply walk our talk. Let’s try harder not to be hypocrites, I think he’s saying.

Except this is not an example of hypocrisy.

In this case, the mistake lies with Benjamin, and then with the reviewer for not explaining and defending himself when challenged.

It’s worth dwelling on this because methodologists, especially serious professional ones like Benjamin and me, are partly in the business of listening to people who have trouble saying what they mean (a population that includes all of humanity), then helping them say it better. He and I need to be very very good at what social scientists call “verbal protocol analysis.” So, let’s learn from this incident.

In order to demonstrate my point, I’d like to see if you agree to two principles:

  1. Context Principle: Everything that we ever do, we do in some particular situation, and that context has a large impact on what, how, and why we do things. For instance, I’m writing this in the situation of a quiet afternoon on Orcas Island, purely by choice, and not because I’m paid or forced to write it by a shadowy client with a sinister agenda.
  2. Enoughness Principle: Anything we do that is good or bad could have been even better, or even worse. Although it makes sense to try to do good work, that comes at a cost, and therefore in practice we stop at whatever we consider to be “good enough” and not necessarily the best we can do.

Assuming you accept those principles, see what happens when I slightly reword the offending comment:

In that situation, your feedback to the team member was poor compared to what you could easily have achieved because:
it did not focus on any positive actions, and
it didn’t use any examples”

Having added the words, what happens if Benjamin tells me that this statement doesn’t focus on positive actions and doesn’t cite an example? I reply like this:

“That’s a reasonable observation, but I think it’s out of place here. My advice pertains to giving feedback to people who feel frightened or threatened or may not have the requisite skills to comprehend the feedback or in a situation where I am not seen as a credible reviewer. And my advice pertains to situations where you want to invest in giving vivid, powerful advice– advice that teaches. However, in this case, I felt it was good enough (not perfect but good at a reasonable investment of my time) to ignore the positive (because, Benjamin, you already know you’re good, and you know that I know that you are good– so you don’t need me to give you a swig of brandy before telling you the “bad news”) and I thought that investing in careful phrasing of a vivid example might actually sound patronizing to you, because you already know what I’m talking about, man.”

In other words, with the added words in bold face, it becomes a little clearer that the situation of him advising his client, and us advising him, are different in important ways.

Imagine that Benjamin spots a misspelled word in my post. Does he need to give me an example of how to spell it? Does he need to speak about the potential benefits of good spelling? Does he need to praise my use of commas before broaching the subject of spelling? No. He just needs to point and say “that’s spelled wrong.” He can do that without being a hypocrite, don’t you think?

(Of course, if the situations are not different and the quality of the comment made to Benjamin is clearly not good enough, then it is fair to raise the issue that the feedback does not meet its own implied standard.)

Finally: I added those bolded words, but if I’m in a community that assumes them, I don’t need to add them. They are there whether I say them or not. We don’t need to make explicit that which is already a part of our culture. Perhaps the person who offered this feedback to Benjamin was assuming that he understood that advice is situational, and that a summary form of feedback is better in this case than a lengthy ritual of finding something to praise about Benjamin and then citing at least three examples.

…unless Benjamin is a frightened student… which he isn’t. Look at him in that video. He exudes self-confidence. That man is a responsible adult. He can take a punch.

Who’s the Real Monster?

“Best practice” thinking itself causes these misunderstandings. Many people seek to memorize protocols such as “how to give feedback… always do this… step 1: always say something nice step 2: always focus on solutions not problems… etc.” instead of understanding the underlying dynamics of communication and relationships. Then when they slip and accidentally behave in an insightful and effective way instead of following their silly scripts, their friends accuse them of being hypocrites.

When the explicit parts of our procedures are at war with the tacit parts, we chronically fall into such traps.

There is a silver lining here: it’s good to be a hypocrite if you are preaching the wrong things. Watch yourself. The next time you fail in your discipline to do X, seriously consider if your discipline is actually wrong, and your “failure” is actually success of some kind.

This is why when I talk about procedures, I speak of heuristics (which are fallible) and skills (which are plastic) and context (which varies). There are no best practices.

I’m going to wrap this up with some positive feedback, because he doesn’t know me very well, yet. Benjamin, I appreciate how, in your work, you question what you are told and reflect on your own thought processes in a spirit of both humility and confidence. YOU don’t seem infected by “best practice” folklore. Thank you for that.

 

 

How I Invented Sympathetic Testing

I did not invent sympathetic testing. Anyone who says I claim to have invented it will have only read the title of this post, but nothing further. Now you know.

I may have been the first in my circle to recognize one specific benefit of sympathetic testing. But if so, that is a minor technical point. Because I know that my recognition came directly from a point of view championed by Cem Kaner. From where I stand, it came largely from him.

Okay, James, this is a bit confusing. Why are you talking about this, today?

Ajay Balamurugadas came to me today asking if I came up with the idea of “sympathetic testing.” This is an idea he’s found in Mike Kelly’s work, which derives partly from my work (on tours). It’s also in the section of Lessons Learned in Software Testing that I wrote (and which was edited by Cem Kaner and Bret Pettichord). He had already asked Cem Kaner, who said he got it from my brother [correction: Ajay tells me he saw Cem say that on a video]. But if so, I know exactly where Jon got it: from me, during a project we worked on around 1997. That’s where he was working for me on a court case where I had to do an analysis of “good enough” quality, using a method strongly influenced by Cem’s thinking about cognitive biases.

I’m talking about this today because the provenance of ideas can be confusing when people work closely together. Who cares who invented it, or named it? You might be surprised. Credit is important to people who live by their reputations in a world of ideas. Reputation is money, to put it bluntly. To achieve high income and control over your work as a tester, one way– and the only way I know, actually– is to build your public reputation. Moreover, a lot of our motivation is the respect of our peers. Therefore, this matters.

This isn’t really about “sympathetic testing”, then. Still, what IS sympathetic testing?

Sympathetic testing, as I think of it, is sometimes slightly confused with the closely related ideas of “happy path” or “positive testing.” Sympathetic testing means testing while affirming (rather than challenging) each assumption, resource, or service. But if that’s all it meant, then it would be the same as positive testing. For me, sympathetic testing means more. It means asking “what is wonderful about this product?” rather than “what is broken about this product?” I can do positive testing all day and still be focused on bugs. With sympathetic testing, I may find bugs, but that is not my focus or purpose– my main purpose is learning; building a rich model in my mind.

The insight I had, when working with my brother (way back before he was a recognized test expert in his own right) was that sympathetic testing made me better at unsympathetic testing. Searching happily prepares me to search aggressively.

Okay, then what’s really troubling you?

What bugs me is that Cem deserves more credit for sympathetic testing. But if he claims it himself, he probably thinks he will anger me, and if he gives me full credit, it would anger him: because Cem and I aren’t on speaking terms, at the moment. (I hope that’s temporary.)

I’m hereby offering him credit. Cem and I collaborated for roughly 16 years. We had probably hundreds of deep conversations about testing. Among those conversations, he lectured me on mental models and biases. We once had a fairly bitter argument about what a model is. He won that argument, thankfully, and I have been comfortable with the outcome ever since. He introduced me to Cognitive Science and pushed me deeper into Epistemology. It was that sensibility that led to me discovering a lot of things: among them the power of sympathetic testing. I’m certain that the first person I ran to with my discovery was Cem, but not only that, this is exactly the sort of thing that he was famous for encouraging: to look at things in a sympathetic way. I can’t remember the conversation itself or what he said. I bet he said something like “That’s just what I’ve been trying to tell you!”

Therefore, “sympathetic testing” is part of our joint work. Between about 1997 and 2007, it’s a good bet that I didn’t have a significant thought about testing which wasn’t also sifted and tested by Cem. We didn’t always agree, but we were extremely close. It’s for that reason that he put my name on his BBST class, even though I didn’t do any direct work on it. He was recognizing my influence on him. I’m doing that now.

I’m my own man. I have reinvented testing for myself (just as I recommend that others do). And yet a few people have deeply influenced me in that process. The three people who have had the most influence on my are my father, Jerry Weinberg, and Cem Kaner. I sometimes joke that I can sell out and offer baloney certifications to gullible testers, just like the ISTQB does– but only after those three men have died and I’m no longer seeking their respect.

[Postscript: Why did I title this piece “How I Invented Sympathetic Testing”? Because it was the best way I could think of to attract the attention of people who might be upset that I would make such a claim. Then maybe they would read this post, and feel a lot better.]

Willful Ignorance on Parade

Michael Bolton is accused of hand-waving in this thread on LinkedIn. (See the comment by Peter).

Michael and I talk a lot about cognition and exploration. We speak in tropes that come from philosophy and various branches of science. Once in a while, some fellow who understands little of what we say assumes that we just made it all up to impress the ladies.

It brings to mind the Large Hadron Collider. Here’s an excerpt from one of their bulletins:

“The very smooth and fast transition to operation with ions was made possible by very good beam instrumentation performance with a relatively low number of charges per bunch, and magnetic behaviour very similar to operation with protons, as expected. These two factors combined allowed the setting-up operations to be completed very quickly, and stable beam operation, with 2 bunches per beam, was achieved in just a few days.”

Gee, if you know nothing about physics, or the LHC project, this might sound very much like hand-waving, too! In this case, however, if it sounds sketchy, it’s probably more about the receiver than the source. After all, there really is a $10 billion device sitting in the ground with 4000 physicists poring over the data it generates.

As ambitious professionals, we need to be able to speak about complex subjects without constantly going back to kindergarten to bring along the people who refuse to study their own craft.

I don’t mind if an earnest seeker, who happens to be ignorant, asks what seems to be a silly question. I will help everyone who wants to learn. And I don’t mind the assertive dissenter who has done the homework and yet has a different style and judgment from mine.

I’m talking about something different: the willfully ignorant blowhard.

Please don’t be like that.

A Tester’s Commitments

This is the latest version of the commitments I make when I work with a programmer.

Dear Programmer,

 

My job is to help you look good. My job is to support you as you create quality; to ease that burden instead of adding to it. In that spirit, I make the following commitments to you.

 

Sincerely,

 

Tester

  1. I provide a service. You are an important client of that service. I am not satisfied unless you are satisfied.
  2. I am not the gatekeeper of quality. I don’t “own” quality. Shipping a good product is a goal shared by all of us.
  3. I will test your code as soon as I can after you deliver it to me. I know that you need my test results quickly (especially for fixes and new features).
  4. I will strive to test in a way that allows you to be fully productive. I will not be a bottleneck.
  5. I’ll make every reasonable effort to test, even if I have only partial information about the product.
  6. I will learn the product quickly, and make use of that knowledge to test more cleverly.
  7. I will test important things first, and try to find important problems. (I will also report things you might consider unimportant, just in case they turn out to be important after all, but I will spend less time on those.)
  8. I will strive to test in the interests of everyone whose opinions matter, including you, so that you can make better decisions about the product.
  9. I will write clear, concise, thoughtful, and respectful problem reports. (I may make suggestions about design, but I will never presume to be the designer.)
  10. I will let you know how I’m testing, and invite your comments. And I will confer with you about little things you can do to make the product much easier to test.
  11. I invite your special requests, such as if you need me to spot check something for you, help you document something, or run a special kind of test.
  12. I will not carelessly waste your time. Or if I do, I will learn from that mistake.

(This is cool! Yong Goo Yeo has created a Prezi of this.)

An Important Comment

This comment came in from Stuart Taylor. I think it’s important enough that I should add this to the post:

Hi James,

I’m not sure if this is meant to be a little tongue in cheek, but that’s how I read it. That said, I fell at point 1. “I provide a service” really?

Yes, really. This is not a joke. And point #1 is the most important point.

This implies there may be alternative service providers, and if my customer relationship management isn’t up to snuff, I could lose this “client”. Already I feel subservient, I’m an option.

Of course there are alternative service providers. (There are alternative programmers as well, but I’m not concerned with that, here. I’m making commitments that have to do with me and my role, here.) And yes, of COURSE you may lose your client. Actually that’s a lot of what “Agile” has done: fired the testers. In their place, we often find quasi-tester/quasi-programmers who are more concerned with fitting in and poking around tools than doing testing.

Testing is not programming. Programmers are not testers. If you mix that, you do a sorry job of both. In other words: I am a programmer and a tester, but not a good programmer at the same time that I’m a good tester.

Please meditate on the difference between service and subservience. I am a servant and I am proud of that. I am support crew. I spent my time as a production programmer and I’m glad I don’t do that any more. I don’t like that sort of pressure. I like to serve people who will take that pressure on my behalf.

This doesn’t make me a doormat. Nobody wipes their feet on me– I clean their feet. There’s a world of difference. Good mothers know this difference better than anyone.

Personally, when I work on a software  project, I want to feel like I’m part of the team. I need the developer, and the developer wants me. We work together to bake quality in from the start, and not try and sprinkle it on at the end as part of a “service” that I offer.

What strange ideas you have about service. You think people on the same team don’t serve each other? You think services are something “sprinkled” on the end? Please re-think this.

I want to be on the same team, too. The difference, maybe, is that I require my colleagues to choose me. I don’t rap on the door and say “let me in I belong here! I like to be on the team!” (You can’t just “join” a team. The easier it is to join a team, the less likely that it is actually a team, I think. Excellent teams must gel together over time. I want to speed that process– hence my commitments) Instead, I am invited. I insist that it be an invitation. And how I get invited quickly is by immediately dissolving any power struggle that the programmers may worry about, then earning their respect through the fantastic quality of my work.

You can, of course, demand respect. Then you will fail. Then five years later, you will realize true respect cannot be demanded. (That’s what I did. Ask anyone about what a terror I was when I worked at Apple Computer. If you are less combative or more intelligent than I am, you may make this transition in less time.)

I may have actually agreed with your points before I was exposed to Continuous Integration, because that’s how my relationship used to be; hence me interpreting this as a light hearted piece. However I know that this kind of relationship still exists today (its here at this place). When I began working with continuous integration, the traditional role of the tester, with a discrete testing phase became blurred, as the test effort is pushed back up stream towards the source. If the tester and developer share a conversation about “how can we be sure we build the right thing, and how can we ensure we built it right” before the code gets cut, and the resulting tests from that are used to help write the code (TDD, BDD, SBE), then both of us can have a nice warm fuzzy feeling about the code being of good quality. The automation removes the repetition (and reduces the feedback) to maintain that assertion.

First, your experience has nothing to do with “continuous integration.” Continuous integration is a concept. Your experience is actually related to people. Specific people are collaborating with you in specific satisfying ways, probably because it is their pleasure to do so. I have had wonderful and satisfying professional relationships on a number of projects– mostly at Borland, where we were not doing continuous integration, and where testers and programmers cooperated freely, despite being organized into separate administrative entities.

(Besides, I have to say… Testing cannot be automated. Automation doesn’t remove repetition, it changes it. Continuous integration, as I see it, is another way of forcing your customers do the testing, while placing naive trust in the magic of bewildering machinery. I have yet to hear about continuous integration from anyone who seemed to me to be a student of software testing. Perhaps that will happen someday.)

In any case, nothing I’m saying here prevents or discourages collaboration. What it addresses are some of the unhealthy things that discourage collaboration between a tester and a programmer. I’m not sure if in your role you are doing testing. If you want to be a programmer or a designer or manager or cheerleader or hanger-on, then do that. However, testing is inherently about critical thinking, and therefore it always carries a risk (in the same way that being a dentist does) that we may touch a nerve with our criticism. This is the major risk I’m trying to mitigate.

[Note: Michael Bolton pointed out to me the “warm fuzzy” line. I can’t believe I let that go by when I wrote this reply, earlier. Dude! Stuart! If you are in the warm fuzzy business, you are NOT in the testing business. My goal is not anything to do with warm fuzzy thinking. My goal is to patrol out in the cold and shine my bright bright torches into the darkness. On some level, I am never satisfied as a tester. I’m never sure of anything. That’s what it MEANS to be a professional tester, rather than an amateur or a marketer. Other people think positively. Other people believe. The warmth and fuzziness of belief is not for us, man. Do not pollute your testing mentality with that.]

My confusion over this being a tongue in cheek post is further compounded by point 2. Too many testers do believe that they own quality. They become the quality gate keeper, and i think they enjoy it. The question “can this go live” is answered by the tester. The tester may be unwilling to relinquish that control, because they perceive that as a loss of power.

Just as excellent testers must understand they provide a service, they must also understand that they are not “quality” people. They simply hold the light so that other people can work. Testers do not create quality. If you create quality, you are not being a tester. You are being an amateur programmer/designer/manager. That’s okay, but my commitment list is meant for people who see themselves as testers.

I guess what i’m fumbling about with here, is that some testers will read this, and sign up to it. Maybe even print it out and put it up on the wall.

I hope they do. This list has served me well.

Either way, you have written another thought provoking piece James, but i cant help wondering about the back story. What prompted you to write it?

Stuart – sat in the coffee shop, but with laptop and wifi, not a notepad 😉

This piece was originally prompted when I took over a testing and build team for SmartPatents, in 1997. I was told that I would decide when the software was good enough. I called a meeting of the programmers and testers and distributed the first version of this document in order to clarify to the programmers that I was not some obstacle or buzzing horsefly, but rather a partner with them in a SERVICE role.

That was also the first time I gave up my office and opted to sit in a low cubicle next to the entrance that no one else wanted– the receptionist’s cubicle. It was perfect for my role. Programmers streamed by all day and couldn’t avoid me.

I think this piece is also a reaction to the Tester’s Bill of Rights (the one by Gilb, not the one by Bernie Berger, although I have my concerns about that one, too), which is one of the most arrogant pieces of crap I’ve ever seen written on behalf of testers. I will not link to it.

And now that I think of it, I wrote this a year after I hired a personal assistant, and two months later had her promoted to be my boss, even though she retained almost the same duties. The experience of working for someone whose actual day-to-day role was to serve me– but who felt powerful and grateful as my manager– helped me understand the nature of service and leadership in a new way.

Finally, a current that runs through my life is my experience as a husband and father. I am very powerful. I can make my family miserable if I so choose. I choose to serve them, instead.

Get Thee to the Konditori

I didn’t see you at the coffee shop, today.

Where were you, tester who told me that you need “certification” because, unlike me, you don’t have a public reputation? Where were you, tester who fears the big machine won’t hire you unless you conform, head bowed, to some lowest common standard? How about you, tester who told me that you dare not stop bullshitting your management about test case counts because “he likes those numbers” and, unlike me, you don’t have the credibility (and therefore the power) to say no without being clapped in chains and thrown into the brig?

I was there as usual, with my hardcover Moleskine and my current favorite pens (the Pigma Micron 01 by Sakura Color Products Corporation and the Mitsubishi Pencil Company Unipin Fineline).

I was working on my intellectual property. What were you doing?

Okay I get it: you were busy with your day job. And when the day is over you are tired and you want to go home. Maybe you have children. Children are important. But don’t tell me you are powerless to change things; that you can’t upgrade yourself. You have the power, but it’s not yet a priority for you. If you have reasonable passion and ambition in this field, you will soar above the problems that so many others feel trapped by. You must take time to develop your craft; educate yourself. Punching the time clock is not enough to obtain a better income and better job security.

One place to start is the Konditori (that’s Swedish for coffee shop… sounds cool and kind of mysterious, don’t you think?).

(This photo, above, is one of two Konditori’s I worked at today, in Stockholm.)

I have to keep developing myself and my intellectual property. It’s the engine by which I feed my family. So, today I worked on Rapid Testing Estimation. I want to roll out my new estimation methodology in time for my next test management class, next week.

(Note: I mostly drink green tea, these days… but it’s Stockholm! You gotta drink coffee in Stockholm!)

But who am I kidding? I work in coffee shops, furiously scribbling in my little notebooks, mainly because I love it. It is practical and effective, sure. Mainly, though, it’s peaceful and… hopeful. I am thrilled to feel that, at any moment, I might experience a breakthrough. And you know those breakthroughs, when they happen, can lead to a new hour or two of cutting edge class material, for which people will pay handsome sums. Last year we had the first conference on Session-based Test Management. My brother and I invented that at a coffee shop (well, it was a Denny’s… that’s pretty close).

We can wipe out bad testing. We can dump commercial certification programs into the dustbin. We can create a powerful craft where testers are well paid and respected. And this is how to do it: Meet me at the Konditori. Bring your Moleskine.

(Only if you love the craft, please. If you don’t, geez man, get a different job!)

The Euthyphro Dilemma in New Zealand

I recently had the opportunity to converse about tester certification with Carol Cornelius, who’s on the board of the New Zealand version of the ISTQB. The discussion went well in one respect: she did not physically run away. (Oh, and she concurred with me on the subject of Stuart Reid, which was nice to hear.)

[Considering that they misrepresent my ideas, perhaps she should have run away. To give two examples, my work on SBTM has been plagiarized in their Test Management Syllabus (page 47 of the Advanced Level Syllabus) and of course, they have taken it out of context and gotten it wrong. And my definition of exploratory testing has also been plagiarized and corrupted (page 51). It’s mischaracterized as a cute little informal test technique when in fact it is an approach, universal to good testing, that applies to any technique, and the citation they give (to a chapter I wrote in an out of date book) directly contradicts what they say about it.]

Well, Carol stood her ground, at least physically. But as the debate developed, she made an odd-sounding claim. She said words to the effect that the ISTQB Syllabus is what it is, and is not subject to her criticism. This shocked me, especially since this admission was made in a rather off-handed way– somewhat like commenting that the moon is in the sky.

The reason for my shock was the sudden realization that I was putting a lot of energy into arguing with a person who was, essentially, behaving as a puppet. Oh dear. I was doing the equivalent of yelling at my television instead of engaging the guy ON the television.

If someone defends a principle that he has not originated and is not free to change, reject, or even criticize, then he is not defending it rationally. He cannot. No rational defense can be made under those circumstances. Rationality, in fact, loses its meaning. What he is doing is simply advertising his commitment. And that has no more weight in an argument than have the words of a baseball (or netball) fan who predicts his team will win the big game.

This triggered a niggling memory in me, and afterward it popped fully to mind: the Euthyphro Dilemma.

The dilemma occurs in the Platonic Dialogue of Euthyphro. Socrates is examining Euthyphro about the source of the notion of piety, or good behavior in humans. Euthyphro says that what is loved by the Gods (all of the Gods) is good. Then Socrates asks:

“Just consider this question: Is that which is holy loved by the gods because it is holy, or is it holy because it is loved by the gods?”

Now this isn’t just a question, but also an attack on the whole notion of appeal to authority. That’s why Carol’s offhanded comment triggered the memory.

Let me apply it to the case at hand: Is the ISTQB syllabus good because it’s a powerful, helpful set of ideas that define genuine testing professionalism, or is it good simply because the ISTQB organization says so?

This question is a dilemma for ISTQB supporters, because if they go with the first option, then they must:

  1. Avoid making any claim or behaving in any way that suggests they believe the syllabus just because they are unable or unwilling to study testing for themselves. (Saying that the ISTQB Syllabus was beyond her criticism violates this one.)
  2. Be prepared to explain and justify any aspect of the syllabus on when challenged by a colleague, just as anyone is normally expected to do for a personal opinion in a professional context. (I felt that Carol did not do this. At one point I cited my definition of coverage, and she agreed with it. Then I began to point out implications of my view that contradicted ISTQB dogma about measuring coverage, and I didn’t hear a coherent response after that.)
  3. Be prepared to change their views (thus breaking with the ISTQB) in the face of compelling evidence or reasoning. (I did not witness Carol do this, but I can understand that she wouldn’t necessarily consider my word as a tester to be evidence. What I can’t understand is why she seemed unaware of the work that my community has done and published, over the years, that does comprise compelling argument and evidence. This is not a new or novel debate. The issues have been clearly and repeatedly and publicly established.)
  4. Admit that they don’t need the ISTQB to learn testing, nor to be recognized as a good tester. (I don’t think this applies to Carol, since my understanding is that she doesn’t consider herself to be a tester, strictly speaking.)

And if they go with the second option, then they are choosing to be zombie non-combatants and can be safely ignored in the Great Conversation of testing.

Ideological commitment can be a bitch. That’s why, in the Context-Driven world, we keep that part pretty simple. There are seven principles that define our program. Of course there are many other common patterns and beliefs beyond those seven, but there is tremendous flexibility, because our whole focus is on DENYING a One True Way of testing.

We see testing professionalism as a matter of vigorous personal study and development. We reject any universal syllabus of testing. That’s why it hardly matters whether some particular definition or claim in the syllabus is also one I hold– because the nature of my commitment and the way I understand is completely different from that of an ISTQB board member.

This is why I say that supporting the ISTQB is, in and of itself, inconsistent with the goal of being a testing professional. A professional tester must own his craft.

Shiva is Annoyed with My Questions

A person named Shiva contacted me on Skype back in May. Then he didn’t say anything for several months, until yesterday we had this exchange.

Shiva:

Hi James!

James:

Hi.

Shiva:

I wanted to set up some time with you to chat about an idea and interest you in it…what would be a good time to chat? Would 1130 AM PST Mon work?

James:

What is it about?

Shiva:

Well, I am on the advisory board of a company in China that does phenomenal work and they are very good in testing and SW dev. wanted to see if you wanted to take advantage of their low rates and high quality and make some money? Especially if you are planning to enter that business?

James:

I run a testing company.

Shiva:

I know.

James:

So… Why would I need a testing company? I already have one!

Shiva:

To improve your margins. Scale.

James:

I don’t see how that would be possible. I do a certain kind of testing that requires a high level of skill. I doubt that any other testing company in the world has that skill. Well, there are a couple, but they are expensive.

Shiva:

That is exactly the point I wanted to walk you through as this company has a phenomenal ability to learn, get resources due to their location and yet do high quality work. It is an idea. I am willing to explore it with you if you are interested. If not, I completely understand.

James:

I would have to see examples of the quality work you say that your company does. Is it posted online? Most companies that say they do quality work don’t do quality work at all. They do terrible work. So, I would need to see what you can do.

Also, I would need to know your training practices. I’d want to see them in writing. If you forward your training materials to me, I could review them.

Shiva:

Sure. Before I go forward, it would be good to have a three-way chat with their president/co-founder, myself and you and I am happy to arrange for samples for you to review – work samples, training practices.

James:

I won’t be interested in talking unless you can show me some basic evidence that we have anything to talk about.

Let me ask you just a few questions:

– Does your test lab document all of its testing in detail? Is every test procedure and action documented?

– Do you have complete expected results documented, too?

– Do you maintain statistics on passed/failed tests? Do you graph them?

– Are your testers ISTQB certified?

Your answers to these questions will allow me to quickly assess your capability. Otherwise, I worry you will waste your time.

Shiva:

James, all these are great questions and I have answers that you will like but I need to think about what you said first. In my opinion, business is not just some Q&A, it is also building cross-company relationships and getting to know each other. I need to think about whether we will be a fit that way at all with each other. Please don’t take this personally but I need to give this some careful thought, or else it may not even work even though we are able to deliver capabilities.

James:

You are right. Business is about relationships. I want a relationship with people who can answer basic questions. These are questions that I routinely answer for my clients.

I’m a testing company, too, right? You know that, right? I know what my clients demand of me, and I’m demanding that of you, too. If what you want is an uncritical client who is easily impressed, you came to the wrong guy.

I’m not taking it personally. I’m taking it as an indication that you are a bit over your head.

Shiva:

There is no need to be rude, James.

James:

I’m not being rude. I’m being honest. It’s not my problem if you can’t handle that.

Shiva:

(Skype indicates Shiva has gone offline.)

NOTICE TO TEST LABORATORIES THAT KEEP CALLING ME:

It is not rude for a potential customer to challenge your corporate capability. That’s normal due diligence. Your job is to speak honestly and forthrightly about what you can and can’t do. Don’t dodge questions.

The reason I’m skeptical is that almost no test lab actually knows what it is doing. And there’s no excuse. Test labs ought to know how to test, but mostly I see labs much better at faking testing than doing it.

What Exploratory Practitioners Are Called

An exploratory DOCTOR is known as… a “doctor.”
An exploratory WELDER is known as… a “welder.”
An exploratory PILOT is known as… a “pilot.”
An exploratory WRITER is known as… a “writer.”
An exploratory SCIENTIST is known as… a “scientist.”
An exploratory TRUCK DRIVER is known as… a “truck driver.”

A non-exploratory doctor is known as… “irresponsible.”
A non-exploratory welder is known as… “irresponsible.”
A non-exploratory pilot is known as… “killed in a plane crash.”
A non-exploratory writer known as… “a plagiarist.”
A non-exploratory scientist is known as… “a tobacco company scientist.” (also “Creationist”)
A non-exploratory truck driver is known as… “lost.”

Can you spot the pattern here?

There is no such thing as an “exploratory tester” except inasmuch as a good tester obviously can and will do exploration as a basic part of his work.

Stuart Reid’s Bizarre Plea

Stuart Reid is planning to do a talk on how we should use “evidence” in our debates about what works and doesn’t work in testing.

A funny thing about that is Stuart once spent 30 minutes trying to convince me that the number “35,000” was evidence of how great the ISEB certification is, as in “35,000 happy customers can’t be wrong.” Such a concept of “evidence” wouldn’t not pass muster in a freshman course in logic and rhetoric. How does he know that the 35,000 people are happy? How does he know that they are qualified to judge the quality of the certification? How does he explain the easily checked fact that you can pick out any three ISEB or ISTQB certified testers, ask them if they think the certification has made them better testers or indicates that they are better testers, and at least two of them will do the equivalent of rolling their eyes and smirking? (Don’t believe me? I understand. So TRY IT, as I do on a regular basis in my classes)

You might think Stuart is attempting a bold and classic rhetorical move: attempting to control the terms of the debate. The problem he has is that he will lose the debate even faster if he actually engages on the question of evidence. This is because there is plenty of evidence from other fields and the history of thought itself to justify the positions of the Context-Driven School of testing. We are winning the debates because we are better informed and better educated than the Factory Schoolers, for instance, represented by Reid. For instance, Rikard Edgren (who says he’s not in the Context-Driven School, but looks like a duck to me) wrote about applying Grounded Theory to testing. I wonder if Stuart Reid has ever heard of Grounded Theory. He probably has, because I probably mentioned it at least once in the hours of debate that Stuart and I have had. He didn’t respond or react. My impression was that he wasn’t listening.

There’s something far more important than evidence that we need in our industry: engagement. People need to listen to and respond to the arguments and evidence that are already out there.

Here’s one sort of evidence I put in front of Stuart, in a debate. I claimed that my school of testing represents a different paradigm of thinking about testing than his does. After giving him examples of specific words that we define differently and concepts that we arrange differently, it became clear that the deeper problem is that he thought I was pretending to believe things that I don’t believe, just to be difficult. He actually said that to me!

This is the last resort of the determined idealogue: poke your own eyes out so that you don’t risk seeing contrary evidence. Stuart’s case rests on pretending that no one else is making a case! His demand for evidence is meant to give the impression that the evidence is not already sitting in front of him being ignored.

Cem Kaner, Michael Bolton, and I have been marshaling evidence, pointing out the lack of evidence against our ideas, and demonstrating our methods for many years. Next week it will be exactly 23 years since I first became a full-time software tester, and nearly 17 years since the first time I stood up at a conference and pointed out the absurdity of “traditional” testing methods.

BTW, here some of the kinds of evidence I offer when challenged about my work:

  • The Sciences of the Artificial, by Herbert Simon (this establishes, based on a body of research for which he won the Nobel Prize in 1978, the heuristic nature of engineering)
  • Collaborative Discovery in a Scientific Domain, Takeshi Okada, Herbert Simon, 1997, (this is an experiment that observed the behaviors of scientists attempting to create and perform experiments together in an exploratory way)
  • The Processes of Scientific Discovery: The Strategy of Experimentation, Deepak Kulkarni, Herbert Simon, 1988 (this study analyzes the basic exploratory processes of science)

The first item here is a book, the next two are papers published in the journal Cognitive Science. See, if Stuart wants evidence, he has to look beyond the desert that is Computer Science. He needs to get serious about his scholarship. That will require him to find, in his heart, a passion to learn about testing.