How Michael Bolton and I Collaborate on Articles

(Someone posted a question on Quora asking how Michael and I write articles together. This is the answer I gave, there.)

It begins with time. We take our time. We rarely write on a deadline, except for fun, self-imposed deadlines that we can change if we really want to. For Michael and I, the quality of our writing always dominates over any other consideration.

Next is our commitment to each other. Neither one of us can contemplate releasing an article that the other of us is not proud of and happy with. Each of us gets to “stop ship” at any time, for any reason. We develop a lot of our work through debate, and sometimes the debate gets heated. I have had many colleagues over the years who tired of my need to debate even small issues. Michael understands that. When our debating gets too hot, as it occasionally does, we know how to stop, take a break if necessary, and remember our friendship.

Then comes passion for the subject. We don’t even try to write articles about things we don’t care about. Otherwise, we couldn’t summon the energy for the debate and the study that we put into our work. Michael and I are not journalists. We don’t function like reporters talking about what other people do. You will rarely find us quoting other people in our work. We speak from our own experiences, which gives us a sort of confidence and authority that comes through in our writing.

Our review process also helps a lot. Most of the work we do is reviewed by other colleagues. For our articles, we use more reviewers. The reviewers sometimes give us annoying responses, and they generally aren’t as committed to debating as we are. But we listen to each one and do what we can to answer their concerns without sacrificing our own vision. The responses can be annoying when a reviewer reads something into our article that we didn’t put there; some assumption that may make sense according to someone else’s methodology but not for our way of thinking. But after taking some time to cool off, we usually add more to the article to build a better bridge to the reader. This is especially true when more than one reviewer has a similar concern. Ultimately, of course, pleasing people is not our mission. Our mission is to say something true, useful, important, and compassionate (in that order of priority, at least in my case). Note that “amiable” and “easy to understand” or “popular” are not on that short list of highest priorities.

As far as the mechanisms of collaboration go, it depends on who “owns” it. There are three categories of written work: my blog, Michael’s blog, and jointly authored standalone articles. For the latter, we use Google Docs until we have a good first draft. Sometimes we write simultaneously on the same paragraph; more normally we work on different parts of it. If one of us is working on it alone he might decide to re-architect the whole thing, subject, of course, to the approval of the other.

After the first full draft (our recent automation article went through 28 revisions, according to Google Docs, over 14-weeks, before we reached that point), one of us will put it into Word and format it. At some point one of us will become the “article boss” and manage most of the actual editing to get it done, while the other one reviews each draft and comments. One heuristic of reviewing we frequently use is to turn change-tracking off for the first re-read, if there have been many changes.  That way whichever of us is reviewing is less likely to object to a change based purely on attachment to the previous text, rather than having an actual problem with the new text.

For the blogs, usually we have a conversation, then the guy who’s going to publish it on his blog writes a draft and does all the editing while getting comments from the other guy. The publishing party decides when to “ship” but will not do so over the other party’s objections.

I hope that makes it reasonably clear.

(Thanks to Michael Bolton for his review.)

Question: How to Rapidly Test Maintenance Releases?

A correspondent writes:

“I have a test management problem. We have a maintenance project. It contains about 20 different applications. Three of them are bigger in terms of features and also the specs that are available. I am told that these applications had more than 1-2 testers on each of these applications. But in this maintenance project we are only 6-7 testers who are responsible to do the required testing. There will be a maintenance release every month and what it will deliver is a few bug fixes and a few CRs. What those bugs and CRs would be is not known in advance. Could you please suggest how to go about managing such kind of assignment?”

My first-order (direct) answer to this question runs about 3,000 words. It’s a slightly improved version of what was originally published in the TASSQ Quarterly, 9/06. It’s a bit too long for comfortable reading in blog form, so here it is as a PDF.

An Aside About Context-Driven Methodology

I also want to say that this PDF is an example of context-driven methodology. Responsible context-driven driven advice begins by inquiring about the context of the reader or specifying the relevant aspects of context in which the presented methods are believed to be helpful. The advice is then presented in a heuristic tone (“this might help”) rather than with an imperative tone (“you should do this”). I can use an imperative tone only when I am taking responsibility for the quality of the work; when I am the boss.

Part of the heuristic way of giving advice is to help the reader see reasons, causes, caveats, and alternatives. As a context-driven methodologist, I know my method ideas are tools, not facts. They must be interpreted and applied in specific situations by specific people. For the same reason, I see all methodology as embedded in a dialog. If I tell you “it is helpful to minimize documentation, because documentation is expensive”, I anticipate that you may reply with a question or a challenge, I try to make it easy for you to do that, and I try to be ready and able to answer you. Through the dialog, we all learn and develop better ways of working here and now on this project. In philosophical lingo, context-driven methodologists understand that each practitioner contructs (in a sense, invents) the craft for himself while immersed in a rich world of signs and signals that may guide us.

If you ever see me straying away from the context-driven path of methodology, I hope you will bring that to my attention. It is through the help of my colleagues and clients that I learn to do this better.

Question: How do you stay sharp as a tester?

Shrini writes: How does a good tester keep his testing abilities sharpened all the times. compare it with keep our body fit as we grow old ( walking, jogging and going to Gym, eating healthyfood etc) – what you suggest for keeping “Tester health”? in “fit and sound”? condition?

Testing is analysis and problem solving. Here is what I did, this past week:

  • I solved about 50 problems from the book “Lateral Logic Puzzles” with my son.
  • Paul Jorgensen sent me an exploratory testing challenge, in the form of a spreadsheet with a bug in it. I investigated the bug and wrote a play-by-play description of what I did.
  • I wrote a Perl script to generate some experimental tests.
  • I practiced Sudoku with my Nintendo DS Sudoku game.
  • I analytically solved a conditional probability problem (the taxicab problem) that is often associated with the Representativeness Bias. This was part of working out a testing exercise based on that bias. (Then I tried the new exercise with Michael Bolton.)
  • I read some of a testing book from 1986 that Mike Kelly lent me. I’m trying to characterize the difference between “modern” testing ideas and those from 20 years ago.
  • This morning, I derived the formula for calculating the distance to the horizon based on eye level. It’s been a long time since I did trigonometry, but it was fun rediscovering sines and cosines.
  • I listened to a few hours of lectures from the Teaching Company about Neo-platonism and other philosophical trends of the dark and middle ages.
  • I skimmed several articles, including Knowledge And Software Engineering: A Methodological Framework To Symbiotic Software Process Modeling, and Blooming E-learning: Adapting Bloom’s Taxonomy into the content of e-learning course to promote life long learning through Metacognition, and Third Cybernetic Revolution: Beyond Open to Dialogic System Theories.
    It may not seem like it from the titles, but they have a lot to do with analyzing testing practices and becoming a better tester.
  • I received Pradeep’s Soundararajan’s startlingly incisive answer to the Wine Glass factoring exercise I gave him (“Describe all the dimensions of a wine glass that may be relevant to testing it.”), which helped me see more angles and subtleties to my question. Then I transpected with Michael Bolton as he worked through the same problem.
  • I worked on answers to testing questions submitted by my readers.

As you see, I stay sharp in testing by finding and solving problems, including testing problems; and reading or listening to philosophical ideas that I use to understand testing better; and by trying to help other testers learn, or by watching them learn; and by actually testing.

I’m not in a project, at the moment, for a paying client. If I were, I would be staying sharp by solving problems for my client. I do my best to find excuses to learn new things while working for pay.

When I worked at Apple Computer, I often stole away to the Donut Wheel, across the street, to read about software engineering. When I worked at Borland, I stayed late and worked on test methodology documents and articles. At SmartPatents, I learned Perl and formed my first thoughts about agile test automation.

Some people, and I know you are like this too, Shrini, sharpen themselves no matter what else is going on.

Question: Tester’s Freedom of Thought

Subha asks:
A tester is usually bound by the constraints of specifications when he does functional testing. But what about usability? How much should the tester’s imagination be allowed to flow?

Hello Subha,

Read carefully– this is important:

The specification does not bind you, as a tester. The specification provokes you. In fact, the spec, the product, the things people say on the project– all of it is provocative to the tester. It might be where we start, but not where we end. If the product behaves in some important way (important either positively or negatively), then it is generally the role of the tester to test it, even if there is nothing in the spec about that behavior.

Doing testing well requires a great deal of imagination.

I think a tester is actually bound by six things that come to mind as I write this: the mission, project culture, the paticular constraints of the project, and the skill and knowledge of the tester. All of these except that last one are to some extent negotiable:

Mission: This is the problem that your clients want you to solve for them; the outcome they want you to achieve. If you don’t honor your mission, you will not gain credibility or retain respect. Be sure that you negotiate a mission you are capable of fulfilling; and be sure that the story of your testing features the mission as its primary plot point. Is usability testing a part of your mission?

Project Culture: The other people on your project have expectations about what you will or will not do. These bind you. You can challenge those expectations and suggest alternatives, but you have to be careful about that. If usability is part of your mission, what methods of usability testing are acceptable or expected within your organization?

Project Constraints: On your project, you don’t have all the time and money to do everything you might find useful or interesting. You may need to find inexpensive ways to do the testing that your strategy calls for. You may need to acquire special tools, or use tools that don’t do everything you wish they did. What kind of usability testing is it even possible to do on your project?

Tester Skill & Knowledge: Even if you were granted permission and resources to do everything you want to do, you would still be limited to the things that you know how to do. If you want to do testing well, you need enough command of testing practices and tools to make that possible. One common problem with testers is that they don’t do enough to educate themselves. Do you know how to do usability testing?

I realize that this is not a detailed answer to your question. What I’m trying to do is frame a way for you to think the issue through for yourself.

Ethical Standards: A tester is bound by ethical standards not to, for instance, lie about the results of the tests, or to misrepresent his ability to do the work. Are you suggesting usability testing for selfish reasons, or do you really believe it will help your client?

Legal Standards: A tester is bound by legal standards. In some cases, there are laws, such as Sarbanes-Oxley or HIPAA, which guide how you must test. Is there any legal reason why you must or must not perform usability testing?


Should Developers Test the Product First?

When a programmer builds a product, should he release it to the testers right away? Or should he test it himself to make sure that it is free of obvious bugs?

Many testers would advise the programmer to test the product himself, first. I have a different answer. My answer is: send me the product the moment it exists. I want avoid creating barriers between testing and programming. I worry that anything that may cause the programmers to avoid working with me is toxic to rapid, excellent testing.

Of course, it’s possible to test the product without waiting to send it to the testers. For instance, a good set of automated unit tests as part of the build process would make the whole issue moot. Also, I wouldn’t mind if the programmer tested the product in parallel with me, if he wants to. But I don’t demand either of those things. They are a lot of work.

As a tester I understand that I am providing a service to a customer. One of my customers is the programmer. I try to present a customer service interface that makes the programmers happy I’m on the project.

I didn’t always feel this way. I came to this attitude after experiencing a few projects where I drew sharp lines in sand, made lots of demands, then discovered how difficult it is to do great testing without the enthusiastic cooperation of the people who create the product.

It wasn’t just malicious behavior, though. Some programmers, with the best of intentions, were delaying my test process by trying to test it themselves, and fix every bug, before I even got my first look at it (like those people who hire house cleaners, and then clean their own houses before the professionals arrive).

Sometimes a product is so buggy that I can’t make much progress testing it. Even then, I want to have it. Every look I get at it helps me get better ideas for testing it, later on.

Sometimes the programmer already knows about the bugs that I find. Even then, I want to have it. I just make a deal with the programmers that I will report bugs informally until we reach an agreed upon milestone. Any bugs not fixed by that time get formally reported and tracked.

Sometimes the product is completely inoperable. Even then, I want to have it. Just by looking at its files and structures I might begin to get better ideas for testing it.

My basic heuristic is: if it exists, I want to test it. (The only exception is if I have something more important to do.)

My colleague Doug Hoffman has raised a concern about what management expects from testing. The earlier you get a product, the less likely you can make visible progress testing it– then testing may be blamed for the apparently slow progress. Yes, that is a concern, but that’s a question of managing expectations. Hence, I manage them.

So, send me your huddled masses of code, yearning to be tested. I’ll take it from there.