I really appreciate your post. I’m not in software testing but I am in IT as a junior sysadmin; I don’t have much professional experience myself, so I’m still trying to adjust my eyes to the waking world of IT buzzwords and bullshit that goes around. I’m still holding onto the hope that I can immerse myself once again in the academic world with like-minded sysadmins.

I’m wondering if you’ve read a lot of philosophy in a past life? I’m always wondering if Wittgenstein is rolling violently in his grave as the field of IT is further “advanced” with terms that are referring to absolutely nothing.

Anyway, thanks again for this post, it gives me a bit of hope in this occupation.

[James' Reply: Yeah, I enjoy philosophy. I consider it a survival skill for senior testers.]

Mr. Pletcher, your meaning for the word “testing” /is/ a sterile subset of the actions taken by a skilled person, that is true. Nonetheless, in the time I have worked in the field I have disproven the claim that asked-for “measurements” could be easily automated on many occasions. The ones that were easy to automate were mostly not the interesting ones. Performance testing changes this marginally, I admit. Often my most valuable testing results were not things the stakeholders ever asked for explicitly.

Also, some key problems with the model you propose are, /observation is not measurement/, and neither is meaning. Tell me the measure of “That ATM just ate my card.” Tell me the measure of “The anti-lock braking system just put itself through a hard reset and rebooted while the car was airborne and our prototype vehicle is wrecked.” Then tell me the measure of an automated test case result which is not meaningfully interpreted by some human.

I don’t work much in the field these days, but if I did get back into it as a day job, instead of a tester, I might call myself an objectives-oriented software systems critic. Criticism still hasn’t been [claimed to be] automated just yet.

[James' Reply: I think lots of people are interested. Not so many want to submit themselves to my editing! Thank you for playing.]

I happen to agree that there are no best practices; continuous improvement would cease if there were. Even so, I would prefer that practical thinkers (who know better) continue applying the phrase “best practice” to their products (or at least to search terms) because they simplify my googling for better practices.

[James' Reply: So you would find it difficult to learn about practices without a guiding adjective? That's the first time I've heard that argument. I'll make a new entry in the wacky reasons list. For me googling is not the primary way I learn about testing practices. Mostly I learn by testing, talking to testers, watching testers, reading blogs by testers whom I trust. That's gives me quite a lot of material.]

Need we banter for years over semantics? If so, attack a more valuable target. … like “Testing”

You referred to “Testing” as an intellectual craft. Testing is not a craft, intellectual or otherwise. Testing is a gerund for a measuring activity — an activity which can easily be automated — there is certainly no intellectual craft in a computer blindly executing measurements.

[James' Reply: That's an odd thing to say. Well, I recommend the Oxford English Dictionary for a comprehensive treatment of the word "craft." What I do in testing, over the last 22 years, and what I teach in my classes, is obviously a craft of software testing. I prefer the word craft to art, because it connotes a little more of an engineering bent.

Regardless of whether you think it's a craft, there is no controversy about this except, apparently, with you. If you want to argue that it not a craft, you'll need to make a much more compelling case to give me something serious to think about.]

Since you termed it an intellectual craft, the meaning I presume you apply for “testing” employs a similar semantic ’sloppiness’ which results in a phrase like “best practices.” To do so is not sloppy, it is shorthand. We say ‘testing’ when our meaning is feature analysis, test design, test planning, and a host of related phrases which must occur to craft a test suite of value.

[James' Reply: I see no semantic issue, here. The craft of testing obviously includes a wide variety of activities. Just as all crafts do. What exactly is sloppy about that? When we speak in language we use words to represent things. My use of words in this case is ordinary, non-confusing, and consistent with the dictionary. You could argue that different people include different things in the craft of testing, but that's not a sloppiness issue, simply a difference of context, community, or philosophy.

The sloppiness of "best practices", on the other hand, is insidious and damaging and totally unnecessary. Use that term and you lose credibility with people who take their ideas about excellence seriously, as I do. Why not say "practices I find interesting?" It's easy. Why even argue about this? You don't mean best practices. You gain nothing from saying it. You confuse people by saying it. So cut it out, man.]

The shorthand becomes dangerous only when and if it results in the sort of effects you described. Calling practitioners of your craft ‘testers’ can (and does) cause them to see themselves as workers who execute tests rather than thinkers who design them. Similarly degrading the consideration afforded by their teammates who call themselves ‘developers.’

[James' Reply: I haven't seen that problem. I'm a tester. To me-- and to lots and lots of other people-- that means I test things. This is far more than "running tests" especially since I don't see testing as an artifact-centered process. Perhaps you come from a very different corner of the field than I do.]

Do the ‘testing’ craft a favor, come up with a better shorthand symbol.

[James' Reply: What do you recommend?]

[James' Reply: I think I did and do understand what it means. I think it's just lazy and wishful thinking masquerading as reasoned judgment. But if someone perverts the language to mean what you say it means, I would apply the same logic to unmask it.]

Rather, it means “more likely to be correct in any given context than any other practice”.

[James' Reply: Don't you see that is just as unobtainable and unreasonable a standard as "best"? How could you make such a claim about any practice? How do you know what my contexts are likely to be?

The view of best practice that you propose still is, at its heart, an attempt to be a backseat driver of someone else's process. Worse, than that-- a blindfolded backseat driver.]

It is important to understand that this term only applies when there is not enough information or understanding to choose between practices on their own merits (which happens a lot). Basically, its like saying, “if you have to choose between all of the available practices, and you don’t know which one to choose, this is the best one to choose.”

[James' Reply: If that's what it means, it constitutes reckless behavior. Please don't reduce engineering to blind trigger-pulling.

To be prudent, instead of reckless, you have to be present in the context and sufficiently skilled, or you have to specify enough about the context and the practice to convey the relationship between problem and solution to the practitioner who IS in that context.]

Now I can guess how you are going to respond, based on your post: developers shouldn’t be choosing practices if they don’t have enough information or understanding to choose them on their own merits.

[James' Reply: Not quite. What I would say is that if you don't have information and understanding then don't LIE TO YOURSELF AND YOUR CLIENTS by pretending that you know what's best, or even what's good.]

And in an ideal world, I would agree with you. In an ideal world we wouldn’t need best practices. If you discover that world, let me know.

[James' Reply: Oh stop with the "real world" talk. That's not an argument. I'm no arm chair theorist. Look at my CV on my website, if you want.

Try living without it. You'll find that you can.]

But back here in the real world, developers have to make decisions that are above and beyond their own knowledge and abilities every day. When that happens, they are much better off choosing the “best practice” than any other practice; it may not necessarily be the right practice for their situation, but it is the best choice because it is more likely to be right than any other choice.

[James' Reply: You seem to be speaking of heuristics. And for some reason you seem to feel that a heuristic that has been blessed by convention is an appropriate substitute for skill. I just don't live that way. I don't need to. Neither do you.]

For example, in chess there are a number of best practices that are taught to beginning players. “Develop knights before bishops” is a good example. It is generally good advice because knights can be developed sooner; they can jump over pawns, whereas bishops have to wait for the pawns to move before they can move. Knights are also short range pieces, so it is important that they be close to the action; bishops are long range pieces and don’t need to rush forward as fast. That said, they are certainly a great many situations where developing a bishop before a knight is the right thing to do. The problem is, a beginner (or very often even an intermediate or advanced player) may not be able to recognize when such a situation occurs. In those cases, what should the player do? I would tell the player to rely on the “best practice”; yes, there is a chance it might be wrong, but it is more likely to be right. Would it be better if the player had the understanding of a grandmaster and could recognize for himself in any situation which is the right thing to do? Of course it would. But he doesn’t have that understanding, and he is playing in a tournament this weekend.

[James' Reply: I play a lot of chess. (I'm "satisfice" on chess. com) I think differently about chess than you do. When I teach chess, I also give general principles. But I don't tell people to "follow" them. In fact, we violate them and see what happens. You are never going to win a chess game against a good player (except if he blunders) by relying on best practice thinking. You must understand the moves and the patterns. You must reason.

I would probably accept many of the principles that you would recommend, but my relationship with those principles is not one of "follow in ignorance", but rather one of conscious understanding and application. I work with students to develop that understanding, which happens by playing a lot of games and reflecting upon them.]

I appreciate what you are trying to say, and I certainly agree that the term “best practice” can be erroneously used to “impress the uninitiated.” But that doesn’t mean that the concept has no value.

[James' Reply: I argue that the concept has negative value. It softens our brains. Look around! It's a meme that opposes excellence. Replace it with "I don't know what's best. Let's try this idea and see what happens." You'll find that you get all the value you thought were getting from "best practices" without the puffery. Puffery opposes learning.]

Found the Buccaneer blog and site and have bookmarked for further reading. Nice to meet you and a kind of relief too.

[James' Reply: Are you familiar with Summerhill? Basically, I would suggest an end to lies and coercion, assuming anyone cares what I think about public policy. But they don't. So I don't cover that in the book.

What I cover in the book is what I did to educate myself... and what I did goes against most of the classic advice about how to study. For instance, I had no particular plan and used no particular discipline, yet my approach to learning seems to have worked pretty well.]

If behind every job or project or problem you substitute a child your arguments become everything I feel when I am given some universal rule of the thumbiest. Options are very important but the kid decides the flowchart and to pretend otherwise puts the kid at risk.

Or maybe I don’t get it. I am just really tired of how that term can perk up ears.

[James' Reply: Education is about to become my field, too. My new book is my attempt to counter the "best practice" of going to school.]

This summed it up for me:
In my maturity model of the industry, promoting a maturity model is mere level 2 behavior. By level 3, we outgrow it.

I mean when you reach that point from being the preachy know it all where everything is ‘by the book’ and ‘best practices’ and ‘I wouldn’t recommend’ and ‘I strongly suggest’ to just accepting that nothing is ‘best’ and ‘context/relevance’ is they key a light goes off and you read this article and you’re like ‘yeah’.

And then you read these people arguing the point (level 2 guys) and think to yourself ‘hey I once was like that - imagine that’ and just shake your head and walk away. The shame of it is that you have to go through level 0, 1, and 2 first before you realize (as the kind in Matrix said) that ‘there is no spoon’.

I wouldn't want to apply it to every project, even as stated, for obvious reasons (a project that lasts 60 minutes, for instance, may not require an evaluation of automated testing. A project that has already been standardized on a tried and true methodology may not require it, either.) But my principle objection is that it is, in fact, not a practice, but rather nothing but a reference to an unspecified activity.]

“Excellently evaluate the use of automated testing” is not a practice, it is an aspiration. Perhaps we can differentiate between the two statements by saying that your statement is the “application of the pracice”?

So to sum up, I believe there are “best pracices”, but there is not a ‘best way’ to apply them practices.

[James' Reply: I'm afraid this is a perversion of the ordinary meaning of the word "practice". But even if we accept it, you have offered no reason to say that the "practice" you've suggested is in ANY WAY better than any alternative.]

None of these are practices. None of these are particularly helpful, let alone "best."]

Some of the projects I’ve worked on, it would have been a positive blessing if we had lost the source code …… All of it.

In order to decide which is the “best practice” for a context, you really need to know the skill/practices. In which case, you had “best practice” with the skill/practice so that you know whether it works or not. It is not allways a good idea to try out a new idea on the critical path.

Unfortunately a number of people think that the best practice is the one they know and they do not need to know any other approaches. So then, a you “best practice” this skill if you want to be part of the decision process as to whether we use it or not would be appropriate.

My “best practice” is that people should learn and become proficient in as many appropriate skills as possible rather than become the ultimate master in one or a few skills. It is quite interesting that many early adopter agilists know more about the waterfall process than the waterfall proponents. They had to be to have the arguments. Sadly many second gen agilists do not understand the traditional processes and are operating based on belief rather than knowledge and experience. The problem with this is that it limits to art/craft of software development.

[James' Reply: Have you read the Book of Five Rings? Sounds like you have.]

I haven't argued that imprecise speech is always bad. In at least one other post I've argued that it's necessary. It's the manipulative usage of that particular phrase I find contemptible-- and there's no other motivation to use it I can see except to manipulate the ignorant.]