But couldn’t the school of testing that you follow depend on the software you are testing.

For example, If I am testing a bomb, might I want to follow an Analytic School’s approach. Making sure I have precise and detailed specs and the bomb works accordingly? Or would you say a context approach might be yes you need to test the specs to the last detail, but hey, maybe you should test what happens if you touch the red button not listed in the specs?

If a Quality School asks the question “Are we following a good practice?”. Is it a bad thing to ask evaluate the defects that were missed and why? And in the end we can determine if we followed a good testing procedure? But if we also learn from our mistakes, would that be more of the context school.

Maybe I have too much of a simple mind to get all this philosophy. Maybe I should just say I see software therefore I test…

[James’ Reply: I think you are confusing practices with schools. Any school could use any practice.]

Upon deploying code to an environment that inherently differs from another, we are shocked with the lack of conformity to our specs. We came to realize that a number of random accidents in the software development, quality assurance, deployment, and management of systems meant that there were several combinations of systems that resulted in poor achievement of the original objectives.

It reminds me of Edward Albee’s observations that a play might have to be author-proof, actor-proof, and audience-proof in order to succeed.

[James’ Reply: Good point, and well said.]

Someone already familiar with a product will be able to tell you whether the documentation or Help is correct, incomplete etc. However only a new user to the product (and thus oblivious - assuming they haven’t received any prior training) will be able to tell you whether the documentation or Help does just that and actually helps them.

Depending upon the experience your new user has of your type of product (is this context specific - they might know how to use your competitor’s product?) or whether they are totally new to the product type, the way in which they use the Help will be different.

I believe this is true with usability testing. If I am testing a website that has a diverse audience from many different backgrounds, I will want testers that know the least about the Web or using a computer. In this case, knowledge of the context would be a detrement to a tester because it is difficult to pretend not to have knowledge once you have it.

Another example would be beginner tester’s luck. I’ve seen testers with a fresh perspective on an application find defects that the hardened, knowledgable, seasoned testers didn’t find in 5 years of testing it. In these cases,puting a new tester in front of the application with no context whatsoever might be more beneficial than bringing that tester up to speed on the context.

[James’ Reply:  I like these examples. In both of them, notice that the tester is acting as a tool in someone else’s process machinery. In such situations, obliviousness is no sin, and may be a virtue.] 

[James’ Reply: Hi Keith. That’s right. I’m a tester, not the project manager, not the CEO, not the programmer. If someone wants to follow XP, that’s fine. I’ll adapt to that.

To be context-driven is not to be uninterested or to have no opinion about what would be better working conditions. It’s just that I know that I’m not in charge. And it’s not only me. Realize that the project manager, etc. also have some serious problems if they expect to drag an unwilling or unskilled team along with them on an Agile adventure. Agile development, as you know, is a personal discipline.

I’ve seen a description of context-driven testing on the web, written by no one I’ve heard of, which calls the Context-Driven school a kind of agile testing. While there are similarities between them, they are absolutely not the same.]

One other example.
Context-driven attitude would not be a good example in situations where the ability to think is reduced. Two examples come to mind.

A) My 5 year old does not have the ability yet to distinguish context as well as I do. While I train him to recognize context, I still prefer that he would follow our specific instructions in most cases.

[James’ Reply: But you expect– and you need– your son to adjust to context in all situations where you have not specified a program to guide his behavior. You expect, for instance, that he will not follow instructions given him by strangers on the street, unless the stranger is a police officer, etc. Furthermore, your son, like all children, learns from playing with rules.

With regard to behavior that you intend to control, you take responsibility for the quality of that outcome, that’s what makes it reasonable. That is one of the big justifications for context-oblivious behavior.]

B) I have noticed for myself that my ability to think gets reduced in extreme stress situations.
When there is a fire in my office building I will follow the evacuation guide. While many other options might be better to evacuate the building a standard evacuation process would avoid chaos.

[James’ Reply: The purpose of training to deal with emergency situations is to maximize the chance that reflex behavior, which remains available to us under stress, will also be effective problem-solving behavior. It’s essentially context-specific training. However, another aspect of dealing with emergencies is to recognize and deal with important context variables.

Recently, my older brother was in a situation where the emergency checklist for his MD-80 airliner advised him to shut down an engine that instrumentation reported to be on fire. He and the other pilot decided that it would be more dangerous to follow that instruction than to skip it, since a one-engine landing in a heavy airplane at high altitude on a hot day is itself quite dangerous. They made this decision because there were no “secondary indicators” of a fire, the assessment of which is a matter of skill and insight. The airline convened a board of inquiry which questioned the pilots deviation from the approved procedure, but ultimately approved their behavior. That’s why we have pilots in airliners!]

[James’ Reply: I don’t see how automatic transmission is context-oblivious. It seems that you are saying it is context-oblivious because the driver is not involved in the transmission decisions. The problem I have with that is that the driver is also not involved in the transmission at all. If you are doing something and you don’t know why you are doing it that way, that’s context-oblivious. But if you aren’t involved at all in a task that is happening, that’s not context-oblivious, that’s task-oblivious. For instance, in all kinds of modern vehicles, the driver also does not control valve timing.

Can you think of a task that the driver does in a certain way, but might not know why?]

‘Any decent context-driven thinker can cite at least three scenarios where taking the context-driven attitude would be a bad idea. Can you think of them?’

Offhand …

1) Some companies care more about being stable, predictable, and repeatable than they care about being _good_. Some have even made a reputation offering products and services that may not be great, but are the same, every time. Most Brand-Name American Fast Food comes to mind. No matter where you eat it, a Big Mac is the SAME - every time. Trying to examine the situation and determine the best thing to do is not what the manager wants you to do; he wants you to follow the operations book.

[James’ Reply: That argues for context-oblivious. The reason it’s a reasonable argument, to me, is that someone else is taking responsibility for the quality of the results.] 

(#2 is not politically correct, but it is my life experience …)
2) While the American Front Line Combat Soldier is one of the best trained in the world, there are times when the solider needs to obey all orders immediately and to the best of his ability, because there simply is no time to think. Two things that come to mind are when the squad leader points and yells “Run!” (which probably means incoming mortars or artillery) or the invasion of Normandy.

Actually, those situations are rare; the American fighting soldier is trained to think and act quickly, to be situationally aware. That said, there will always be a time when an NCO makes a statement with authority and it needs to be obeyed unquestionably or people may die. (This is where trust comes in)

[James’ Reply: This is a good example. Though there are exceptions (a soldier is not required to follow an unlawful order, for instance), this is also a situation where it may be correct to be context-oblivious. The non-commissioned officers are probably context-specific. The commissioned officers probably need to be context-driven. The great generals of history certainly were.]

3) Sometimes authors and speakers need to use a bit of hyperbole when making a point. For example “‘Any decent context-driven thinker can cite at least three” sounds a lot more emotionally appealing that “It has been my experience that any decent …” or “All of the decent context-driven thinkers who I have met have …” We see the point. (Ok, that one was for grins.)

[James’ Reply: I wouldn’t call that hyperbole. I was not trying to make an generalization about the whole world based on my experience of a piece of it. I was speaking my opinion as to something that defines the attribute of “decent” as it pertains to a context-driven thinker. Since I didn’t identify it as my opinion, I guess you could say it’s a context-imperial statement, i.e. “if you don’t think this way, I suggest that you start thinking this way.” Is it a good thing to do that? I don’t think it’s necessarily bad, but neither is it necessarily good.]

4) If someone else is falling into a trap of one of the other schools, engaging them in a discussion may fail. What you can do that point is to take a logical position from a different context and adopt it into the current problem. The other person may immediately recognize that you are making a mistake, criticize you for it … and have an “aha” moment.

[James’ Reply: That’s an interesting example! Non context-driven behavior as reductio ad absurdum.] 

5) Sometimes I like to do context-oblivious manual labor to relax my brain.

[James’ Reply: Not really an argument, though, is it? No matter how sleepy I feel, if I’m driving my car, I’m not allowed to go to sleep before I park it. Just because you don’t feel like doing a good job doesn’t necessarily release you from the responsibility of doing it.] 

6) There are some things to which we are context-specific, like “I am breathing oxygen”, that, most of the time are not worth investing our time in thinking about. Of course, throw me in a lake and bind my legs, and my thinking will change.

[James’ Reply: Often we are oblivious to that. Recently, two college students died when they crawled into a large helium-filled balloon. They didn’t know that helium, although non-toxic, does not solve the same problem that air does. But if you do SCUBA diving then it breathing becomes context-specific. I know why I am puffing on this regulator. I am trained in it.

Clearly, Matt, you are onto a method, here. All you have to do to imagine ways in which it may be good NOT to be context-driven is to imagine how the OTHER attitudes might sometimes be a good thing.] 

[James Reply: I leave that definition to you.]

Consider an automatic transmission - changing gears automated based on some defined gear range values (best practice, huh). While it does not consider the context it reduce fuel efficiency and power compared to manual. And still most cars sold in the United States since the 1950s have been equipped with an automatic transmission. Is Europe a context ?

[James’ Reply: Are you saying that it increases or decreases fuel efficiency? In any case I’m not sure what that example indicates, because it is easy to argue that automatic transmission is good or bad, depending on context. I prefer to drive exclusively manual transmissions (it’s more fun), but my car is an automatic due to my wife’s fear of having to start the car on a hill. We both can drive automatics, hence that is what we have. Fuel efficiency doesn’t matter at all, to me.]

I also consider myself to have a context-driven attitude. However I like testers who cultivate the skill to succeed in one stable corner of the world and as a manager try to give them tasks to suit their own skill and seldom - to expand their world.

[James’ Reply: What’s better about that kind of tester than the kind who adapts? Are you saying that you would not hire someone like yourself?]

I’ve recently written a testing chapter for methodology guide for our company. As I’ve blogged previously I don’t like to document what and how to be done. Still I accepted this challenge. In this chapter I used a lot words like “By default, it is a common approach to, it is recommended to, typically” to describe what someone could call the best practice.

[James’ Reply: When I write articles, I avoid using those phrases without also talking about alternatives. If it’s the default to do X, why is that the default, and what other choices are acceptable? I don’t say “X is recommended”, I say something like “I recommend X, in situation Y, because Z.” I try to talk about problems, solutions, and the problems those solutions could create.

However, writing articles is different from writing a methodology guide for a company. When I do that, I prefer to record policy and reference information. I don’t try to write down instructions to convey skill and knowledge. That is much better conveyed through person instruction and supervision over time.]

Sometimes I describe few different practices which “depends on software type/test type/project phase/etc.” . And stressed out in the most significant places that “It is up to the Project Manager to decide…” . While doing this – I don’t represent the context-driven school while documenting the best practices, do I? And still there are certain benefits of having the document. If only you avoid two things: 1) obey the document 2) hope to improve process by changing the document

[James’ Reply: I find there’s art and discipline to writing a guide to methodology. It might help us to have a context-driven methodologists guide to giving advice. Maybe I’ll write one.]

bq. I think a general case probably applies - it is a mistake to (try to) explain things in a context-driven way when doing on-the-job training. Just tell the person what to do, and maybe drop some hints on the context. If they have the aptitude to learn, then they will take the initiative and follow up on those hints on their own.

Not to rake you o’er the coals, Steve, but I think it still depends. A couple of quick examples:

If I am running a business (such as a fast food franchise), rather than conducting an apprenticeship (such as once was the case in craft guilds), the balance of time spent explaining context should be different. If I am explaining “don’t cross the street without holding someone’s hand”, I expect how I explain it will differ if my subject person is 2, 22 or 95 years old.

I prefer apprenticeship contexts to follow-the-rules franchise/Harvard MBA ones, especially when the mentor says “just try it my way for {x} period and we can discuss what we both see as an outcome”.

And yes, many apprenticeships are-were run as “just follow the rules” because world-lessons (when actually learned) sink in in a way that word-lessons do not. I think that’s at the heart of your approach; am I right? I’d add that another reason is that people are neither perfect learners nor perfect teachers, and time and attention are limited resources.

Classical 20th-Century business school taught managers to make “humanresources” (I always run the words together to remind me the term is-was Newspeak) as fungible as possible. “There are no irreplaceable people”. True, but but but… not every business context is an assembly line. I’m sure you agree!

I took high school physics from a teacher whose attitude toward my problem-solving was “That’s just wrong. Points off for not being right. Full stop.” It was not through any effort on his part that I took Physics as my major in college. If Feynman had been my instructor, I have the feeling that the tone would have been more like “Hmm, well, I can see where you might think that, but here’s what we actually see, so the effect you’re talking about is probably vanishingly small, and nobody has reliably produced it yet. So we don’t find your approach useful.”

Big difference, same domain (”teaching physics”); different (meta?)context.

[James’ Reply: Good point. That is, actually, one of the contexts in which I discourage context-driven thinking: the context in which you are, as yet, oblivious to context, and you are under the supervision of someone who take responsibility for the value of your work.

Remember the line from The Hunt For Red October?

]