Today I am writing about the Context-Driven school of software testing. This is a community that initially coalesced around Cem Kaner’s Los Altos Workshops on Software Testing (the LAWST meetings), in the late 90’s. I am a founding member of that community.
In the last two years, more people have joined the community of active context-driven thinkers, helping to develop the notion of what “context-driven” means for testing. I feel like we have a critical mass.
Schools of Thought
In this context, “school” relates to this definition from Merriam-Webster’s Dictionary:
4 a: a group of persons who hold a common doctrine or follow the same teacher (as in philosophy, theology, or medicine) the Aristotelian school; also: the doctrine or practice of such a group b: a group of artists under a common influence c: a group of persons of similar opinions or behavior; also: the shared opinions or behavior of such a group other schools of thought
This definition is a good start, but we are referring to a stronger sense of school: a paradigm; an ontology; a way of organizing, experiencing, and thinking about the entire field. In other words, in the way I use the term, it’s not possible to belong to more than one school of testing at a time. If you tried to be in two schools at once, the school I think you would actually belong to would be a new school called Confused.
The value of the school concept is that it allows us to make progress in our work without constantly having the same fruitless arguments with people who are pursuing a fundamentally different view of the field.
I see the testing world in terms of six major schools of testing. These include the “breakthrough� schools of Analytical, Control, and Test-Driven Design, and the “pragmatic� schools of Oblivious, Factory, and Context-Driven. The breakthrough schools generally hope to reduce or eliminate the need for independent testing. The pragmatic schools generally accept the idea of dedicated software testers and deal with the problem of how testers should get the job done.
To the best of my knowledge, only my school has named itself. Bret Pettichord, Cem Kaner, and I, came up with the names for the other schools. More on the first five in another post, right now I want to focus on the world from the Context-Driven school perspective.
The Context Driven Basics
Cem Kaner came up with the term “context-driven”, in 1999. Cem and I drafted the principles of the Context-Driven School in 2001. Context-driven is all about people solving problems.
“Context-driven” means to match your solutions to your problems as your problems change. The principles of my particular flavor of Context-Driven methodology can be found here.
Consider this rant:
Stop memorizing things people say are solutions, then applying them without understanding how or if they actually solve the problems you face.Don’t pretend to solve problems– really solve them. Don’t solve pretend problems– really solve real problems. And you know what? Those problems are going to shift and change, sometimes day by day or minute by minute.
If you think you can do good work by following some guru’s scripted Best Practice, you might as well blindfold yourself, take a dozen sleeping pills, and drive the Daytona 500. But the nice thing about driving blind in the Daytona 500 is that you know when you’ve crashed and burned. The hard thing about software testing is that you can do terrible work for years and not realize it until some uber-bug that you could have and should have found nails some poor user right in his “GUI components.” Even then, you’ll probably just blame the developers for putting the bug there in the first place.
If you want to take responsibility for the quality of your own work, practices won’t save you: what saves you is your skill, plus your ambition to use your skill. I’m not saying you should ignore practices, nor should you hate them. Just don’t worship them.
To learn to test is to learn to solve testing problems in every situation (we call them contexts) that you encounter. If you do that, your process will unfold naturally, your methods will be fluid, your practice balanced and flexible. So ask yourself, do you have the skills you need? Ask yourself, is your testing fitting your context?
(That’s the end of the rant.)
I like ranting, sometimes. In a rant, I can bring the world into sharp focus. I can say bold, definite things. Unfortunately, the rant above, while advocating the context-driven attitude, is not itself an example of that attitude. Do you see why?
Let me explain…
Four Attitudes Toward Context
When I say “context” I mean the totality of a situation that influences the success or failure of an enterprise. If I want my testing to be successful, I need to behave, somehow, in harmony with the aspects of my world that make testing success possible. For instance, I’m not going to introduce the test technique of cause-effect graphing into the context of a team that has no idea what that means or how to do it, and no time or resources to figure it out.
Everybody copes with context. I see four interesting flavors of coping:
- Context-Oblivious
To be context-oblivious is to be unaware of the relationships between the way you work and the many aspects of the context in which you work. A context-oblivious practitioner is aware of doing things, but can’t tell us why it’s done that way. This is typical of novices, of course, but also of experienced people who do not like to think about methodology (a methodology being a system of methods; a method being some way of doing something).
Completely context-oblivious testers simply don’t read blogs like this. What would be the point?
- Context-Specific
To be context-specific is to be adapted to a context and to know that you are adapted. You know what problems you face, and you solve them. But those problems don’t change, and therefore you don’t need to adjust your solutions.
Most of us Americans would say we know how to drive a car. When we say that we are thinking of a particular context: driving on ordinary public roads in ordinary weather in ordinary cars. Any ordinary car driver in America would know that our driving skill in that context probably does not carry over to driving a race car in the Daytona 500 (even without a blindfold). Still less does it carry over to “driving” an airplane, even though there are many elements in common between airplanes and cars.
I don’t need to learn how to drive a race car, because I’m not in that context, and I don’t even need to learn how to drive in unfamiliar situations, because I simply don’t drive in unfamiliar situations.
It’s easy to believe that being context-specific is good enough, but beware: What if you suddenly lose half your team, or part of your testing get outsourced, or features of your product change radically, or you have to adapt to use a new tool. You may not have the option of adapting to one context only.
- Context-Imperial
To be context-imperial is to keep your solutions the same while striving to change the context around you to fit those solutions.
A classic example is when a tester refuses to test without a “complete specification” instead of learning how to test with or without a spec.
Another classic example is the typical testing consultant who asserts we should all use some Best Practice he has in his mind. When it doesn’t work, he says we must not be using “engineering discipline” to follow the golden path to success, or maybe we don’t have “upper management support” to force us to do the right thing. The solution can’t be wrong, so the context must be.
A modern example is the Agilist who insists that test-driven design is the only good way to write software, and the all software must therefore be written in an object-oriented language, and that 100% of all tests must be automated, because otherwise… well… it’s just not Agile! Kent Beck has spoken! Don’t anger the gods of Agile!
It’s the context-imperialists who bug me most. They are like doctors who prescribe medicine to patients they’ve never examined.
- Context-Driven
To be context-driven is to behave as if there is literally no such thing as a best practice. All we can hope for is good practices in context. Context-driven methodologists are very interested in learning about practices. Then they decompose and recompose them to fit the situation at hand.
To summarize the attitudes of practitioners…
- The context-oblivious simply follow convention.
- The context-specific cultivate the skill to succeed in one stable corner of the world.
- The context-imperial insist that the world change to suit their own skill.
- The context-driven cultivate the skill to adapt to a changing world.
These Attitudes Are Not Mutually Exclusive
I said the four types above were attitudes, not schools. (Note that I’m using the same label “context-driven” as both the name of an attitude toward context and the name of my school of testing thought which is centered on that attitude.) That’s because these attitudes, unlike schools, can mix together.
Even the most ardent context-driven methodologist (and I suspect I am that person) cannot be aware of all the aspects of context that matter in every situation. For instance, if a tester is secretly planning to sabotage the project, I don’t necessarily take that into account when planning that project. In that respect, I am context-oblivious.
I am also partly context-specific, because all my thinking about testing is colored and biased by the kinds of projects I have worked on in the past. Since I have done little military work, for instance, my ideas and calculations are probably different than they would be if I had grown up as a tester in that world.
I am also context-imperial, in some ways. If you want me to design a test process for a commercial product with the requirement that every single test be procedurally specified and placed into a test case management system, I would probably tell you that it sounds like a formula for bad testing and I would try to get you to change your mind. I’m not confident I can be successful in that context.
Despite all that, my self-image and my pride as a tester lies mostly with my belief that I can and I should be able to see a wide range of solutions to nearly any problem. That’s what being skilled in this craft of testing means to me.
Look Again at the Rant
See the unqualified assertions in the rant (e.g. “Those problems are going to shift and change…�)? When directed to a person in a context I know nothing of, they are the hallmarks of the context-imperial attitude. The context-driven way is to make speak in conditionals (“If your problems shift and change…�) and qualifiers (“…then you might find it useful to learn how to adapt your solutions to fit.�), or else stick to speaking about my own experience (“I find it rewarding to study the dynamics of testing problems in general, so that I can solve them no matter how they shift and change. This is because, as a consultant, I am asked to give advice in a wide range of situations.�)
If you look at my writing on this blog closely, you will notice that I use conditionals, qualifiers and personal experiences a great deal. Now you know why. It’s the price of belonging to the Context-Driven school.
Context-Driven Challenge
Any decent context-driven thinker can cite at least three scenarios where taking the context-driven attitude would be a bad idea. Can you think of them?
