It may be worthwhile noting that I used to leave off all the letters after my name. Cem convinced me that it would probably be helpful when dealing with most prospects and clients.

As you and I have discussed, the certifications I hold are enablers for me. I can argue against a certification more credibly while holding them. I have also been able to influence the CSQE BOK (a little) and rewrite the certification training materials (a lot more) because of the certifications. When I teach the CSQE Preparation class in Silicon Valley, I can teach a context-driven approach along with the party line.

Doug

Can’t agree more.

Looks like terminology even more confusing in performance testing. I’d be very interested in getting any feedback about my struggling with terminology http://www.testingreflections.com/node/view/6855

Thanks!

Alex

Nevertheless is it difficult to have a 100% clear definition:
First you can say that a chair is something where you can sit on, but you can also sit on a table. Then you could state that a chair is something where you sit on and has 4 legs, but a table could also have 4 legs,…

Here is the definition from Wikipedia:
“A chair is a piece of furniture for sitting, consisting of a seat, a back, and sometimes arm rests, commonly for use by one person. Chairs also often have four legs to support the seat raised above the floor. Without back and arm rests it is called a stool.”

Which is also not 100% clear (remark the words sometimes, commonly, often,…), but we all know what a chair is. If you try, you see that it is not easy to explain what “color” or “pain or “warm” or “happy” means.

It is of course true that some people want to use words, labels or definitions because they are fancy and I agree with Jeroen when he wants to keep it simple. A label has to evolve personally. It’s like a child that maybe does not understand that some families don’t have a dog while its family does indeed have one. Or that the family dog is nice, but some other dogs are better left alone. Later on the child will understand. The same goes for labels (and terms in general). We all probably remember something from school that was not clear in the beginning, but when evolving it became clear (vectors, matrices, the connection between magnetism and electricity,…).
What I mean is that, instead of trying to have a waterproof label (and confuse the listener), I try to explain (as simple as possible – adapted to the listener) what the label means. Then I let it rest while this person can think further on this topic. Later on more explanation can/ will follow.

Birth of a word: A story I heard somewhere from some one. There was a person identified by a King in his region and was challenged to do something new and different to prove himself in one night. The person thought for a while. He started scribbling a word on all the walls and possible places (as there was no internet for him to spread the word ) for which he himself does not know the meaning of. Next day morning everyone in the region was wondering what that word means and gave a meaning for it and started using it in their conversation for fun and it lasted so.

The above story may be imaginary and nothing to do with the terms being introduced in testing community or else where. But a general thought I wished to share.

Getting to the soul of the post, testing is gaining the right reorganization as not a robotic task but does involve thinking process. This definition is due to some contributors of the testing society like James, Michael Bolton, Pradeep, Srini and others.

Terms are introduced to refine the task it defines and basically not restricted to the literal meaning of it when it counts to qualify the product. More than a lengthy definition for an action a word helps one get a model in mind than what the word defines. But the extent the model grooms up in a persons mind depends on the knowledge and action one could relate to the word.
Example: Ad-Hoc testing, Monkey testing were the words used. With a refined definition “Exploratory testing” was introduced. The definition and word gives a good feel for the testers when they do it understanding the definition when compared to Ad-Hoc testing irrespective of Exploratory testing is also Ad-Hoc testing. (Inverse not true)

When I say my management “Sir, We will do Exploratory testing.” They say “Ah!! Ad-Hoc testing? We know”

[James’ Reply: A lot of people think they know what ad hoc testing is, but very few do. I use the term ET because fewer people think they know what it means, so they are more willing to listen.]

There are people in world yet to understand the difference with the similarly sounding words. But, it is happy to know that there is a community intending to spread the exact meaning of it without getting frustrated for it not reaching all.

In this testing world the introduction of new terms or redefining the existing terms are always welcome and hence there are always words of similar meaning with a very minute change in the definition being introduced by many in the field trying to give a apt definition.

But, in general (moving away from testing terminology discussion) certain words does hold the meaning the user wants to convey irrespective of the wrong usage.
Example: From the post by Steve, he has used the word Post Script. He wanted to convey please note. The word still does the job of asking us to note. But when researched on the actual meaning and the story of it’s birth, the word post script was introduced when we were posting letter and was scripting on paper with pen or pencil. In case if the important information was forgot to be mentioned in the script flow, introducing it in between the scripted paper was impossible and hence the word post script was used to mention the same towards the end of it. The usage of it has become so usual that the meaning of it changed.

When it comes to business the terms become necessary to convince the client and the customer too. But, not that the identified term speaks for all that is done or all that is missed.

I totally agree, James. However, if we are looking at this from an opportunity cost perspective, then wouldn’t putting a million characters into an edit field be one of our most cost-effective means of exposing system deficiencies or weaknesses?

[James’ Reply: Opportunity cost in economics is the value of the thing you didn’t do. If you test the million character thing while neglecting to test something else that matters, such as the new export feature, or support to double-byte characters, then your client may cry foul.]

I wouldn’t imagine that putting a million characters anywhere would be an example of a real life use of the system unless, maybe, we’re talking about something like a novel database or something which may require that much input. However, in a black box environment I sometimes feel like the most extreme methods will help to give us an understanding of what the system doesn’t do well.

[James’ Reply: I’m not against doing it, in principle. I’m just saying we have to make choices.] 

As an example, about 2 years ago, I worked for an e-Commerce solution provider who specialized in delivering digital media. I started the QA department there and I actually managed to implement a system whereby we were able to make consistently-reliable daily releases, while maintaining 99.999% uptime. So I thought my processes were sound.

There is one time, though, that really sticks out in my mind. There was an incident that actually prevented the delivery of digital products from the system for a number of hours. It was interesting because there had not been a change to the page in months, so I knew we were in trouble. So I had to put on my developer hat and jump into the code. After following the process through each page I made an amazing discovery. Inside the code was some logic built to step over the area where a product is delivered to the client through download when the length of the order id is over 8.

The code was about 2 years old and had never been touched since the inception of the department and was actually commented with “I need to fix this. We need to make a database change to handle this soon” Obviously that never happened and then the code sat there for some time before it blew up as the order ids rolled over into the 9 digit territory.

The issue was that within the database the developers had stored order ids in two spots. The orders table had it listed as int(16) and the downloads table had it at int(8), so once that rolled over, it would create an error. So the band-aid was put into place, then when I had implemented a system that accepted daily, incremental changes, we were not seeing possible errors or testing legacy code that could have its own bugs.

However, I suppose these are the growing pains or hard lessons that we sometimes learn. I think that what I’m trying to illustrate is that, while the greatest amount of value might be placed on a small subset of valid values, I think that finding an error outside of your accepted bounds can sometimes have greater payoffs.

[James’ Reply: It may. The art of testing is largely in the selection of what part of the product space to sample.]

For example, it is our expectation that a valid value will be accepted without issue. If it doesn’t accept that value, then the project halts and the error is fixed. However, when we have issues that are exceptional, such as adding a value outside the “normal” range, these bugs are sometimes “accepted” without knowing the true impact, so I might argue that knowing about these issues provides us, as testers, with more ammunition to use in our testing process.

For instance, if a million-character text line is truncated, then that’s fine. However, if that runs into another database entry (ie in a flat-file case) or if it creates a database error that is displayed to the client (in a web environment) then we’ve exposed something that is truly intriguing and can be further exposed and used to create, ultimately, irreparable damage to the system.

The example I gave previously is completely unsupporting of this, since it represents a value that, really, should be valid, but I would make a case that even though your valid cases are important, the “rabbit holes” are often found outside of those valid ranges.

To expand on Steve’s thoughts, I’ve always had this notion that “There are no intermittent bugs”. I use this sometimes, but not often. Another thought, and this is what I’m really looking to support, is that “Testing is infinite.”

This concept is really, very, deep I think and it provokes a lot of thoughts, like “How can testing be infinite?” I sometimes feel like I’m in the 70s, engulfed in a big cloud of smoke or something when I think of it, but let me support it. We use our standard Equivalnce Class Partitions (one of those wonderful labels), but any given one of those values within those partitions could be bad. Outside of that, numbers can continue on indefinitely either either direction, either plus or minus. This is why we construct boundaries in the first place. Even with that, though, there is the element of time. This accounts for many “intermittent” issues. However, given that all those circumstances are the same, the error would happen again, so it really isn’t intermittent.

I think that this is the difficulty with testing, in general. People want to place labels on things just because they can. There, really, should be no “boundary” testing, a better name for it might be “Because I Can” testing. “Why would you put 1,000,000 characters in that line?” Because I Can. I just think it helps to provoke that creativity opposed to supressing it.

Or am I really in the 70s here?

[James’ Reply: You still have to answer the opportunity cost question. What else are you NOT doing that might be BETTER than putting a million characters in an edit field? Testing is infinite, that means we have to take a sample of it. Now we have interesting choices.]

[James’ Reply: Cool.]

Although I am not new to testing (18 years in the biz), I am new to various terminologies and methodologies, as well as to the community I am now starting to see are out there. I am excited about finding all this, as I have been on my own little island for many years now.

When I first began testing (before this ‘internet’ business came about), I did some research, bought the bible (Testing Computer Software – Kaner, Falk, and Nguyen) and sought out certifications, groups, communities, and anything else I could get my hands on. I was very disappointed at what little I found (except for the bible of course). I chose terms that I use to this day. I used them to convey my testing practices and ideas to those who knew nothing of what I did or was attempting to do. I found in this field that people wanted me to ‘just test it’. I put processes into place, I ‘tested it’, and gave them the information they required so they could make their decisions. I’ve built many ‘QA’ departments for many software companies that had none. I found that some used buzz words and some did not. In the end, the deliverables were what mattered. And even ‘they’ did not know what deliverables they were looking for. They just needed someone there to hold their hand and tell them it was going to be ok. Put a band aid on it…kiss it better. Then, they felt their baby could be set out into the world. They didn’t care how I did it…or what I did. I came up with my own terms – a mix of ones I’d heard over the years – to convey to ‘them’ what I was doing, and how I believed it was making their baby ready for the real world. I found that I have used different terms at different companies. I followed management’s lead – how they thought, how they spoke, and what they expected. I found terms that described what I did and that meant something to each of them. I gave them the info they needed with these terms, but did everything I could to provide them with the best testing possible with limited time and resources. All they knew was that I ‘tested it’.

What I’m trying to say in my own babbling way, is that terms are terms. Somehow, we are not yet mature as an industry, nor do we have standards (at least that we can all agree on). Over the years I have been frustrated by the amount of explaining that needs to be done…explaining to testers, explaining to management, explaining to development, etc. It takes up a lot of time….time that could have been spent being creative, and doing what I ultimately love doing. Arguing and discussing semantics is not what I want to spend my time doing. So, I throw out terms and get a feel for what people mean by them. I grit my teeth when someone says they want to invite Q&A to a meeting and discuss quality. I politely tell them it’s QA and leave it at that. Even though I’m not in the biz of assuring quality, I pick my battle…and just taking the ‘&’ out is enough sometimes. I do this so I can carry on doing what I do…and not get wrapped up in terminology. I agree we need to simplify our terms. I have no idea how to accomplish this, as everyone seems to speak with a different dialect.

I’ll give an example. I am a first generation Canadian. I am fluent in the language my parents came here with I do not know the nuances, but I can get my point across. I do not now, nor have I ever had an accent. My Canadian husband (who only speaks English) has noticed something of which I was never aware. Around my parents (and their friends), my English changes. It becomes broken and simplified. I shift into a slight accent and start to roll my r’s. My hand gestures increase. I speak louder and faster. I assimilate. I’ve done this for nearly 38 years and have not noticed it. Now that I am aware of it, I notice it. A lot. It’s not just with my parents, or even people from their culture. I’ve noticed it work with my testers who are new to the country. Half way through the conversation I’ll notice my r’s or something similar and wonder why I’ve changed my speech? I’ll do it at the mall or my son’s swimming lesson when there’s someone with an accent. It seems to come out to a greater extent, the closer the culture or language is to the one I am familiar with. Nevertheless, it still comes out.

Now, after reading this, and thinking about it – I realize….this is what I’ve been doing in my career. Simplifying my language and changing it depending on who I’m speaking to. I streamline it so I can get my point across as painlessly as possible. The terms I use might not be agreed upon by everyone in the software community, but they work. They communicate the points I want to make so that the customer (be it management, development, etc.) leaves the bakery happy. That is the end result I wish to achieve.

Interesting topic and a great read!

Vesna

[James’ Reply: From your comment I gather that you haven’t done a lot of writing about testing, yet. I hope you do. I’d like to read more of what you have to say.]

I couldn’t find the actual book about the figures, though I think this site might give an example between the differences of verbal and non-verbal communication:
http://www.rsc-ne-scotland.ac.uk/ie/Relationships_with_Customers/Establishing%20and%20maintaining%20relationships%20with%20customers%20version%202-130.htm

Of course there are other models to investigate communication between human-human, I liked this model as I think it can also be used between Human-Machine.
For example: If a tester is behind a PC testing his piece of functions and he seems to work hard and looks happy then that might be caused because his test scripts are understandable and he finds defects or perhaps the functionality he is testing finally works. He is currently communicating with his machine. (using the test scripts as “verbal communication” and the attitude is could be classified as non-verbal.

Because testers are not robots I think my should look further then only the results of the executed or not executed test scripts. The non-verbal communication a tester makes during testing can also be used as trigger. If the tester is not happy, what could be the reason. Are the test scripts not good enough? Are the requirements not clear? Is he able to define test cases using the test techniques the manager asked?

If during the specification phase the tester has question marks in his eyes it could because the documentation is not suitable to use the asked test technique, or the tester doesn’t understand the technique completly. In the last scenario he might continue using the technique as he think he should. in this case the functionality is encoded not properly. This would result in an wrong test scripts or not complete test scripts. Using these test scripts to communicate with the system could lead to wrong assumptions. The system respons according the test script expactation, though the technique was not used properly. Claiming based on “used technique” a false feeling of trust might be created. As the tester shows a happy face during testing.

What I’m trying to say is that we should not trust on the word of the tester when he claims he knows his technique. We should also look at his non-verbal communication. If there are signs that there is misunderstanding in this case techniques we should try to use those labels he understands. As I mentioned before: let him play the tune he knows.

Please don’t understand me wrong when I say to make it simpler. I think the labels for test techniques and the rules could be compared with the grammatic rules for testing. If you want to talk the test language well you have to understand that grammatic. Only to communicate with others for whom testing is as a foreign language you are still be able to say what you want to, only using simpler words to obtain your goal.
For example my native language is Dutch. Therefor Im not that good in English. Asume I am hungry and in the bakkery store I could ask for a bread in several ways like:
a. 1 bread please
or b: I would like to buy one brown bread with peanuts onto it, please.

In testing it could be something like:
a. Please test the business rules with various data you know from production
or b: Please test that functionality using Boundary Testing

In both cases you will obtain your goal, I get my bread or functionality tested. Only as tester I identify and accept the risk that the results returned are not on the same level. Still the non verbal communication might be the same. In situation a: Im happy that Im getting my bread, feeling not uncomfortable not finding the words and accepting that every bread is good enough. In situation b Im happy as I get my bread the way I wanted. In both situations once again a happy customer leaving the backery.

I want to thank you again because you triggered me to think further to think beyond some borders.

With regards,
Jeroen

[James’ Reply: I don’t know what it means to say that 90% of communication is non-verbal. I bet that doesn’t mean anything. Even if communication “amount” were quantifiable, which it isn’t (I’m talking about human communication, not machine communication), and even if the 90/10 thing were true, which I doubt, it probably doesn’t apply to communication about an established technical practice.]
Mainly we use labels for test
techniques for communication with other testers. The output of these can be seen as the
communication language we talk to the system. The results of the conversation with the
system is what we might call behaviour. The feedback we get from that behaviour has to be
interpreted by us testers. As you see some noise can exist. Do we translate well before
sending and are we able to re-translate the feedback. As that will give us the judgement of
the quality.

[James’ Reply: Sapient testers are not comparable to robots. As testers we do not merely behave as stimulus/response systems, do we?]

It is similar to the “The Shannon-Weaver Model”

http://www.cultsock.ndirect.co.uk/MUHome/cshtml/introductory/sw.html

Assume if we are the encoder of the requirements into testcases we have to make sure we
identified the decoder well. In this case the decoder could be other testers. To feed the
other testers to communicate with the system we create testscripts. (channel/message) If the
testscript is of a certain level there is less worry about the decoding it to the system.

[James’ Reply: The Shannon/Weaver model is fine, but there are better ways to think about human-to-human communication. Humans make meaning (it isn’t just decoding) and significance (which isn’t in this model at all) out of the messages they receive. Furthermore, the testing done by a tester may have little to do with messages he receives about it. A sapient tester is not a passive transceiver.]

To identify the decoder we have to look at their skills. Is the tester able to understand
the meaning of the message?

Currently I’m on an assignment were we have lesser educated testers on testing, but have full
knowledge of the system and processes. They are asked to test and give advice. Not only
about the quality, also about the question if they tested enough. There was no time to
educate them with test techniques. What I did was helping them to classify their test cases

in the following categories:
1. Basic functionality
2. Business Rules
3. Screen/ Layout/ Forms/ Reports
4. Exceptional/ Combinations

We linked them to each process they know. I reviewed the test cases related to the
functional designs and based on my knowledge of test techniques I could advise them to think
about more test cases. When we were ready I asked them if they were comfortable with this
number and if all these test cases pass do they have enough information to give advice.
And did the organization accept the risk that we cannot tell exactly what we did not test.
What we did test was easy as it are the test cases. So the coverage is harder to identify.
In this example you talk about communication with the system, this can only be “verbal” only
the interpretation of the feedback of the system is send into the organization. That is done
those 10% verbal and the leftover is nonverbal. Using this classification made the tester
quite comfortable and sure about what he did. He was happy and able to tell the organization
about it.

Perhaps my credo would be: “Play the tune you know well!”
With this credo I mean that we should not try to convince people that everything can be done
better. It is better to explain what you did, you did it with all your means. Playing
another tune might give a wrong noise and might even be hard to learn.

You might even think about levels of testers:
1. those who can’t play a tune
2. those who know about 3 tunes
3. those who know all the tunes

Merely I’m in projects with level 1 and 2 testers. Although I’m eager to teach them other
tunes if it results in false sounds it is better to stick to what they know. And communicate
on that level. Are there risks? Of course. Do we still play the same tune? Hard to say, I
learned that for the song “Tonight” there are over 14 artist who named their song with this
name and the tune is each time completly different. Still it should be easy for me to ask
them how their tune sounds like. And communicate in those words and perhaps not in words
from test profession as boundary value analysis sounds very nice, only will they understand?
I was never against using labels for test techniques (as grammer teachers they also trying
to improve their words and logic to make communication better) still does it always help to
communicate?
I suggest investigate the tunes we can play together, check if the melody is the same and if
the words of the song are in the same language and order.
As stated in the beginning, I think I will not succeed with this challenge. Still it might
help playing the same tune to increase the nonverbal communication of our project members.

With regards,
Jeroen

[James’ Reply: Words can refer to experiences, or convey commands. The meaning of words is always open to question. The use of words must adjusted for the context, as you show that you did. But don’t forget that testers reason, too. I would say something like 99% of what sapient testers do is conceived internally, not passed to them from the outside.]