- my 2 cents.

[James’ Reply: Actually, if you automate a test that really was executed sapiently, first, then whatever the test is after it’s automated is unlikely to be the same as before. This is true for a couple of reasons. One is that NOTHING is exactly the same twice. The more interesting reason, though, is that a sapient action is never the same as a non-sapient one, even when they look the same. If I asked you to check the contents of a window on the screen, and you noticed that the window was minimized, you would instantly consider un-minimizing the window to check it. A program, however, must be explicitly told to look for that problem. “Check the contents of the window” to a skilled human, means something far more than the same thing issued as a command to a program.]

Perhaps now I’ll have another tool to convince managers that automation is only a part of the solution.

Jerrad anderson

[James’ Reply: By definition a non-sapient process cannot duplicate a sapient process. The reason to automate is not that we think we can duplicate sapience, but because we believe we gain something useful by using a tool, and we believe that sufficiently compensates us for what we lose.] 

AUTOMATION got its birth from SAPIENT Process,right ? If i am a no clear-eyed person, how can i automate a test process or any other else.

[James’ reply: Yes, automation is born from a sapient process. How you can automate a process depends on the process, the technology you use, and what you mean by the term “automate”.]

Lets take a look into Load Testing.(i)The way i develop a script is a SAPIENT, (ii)i will make use of my mind/wisdom to automate the script( again SAPIENT) in generating the load,(iii)my script will communicate with the other scripts deployed in different systems,(iv)generate results (v) will analyze the results.

[James’ Reply: It’s not clear to me what you mean by “automate the script.” Are you talking about replacing a process that is done by humans with one that is done solely by a machine? If so, there are going to be differences between what humans do and what your automation does. Those differences may be important.]

I used the term AUTOMATE to describe all the above said SAPIENT tasks,like a combo.In my context, AUTOMATE is a child term to SAPIENT.

Sorry if i drive you to a wrong direction.

[James’ Reply: I hope what you’re trying to say is that you guide the non-sapient aspects of your testing with your personal sapience. That’s what I do, to. Sapient testing may involve quite a lot of tool use.]

[James’ Reply: Computer software IS automation. When we test it we are testing automation. Therefore it’s hardly surprising that software testers would be familiar with and comfortable with the idea of using automation. Many testers in our industry, however, do not use automation to execute tests unattended.]

This would imply that *all* processes seemed to be algorithmic, would not require a human (skilled) being.

James - refer to our discussion on process during STAR EAST - you mentioned that “cooling of the coffee” is a process and you distinguished from “method” - dropping ice cubes. Process is what actually happens in an open systems environment with physical laws in force and human interactions influencing it in big way. Process is description of something that happened …

I am not sure if one can say all processes are algorithmic by nature. Testing surely not the one.

Most of the people when they say “process” - they confuse that to a written documentation (describing “what is *supposed” to happen”). I believe the process describes what actually happens.

>>> [Yan] …playing Chess a sapient process based on your definition? It would have to be no right? Since we already have machines that will never lose another game to another human (they still draw, yes.), the process does not “rely” on skilled humans.

Yan, seem to indicate here is that since there are computers (algorithmically superb) that can defeat a skilled human chess player – Act of playing chess is a non sapient Process. There are instances where a human being defeated the computer in chess game. I disgree with his notion that playing chess is a non sapient Process
http://en.wikipedia.org/wiki/Garry_Kasparov#Chess_against_computers.

>>>>[James] … If you define chess as the problem of working within a certain set of rules to achieve a certain result, then chess was, at one time, a sapient process, and now it is not necessarily a sapient process

So a process becoming sapient is a situational thing, context based – right? A process can change from “sapient” to “non sapient” based on the context? Do you see a possibility of software Testing becoming “non sapient” in near future?

Shrini

[James’ Reply: A sapient process loses its sapience when it loses its reliance on a skilled human (though I should point out that skilled human is redundant to a certain extent– all normal humans have skills that no machine has). When a process no longer relies on sapience, it is a different process. It is literally a stupider process, but maybe that’s okay. Humans can be expensive, after all.]

“If I take the first 10 characters of each line and use that as the primary key in the foo table, does the rest of the data correspnd to (blah from the database)?”

Of course, this won’t catch that wierd flicker on the screen when the answer is correct but the “help” link mysteriously disappears. Still, I want to automate those simple, brainless things, so I can spend my time on more important things.

[James’ Reply: Well, not only does it not capture flickers, but the tests you just sketched out cover very little of the test space, in any sense. They may be useful, but on the other hand, they take some time to write and maintain, so you may not be saving the time you think you are saving. Test automation is another mouth to feed. To me it is not clear as a general rule that automating even tests that are non-sapient provides sufficient value. But sometimes I feel that it definitely does. I am not anti-automation. I am anti-bullshit. I use tools and partially automated testing, but I don’t pretend I am replacing my sapience with a tool. What I’m doing instead is reducing the sapience of the process without replacing it with something of equivalent value, while calculating (and gambling) that the sapience I drop isn’t critical to my test strategy as a whole.] 

But I think automating everything is crazy, especially at the customer-facing level. (I can see automating a _LOT_ of developer-facing tests, but that’s a different thing.)

Anyway, here’s something really interesting:

Remember how FIT was supposed to automate all acceptance tests and we wouldn’t need software testers?

The customer would “just” describe the tests in FIT?

Well, it turns out that people in the test automation world are starting to realize that FIT isn’t really very good at making sure that the software, welll … works. Instead, FIT now has other benefits:

“So the more important purposes of the business-facing tests are (a) to help explain more clearly to the programmer what the product director wants, (b) to give the programmers a tool to structure their work and know when it’s done, (c) to help the product director think through what she wants, and (d) to help people coming upon the project later to understand something about what it’s supposed to do.”

—> This is From Brian Marick’s Examplar blog (http://www.exampler.com/blog/2007/06/01/two-conference-ideas/), and it’s good stuff, but that’s not what I want to focus on. Instead, I’d like to focus on this:

“I’m not convinced that automated business-facing (acceptance) tests are, in the presence of good programmer testing, all that incredibly useful as tests—as, that is, regression tests, tests that will find bugs created when a programmer changes something here that causes a failure over there.”

—> Again, that is from Marick.

I know of multiple training vendors who suggest 100% test automation using FIT - that using FIT or FITness is the way to do it.

What’s the expression I’m looking for? … Perhaps “It looks like that ship has sailed”?

I liked the comments on testing APIs, as automated testing of an API can only proceed once you have a usable, tested API design — which cannot be done without plenty of sapient testing. Automated testing can help once you reach that point, as the automated testing can uncover implementation defects you introduce when you have to change the design of the API.

Conversely, you can automate the (non-sapient) testing of a GUI, which would be handy once you have finished the sapient testing of the GUI. Usability is only sapient-testable, but whether you lose any functionality from a change may (I repeat may) be automatically testable.

To sum up your position as I understand it (my paraphrase) “All good testing is sapient. However, you can use tools so that you concentrate on the high-level part of testing, while the tools handle the low-level, fiddly little details.”

[James’ Reply: Your paraphrase is close to what I’m trying to say, but I think the division is more interesting than high level vs. details. Sapient handling of details helps us generate new ideas. My brother once clerked at a bookstore, and he had to field a lot of questions about where to find particular books. Because he was in the midst of that question stream, he learned that many people come into book stores looking for books that Oprah Winfrey mentioned on her talk show. From this he conceived an idea to establish a special table just for Oprah books. This was before bookstores generally did that sort of thing. I think if the customer questions were taken by an automated system, it would have been far less likely for anyone to have conceived of the Oprah table idea.

Working with details may be important. All I’m saying is that some things we can automate without necessarily transforming them, while other things are transformed by automation, and still others simply aren’t automatable at all. We need to take these distinctions seriously, instead of laughing them off the way most managers and programmers seem to do.]

Could we call that foolish testing? Or absurd testing? Or vacuous testing? Or maybe asinine testing?

Oh, the fun I can have with a Thesaurus. I am now thinking this may be vacuous comment.

I think I like asinine testing.

If I hand a “test” over to a machine to perform an activity on my behalf, perhaps I am not testing; I am only delegating. Much the same way a manager delegates tasks to his/her sub-ordinates. The effect is somewhat the same: a distribution of energy in an attempt to accomplish as much/more in no more time it would take to do it myself. And the task is deemed completed when some authority puts the stamp on it. An oversimplification for sure, and I’m assuming that the workers can think for themselves (hopefully).

Is it silly to suggest that at the point that one re-enters the testing theater to observe the results of all that delegation - THAT is itself the test and not a moment before? It is purposeful and observable, two distinctly human traits. They require the human element. They insist upon “sapience”.

Otherwise, the computer is doing what Yan suggested: It is just running another algorithm, program or service. Without the human there to observe, suggest, imply, define, prioritize, guess, or bang; the computer is like the tree that fell down in the woods. Perhaps automation is nothing more than condensing the time when we can re-enter so that “actual” testing can begin?

[James’ Reply: I have no problem calling the thing that the tool does “testing”. It’s just that the tool can’t fulfill the entire mission of good testing. It can’t do the whole job of good testing, and it can’t do certain critical parts of the job. I think of my tools, from my pencil on up, as being in a partnership with me. A sapient process relies on a person, at some point, by definition. I think there can be completely non-sapient testing, it just would be really bad.]