Comments on: Question: How Many Times Should You Run a Test? http://www.satisfice.com/blog/archives/67 The Consulting Software Tester Mon, 15 Mar 2010 09:24:04 +0000 http://wordpress.org/?v=2.7 hourly 1 By: Danny Faught http://www.satisfice.com/blog/archives/67/comment-page-1#comment-181873 Danny Faught Fri, 17 Apr 2009 15:14:55 +0000 http://www.satisfice.com/blog/archives/67#comment-181873 Do people really ask about how many times to run a test, without giving any other context? Asking about repeating a test on the same build of the software on the same configuration is different than asking about which of the different builds of the software a test should be run on or which configurations to run a test on. <i>[James' Reply: That blog post was motivated by someone who did ask that question.]</i> Do people really ask about how many times to run a test, without giving any other context? Asking about repeating a test on the same build of the software on the same configuration is different than asking about which of the different builds of the software a test should be run on or which configurations to run a test on.

[James' Reply: That blog post was motivated by someone who did ask that question.]

]]> By: Pajton http://www.satisfice.com/blog/archives/67/comment-page-1#comment-62743 Pajton Wed, 05 Sep 2007 08:35:40 +0000 http://www.satisfice.com/blog/archives/67#comment-62743 What about regression testing when we want to check if any old bug is reintroduced? <em>[James' Reply: Okay, what about it?]</em> What about regression testing when we want to check if any old bug is reintroduced?

[James' Reply: Okay, what about it?]

]]> By: insectivorous http://www.satisfice.com/blog/archives/67/comment-page-1#comment-4306 insectivorous Tue, 03 Oct 2006 16:48:27 +0000 http://www.satisfice.com/blog/archives/67#comment-4306 The job is to detect and to fix (in the sense of shepherding) software defects. It's unbelievable how much nonsense you can cut through by just asking yourself, "What's the most useful thing I could do next?" It's not rerunning old or even variant tests that provide no new information. It's not running a new test that won't provide new information. It's not beetle-tracking in the name of "thoroughness", without the use of the tester's intelligence to guide the process effectively. If you're not making progress toward the goal, then just exactly what are you doing? The object is to find bugs. (Well, and to adminster the process of same.) If what you're about to do isn't the most effective thing you can think of towards that end, then you should just about certainly be doing something else. If what you're about to do is solely in satisfaction of some arbitrary metric, then you certainly should be doing something else, (That or you're trying for an ISO-9000 qualification, which tells you what that standard's worth.) And if you're repeating tests because you can't think of anything else to do, it's time for a long break, some meditation on your craft, and maybe a nice raw steak. So look at it this way: if you're doing the same old thing, you won't find anything new to break, so you won't make any coders cry, and what fun is that? The job is to detect and to fix (in the sense of shepherding) software defects. It’s unbelievable how much nonsense you can cut through by just asking yourself, “What’s the most useful thing I could do next?” It’s not rerunning old or even variant tests that provide no new information. It’s not running a new test that won’t provide new information.

It’s not beetle-tracking in the name of “thoroughness”, without the use of the tester’s intelligence to guide the process effectively. If you’re not making progress toward the goal, then just exactly what are you doing?

The object is to find bugs. (Well, and to adminster the process of same.) If what you’re about to do isn’t the most effective thing you can think of towards that end, then you should just about certainly be doing something else.

If what you’re about to do is solely in satisfaction of some arbitrary metric, then you certainly should be doing something else, (That or you’re trying for an ISO-9000 qualification, which tells you what that standard’s worth.)

And if you’re repeating tests because you can’t think of anything else to do, it’s time for a long break, some meditation on your craft, and maybe a nice raw steak.

So look at it this way: if you’re doing the same old thing, you won’t find anything new to break, so you won’t make any coders cry, and what fun is that?

]]>