Comments on: Fighting Bad Test Documentation http://www.satisfice.com/blog/archives/19 The Consulting Software Tester Fri, 03 Feb 2012 10:21:43 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Paul Berry http://www.satisfice.com/blog/archives/19/comment-page-1#comment-162764 Paul Berry Thu, 04 Dec 2008 15:28:13 +0000 http://12.165.213.55/blog/?p=19#comment-162764 For me personally, there is a tension between the (perceived) need to document what I'm doing and actually getting on with the testing work itself. With generally short periods of time in which to "do testing" anything that takes you away from interacting with the software under test, especially admin, is assumed to be negative. The problem is do the need to keep track of what you've done in a systematic way without it interfering or steering your progress too much. However this pales into insignificance compared to having to come up with a test plan/strategy based on documentation alone, before you even get to see the product. Depending on the personality trait of the tester this can either be a useful exercise or some form of mental torture. I have yet to experience for myself or witness in others the former. The hard thing to reconcile is that formulation of a test plan/strategy makes sense from a control or management point of view but it does not represent how people think or behave in the real world. Yet if a company has a policy for producing test documentaiton then what tends to happen is that rather poor and basic test plans are written because the process is seen to be just another box-ticking exercise. This is because it may have lot of theoretical value but this is rarely borne out in practice. Sometimes, if a detailed and accurate specification can be worked from (very rare!), the corresponding test plan can be pretty thorough. However I've yet to see anyone actually follow one that closely: it's just a base for jumping off. But if that's the case, why not just use the specification as a springboard and form a test plan as you go along? Ideally testers should inspire each other with their documentation but truth be told if people hardly follow their own plans, you can be pretty sure they almost never follow someone else's mess either. To sum up, I'm not saying don't document; I'm saying don't fret about documenting what you're going to do before you do it. Even if you can come up with something decent, it will almost immediately be undone by the process of actually using the product under test. <em>[James' Reply: Useful thoughts. Thanks.]</em> For me personally, there is a tension between the (perceived) need to document what I’m doing and actually getting on with the testing work itself. With generally short periods of time in which to “do testing” anything that takes you away from interacting with the software under test, especially admin, is assumed to be negative. The problem is do the need to keep track of what you’ve done in a systematic way without it interfering or steering your progress too much.

However this pales into insignificance compared to having to come up with a test plan/strategy based on documentation alone, before you even get to see the product. Depending on the personality trait of the tester this can either be a useful exercise or some form of mental torture. I have yet to experience for myself or witness in others the former.

The hard thing to reconcile is that formulation of a test plan/strategy makes sense from a control or management point of view but it does not represent how people think or behave in the real world. Yet if a company has a policy for producing test documentaiton then what tends to happen is that rather poor and basic test plans are written because the process is seen to be just another box-ticking exercise. This is because it may have lot of theoretical value but this is rarely borne out in practice. Sometimes, if a detailed and accurate specification can be worked from (very rare!), the corresponding test plan can be pretty thorough. However I’ve yet to see anyone actually follow one that closely: it’s just a base for jumping off. But if that’s the case, why not just use the specification as a springboard and form a test plan as you go along? Ideally testers should inspire each other with their documentation but truth be told if people hardly follow their own plans, you can be pretty sure they almost never follow someone else’s mess either.

To sum up, I’m not saying don’t document; I’m saying don’t fret about documenting what you’re going to do before you do it. Even if you can come up with something decent, it will almost immediately be undone by the process of actually using the product under test.

[James' Reply: Useful thoughts. Thanks.]

]]> By: Sajeev Kesavan http://www.satisfice.com/blog/archives/19/comment-page-1#comment-115877 Sajeev Kesavan Mon, 31 Mar 2008 09:51:53 +0000 http://12.165.213.55/blog/?p=19#comment-115877 From my perspective , I see that documentation is very much required. Yes Bad documentation is Bad but good documentation is also Good.It is just not to secure a turn over , but shows necessary traceability , displays more often the coverage of tests, to understand the weightage of tests written for the various modules. Documentation should be a part of the testing and if we find documents is taking a lot of time, then its upto management to plan their test schedule including the time required for the documentation. Exploratory testing should be off the shelf outside the documents and interesting test scenarios that has identifies issues should again be documented. Thats my cents of experience. Agree with James on most of his points ,:"Demonstrate the value of concise test documentation (matrices, outlines). " and " Bad documentation is not good " and yes smart documentation is required. From my perspective , I see that documentation is very much required. Yes Bad documentation is Bad but good documentation is also Good.It is just not to secure a turn over , but shows necessary traceability , displays more often the coverage of tests, to understand the weightage of tests written for the various modules. Documentation should be a part of the testing and if we find documents is taking a lot of time, then its upto management to plan their test schedule including the time required for the documentation.

Exploratory testing should be off the shelf outside the documents and interesting test scenarios that has identifies issues should again be documented. Thats my cents of experience.

Agree with James on most of his points ,:”Demonstrate the value of concise test documentation (matrices, outlines). ” and ” Bad documentation is not good ” and yes smart documentation is required.

]]>