In your paragraph beginning with “What counts as disclosure to the customer?” in the sentance “For their purpose (help the developers troubleshoot the bug), these reports mgiht have been marvellous.” you have a typo with mgiht vs might and a mis-spelling of marvellous vs marvelous.

I love your work and use it daily. Although there are those who do not listen…

Eric

[[Thanks for your bug report. Fixed ... Cem]] 

It occurs to me that if software companies are obliged to provide lists of known defects to customers, the obvious result of this is that the companies reduce their testing efforts and quietly discourage formal reporting of bugs so that they can claim to not have known about their software’s defects. Companies who test their software well will have intimidatingly huge lists of ‘known defects’ which will hurt their market position, because the everyday consumer won’t realise that a long list of known defects is in fact a sign of careful testing of the product (and is likely to mean that the more serious defects that lurk unannounced and unfound in competitor products have been found and fixed in this one).

So this recommendation has the potential to introduce barriers to good software testing.

[[This is a common misunderstanding of the proposal, usually spread by vendors who don’t like it.

If you don’t test your product, then at the time of release, you indeed won’t know about its bugs. So far, so good, right?

Of course, you also miss serious bugs that you really would have wanted to fix, bugs that will hurt your reputation in the market, kill your sales, drive up your tech support costs–people search for bugs in their code and fix them for a reason.

But even if you dodge finding your bugs, not long after you release your product, people call you, send you letters, write reports in magazines–guess what. You just heard about your bugs. Now you have to disclose them, just as if you had found them yourself.

So you achieve what with this strategy?

• Bad customer relations
• Crappy bug-filled product
• High tech support costs
• Bad magazine reviews
• And you still have to tell people about your bugs.

In most cases, it would be a lot cheaper to find and fix (or document) serious bugs before release.

– Cem Kaner ]]

As you write a piece of code, you may be aware of limitations of that piece of code (e.g. this function will fail if the string is > 50 characters). You won’t necessarily have time to handle for that limitation — in which case it becomes a defect.

If you don’t have time to handle for the limitation, then you don’t have time to document the limitation. Lacking time to handle or document the limitation, you don’t have time to assess the possible impact of the limitation.

(i.e. this function can fail, but which functions depend on this fuction? and ultimately what pages, forms, or business processes would then be impacted? and how valuable are they? ultimately this is what tells us if its serious enough to be a material breach).

So being momentarily aware of a limitation (which becomes a defect) isn’t the same as being aware of the possible implications of that defect.

A project that triages all limitations as they arrise and makes informed decisions about the possible implications of those limitation…. will never ever ship.