Re: “giv[ing] us a chance to back out”… Aza Raskin has written about the importance of “undo”. But there are classes or aspects of software and software “side effects” that are not supposed to *have* undo–Sarbanes-Oxley audit trails and the like. Persistent store is a tricky thing, Especially when so much software sucks.

[James' Reply: When you claim that things are better with TDD, etc., I assume you also intend to say "all other things being equal." But all other things may not be equal. I routinely encounter Agilists who seem to pursue their notion of Agile testing while disdaining or refusing to study sapient system testing skills.

However, there is some solid value in the practices you cite, if we make all the customary charitable assumptions. I'm not denying that. I just think it doesn't solve the problem, because the problem is out of our hands.]

You call out all the potshots organizations and individuals have taken at getting to higher quality, but you don’t give them the credit for trying to produce higher quality. After all, Porsche adopted some of these pedantic measure, called Lean Manufacturing, and now are able to produce defect free cars at the end of the assembly line. See “Lean Thinking.”

[James' Reply: Nothing can be "defect free" except in the imagination of people with insufficient imagination. But all you have to do is talk about a consistently pleasing experience (a lower standard, perhaps, but its what I've been talking about). Even my Saturn gives me a consistently pleasing experience. My desktop computing environment does not.]

I’ve heard this expressed a number of ways - Entropy, the Second Law of Thermodynamics, “All good things must come to an end.”, “Rome didn’t fall in a day - but it did fall.” - It makes me sad.

My father in law, David Ellinghausen, who was a manufacuturing quality engineer, once told me that “all it takes is one very dumb vice president to mess everything up. That’s why you have quality engineers. We institutionalize the system so that he can’t destroy it all with a poorly placed order.”

oh, he used different words, but I think I got the main point right.

All this makes me very sad. I, however, am a privateer, and I only accept letters of marquee from organizations that are trying to build something great.

In America, we still have the resounding echoes the freedom that our forefathers fought for us in the 1700’s. Building great software may be a losing battle to time … but I’m in the fight anyway.

Rock On, Indeed.

I think your ABS example is bad one. There is no ambiguity about ‘quality’ in this case. The ‘placeholder’ that you mention was a number - the hazard rate (risk of default).

[James' Reply: That number is just the end result of a giant unknowable abstraction. It's that abstraction that I was referring to.

But I suppose you're right, in the sense that the purpose of credit default swaps is to transform something with complexity and meaning (a particular mortgage) into something that has liquidity and seeming simplicity. So, perhaps I'm referring to the quality of the decision to buy such securities, rather than the security itself.]

I think often times the statement that “quality is built into it” is useful in delineating job roles. The Quality Assurance umbrella encompasses more than just testing a product. It is about establishing repeatable processes with the goal of ensuring any product following that process will have consistency and thereby value. Testing is just one tool under the QA umbrella. In a nutshell, testing is a quality control function. The product is compared to some oracle and the results noted. This job and the information it provides is important, but it is just one part of a bigger picture.

So, I agree, the statement that “quality is built into” something is a myth that should not be overindulged. That’s why I limit that statement to help delineate the different roles between someone in QA and someone in testing.

[James' Reply: Remember that a "repeatable process" is yet another ephemeral arrangement, to some degree. In essence, it could be said that no process is repeatable, or that every process is repeatable, depending on where you draw the line.

So, when somebody tells me about their repeatable process, I immediately look for the parts of it that don't repeat or the parts of it that shouldn't repeat. I look for residue and side effect-- what's happening that is not being featured or spoken of.]

A colleague challenged my statement as above and said … if it is a relationship and personal between the user and product … then how to measure quality?

I answered him “considering quality as something built into the product is a MYTH. A myth sometimes provides a primitive model to understand complex things but we should not confuse myth to be the reality. We need to externalize the notion of quality as something that resides in a virtual space between product and its user, its environment (thanks to Neil). If you know and you are sensitive about external manifestation of quality - you will change the methods and tools to measure and interpret the quality”

James, I hope you have better answer “how to measure quality”

Shrini

We gather information about quality. We reason. We make judgments. That's how it works. It's not a scale, but a mind that does the weighing.]

Nice post! In particular, I like the shifting of focus onto the notion of relationships.

If I may, I’d also point out that qualities such as “quality”, the values embodied by practices, and the evolving nature of relationships over time all come down to intent. Amongst other things, without a clean connection to intent those other things devolve into static, abstract notionals devoid of life.

Rock on,
John

[James' Reply: Rock on.]

Wondering if you have read Zen and the Art of Motorcycle Maintenance? Get this thought from there?

[James' Reply: I did read it. At the time it confused me, because I thought it was obvious what quality is. (I was young.) Later, I read Lila, the sequel, and that did the trick.]

One note I would like on Shrinis comment:
People are trying to measure quality in the ways you described in order to see process in their daily work. This is based on the assumption that measured intrinsic quality will lead to extrinsic quality experiences as well. This assumption violates the oracle behind this. It’s just a heuristic. All these measures are placed in order to see when something bad happens. The other way round is not valid all the time - i.e. my measurements show that I have improved, but customer’s perceived quality is different from that.

Another side note: Measurements usually focus on technical aspects of the software. Most users don’t deal with the technical part of your products.

I see many fail to consider that quality is lives OUTSIDE the product - in a virtual world that connects product to the user – it is a relationship”.

Hence, Bug counts, all sorts of metrics of software that attempt to look for quality inside the product (defect density, defect leakage %, testing effectiveness) make a futile attempt.

They horribly trivialize and simplify the notion of quality as intrinsic attribute of the product - giving quality a flavor of absoluteness. Relation ship by definition involves two entities. One is the product and other is user. Hence quality is a very personal … and software metrics attempt to remove “personalization” and generalize quality as a something that is built into the product.

Shrini

The only thing that I’d question is: “The product is the experience that the user receives”. I’m not sure this is the full story. A user is an important stakeholder but is not always the most important stakeholder in some situations. I work on corporate software and in many cases the emphasis has been too much on the user experience at the expense of concerns of cost of ownership over time and intersystem interoperability. That’s to say applications end up as pretty balls of mud before they’re even shipped. I would say that a product exists an interstitial space between its users, its environment (eg enterprise) and time.

[James' Reply: Okay, that sounds good.]