Quality is Dead #1: The Hypothesis

Quality is dead in computing. Been dead a while, but like some tech’d up version of Weekend at Bernie’s, software purveyors are dressing up its corpse to make us believe computers can bring us joy and salvation.

You know it’s dead, too, don’t you? You long ago stopped expecting anything to just work on your desktop, right? Same here. But the rot has really set in. I feel as if my computer is crawling with maggots. And now it feels that way even when I buy a fresh new computer.

My impression is that up to about ten years ago most companies were still trying, in good faith, to put out a good product. But now many of them, especially the biggest ones, have completely given up. One sign of this is the outsourcing trend. Offshore companies, almost universally, are unwilling and unable to provide solid evidence of their expertise. But that doesn’t matter, because the managers offering them the work care for nothing but the hourly rate of the testers. The ability of the testers to test means nothing. In fact, bright inquisitive testers seem to be frowned upon as troublemakers.

This is my Quality is Dead hypothesis: a pleasing level of quality for end users has become too hard to achieve while demand for it has simultaneously evaporated and penalties for not achieving it are weak. The entropy caused by mindboggling change and innovation in computing has reached a point where it is extremely expensive to use traditional development and testing methods to create reasonably good products and get a reasonable return on investment. Meanwhile, user expectations of quality have been beaten out of them. When I say quality is dead, I don’t mean that it’s dying, or that it’s under threat. What I mean is that we have collectively– and rationally– ceased to expect that software normally works well, even under normal conditions. Furthermore, there is very little any one user can do about it.

(This explains how it is possible for Microsoft to release Vista with a straight face.)

I know of a major U.S. company, that recently laid off a group of more than a dozen trained, talented, and committed testers, instead outsourcing that work to a company in India that obviously does not know how to test (judging from documents shown to me). The management of this well-known American company never talked to their testers or test managers about this (according to the test manager involved and the director above him, both of whom spoke with me). Top management can’t know what they are giving up or what they are getting. They simply want to spend less on testing. When testing becomes just a symbolic ritual, any method of testing will work, as long as it looks impressive to ignorant people and doesn’t cost too much. (Exception: sometimes charging a lot for a fake service is a way to make it seem impressive.)

Please don’t get me wrong. Saving money is not a bad thing. But there are ways to spend less on testing without eviscerating the quality of our work. There are smart ways to outsource, too. What I’m talking about is that this management team obviously didn’t care. They think they can get away with it. And they can: because quality is dead.

I’m also not saying that quality is dead because people in charge are bad people. Instead what we have are systemic incentives that led us to this sorry state, much as did the incentives that resulted in favorable conditions for cholera and plague to sweep across Europe, in centuries past, or the conditions that resulted in the Great Fire of London. It took great disasters to make them improve things.

Witness today how easily the financial managers of the world are evading their responsibility for bringing down the world economy. It’s a similar deal with computing. Weak laws pertaining to quality, coupled with mass fatalism that computers are always going to be buggy, and mass acceptance of ritualistic development and testing practices make the world an unsafe place for users.

If we use computers, or deal with people who do, we are required to adapt to failure and frustration. Our tools of “productivity” suck away our time and confidence. We huddle in little groups on the technological terrain, subject to the whims and mercies of the technically elite. This is true even for members of the technically elite– because being good in one technology does not mean you have much facility with the 5,000 other technologies out there. Each of us is a helpless user, in some respect.

Want an illustration? Just look at my desktop:

  • Software installation is mysterious and fragile. Can I look at any given product on my system and determine if it is properly installed and configured? No.
  • Old data and old bits of applications choke my system. I no longer know for sure what can be thrown away, or where it is. I seem to have three temp folders on my system. What is in them? Why is it there?
  • My task manager is littered with mysterious processes. Going through, googling each one, and cleaning them up is a whole project in and of itself.
  • I once used the Autoruns tool to police my startup. Under Vista, this has become a nightmare. Looking at the Autoruns output is a little like walking into that famous warehouse in Indiana Jones. Which of the buzillion processes are really needed at startup?
  • Mysterious pauses, flickers, and glitches are numerous and ephemeral. Investigating them saps too much time and energy.
  • I see a dozen or two “Is it okay to run this process?” dialog boxes each day, but I never really know if it’s okay.  How could I know? I click YES and hope for the best.
  • I click “I Agree” to EULAs that I rarely read. What rights am I giving away? I have no idea. I’m not qualified to understand most of what’s in those contracts, except they generally disclaim responsibility for quality.
  • Peripherals with proprietary drivers and formats don’t play well with each other.
  • Upgrading to a new computer is now a task comparable with uprooting and moving  to a new city.
  • I’m sick of becoming a power user of each new software package. I want to use my time in other ways, so I remain in a state of ongoing confusion.
  • I am at the mercy of confused computers and their servant who work for credit agencies, utility companies and the government.
  • I have to accept that my personal data will probably be stolen from one of the many companies I do business with online.
  • Proliferating online activity now results in far flung and sometimes forgotten pockets of data about me, clinging like Spanish Moss on the limbs of the Web.

Continuous, low grade confusion and irritation, occasionally spiking to impotent rage, is the daily experience of the technically savvy knowledge worker. I shudder to think what it must be like for computerphobes.

Let me give you one of many examples of what I’m talking about.

I love my Tivo. I was a Tivo customer for three years. So why am I using the Dish Network and not Tivo? The Dish Network DVR sucks. I hate you Dish Network DVR developers! I HATE YOU! HAVEN’T YOU EVER SEEN A TIVO??? DO YOU NOT CARE ABOUT USABILITY AND RELIABILITY, OR ARE YOU TOTAL INCOMPETENT IDIOTS???

I want to use a Tivo, but I can’t use it with the Dish Network. I have to use their proprietary system. I don’t want to use the Dish Network either, but DirectTV was so difficult to deal with for customer service that I refuse to be their customer any more. The guy who installed my Dish Network DVR told me that its “much better than Tivo.” The next time I see him, I want to take him by the scruff of his neck and rub his nose on the screen of my Dish Network DVR as it fails once again to record what I told it to record. You know nothing of Tivos you satellite installer guy! Do not ever criticize Tivo again!

Of all the technology I have knowingly used in the last ten years, I would say I’m most happy with the iPod, the Tivo, and the Neatworks receipt scanning system. My Blackberry has been pretty good, too. Most other things suck.

Quality is dead. What do we do about that? I have some ideas. More to come…

81 thoughts on “Quality is Dead #1: The Hypothesis

  1. Yes, most software does suck.

    But, do not despair. You actually use only a small fraction of what’s out there, and you can choose most of that.

    A few points:
    1. Software quality is a fairly new concern. Software only (relatively) recently started getting so big that it couldn’t fit into people’s heads anymore. There are movements now to take control of it. Eventually the same sort of stuff we take for granted (encapsulation, abstraction, etc.) will also include testability, fault tolerance, and actual quality.

    2. You may have to live a little like a monk, but you can surround yourself with good software. It’s kinda zen, really, and makes life a lot easier. It’s what I’ve done and I’m 10,000% happier for it (I measured :->).

    – First rule: buy hardware from the vendor that makes your OS. It’s the toughest, but not terrible. Mostly this leaves Sun and Apple. I’ve got one of both (laptop & desktop, guess which one’s which :->) and that \one number to call\ security is actually pretty good. OpenSolaris 2008.11 is pretty good. Donno what’s causing your force quits on the mac so often, I get about 1 every 4 months. I suspect it is Netflix streaming.

    – Second rule: For everything else, you have virtualization.

    – Third rule: trust old stuff first. Tar for backup, perl, python, or bourne for scripting, text files for most data. Emacs, bbedit, or vim for editing. Only when you get a real advantage (e.g. zfs snapshots) do you get fancy.

    – Fourth rule: vendors matter. Some vendors just do terrible jobs (was that Netflix streaming, with the Silverlight plugin?), some do mediocre jobs (apple), and some are pretty good (gnu, sun). Nobody’s great, even nasa crashes a lander now and then. But, the higher percentage of your apps coming from the better vendors, the more reliable your systems will be.

    3. Remember to apply your expectations for other systems’ quality to your own code. It’s an eye opener! Also, it helps you learn to build reliable configurations of unreliable parts. Which, so far, has been the most reliable setup I’ve seen.

    Cheers brother,

    [James’ Reply: Some seriously good ideas here. It steals my thunder a little bit, since I was going to advocate some of this in my next post. Thanks, man.]

  2. You get some quality from testing after development takes place. But a much better way is through TDD (which I suggest that you look into.)

    This is the ONLY way to have true unit tests around all code and to implement the leanest possible code for a particular solution.


    (White Box Testing) > (Black Box Testing)

    (The cost of a bug that is passed to a tester) > (The cost of a bug fixed before going to a tester).

    [James’ Reply: Yes, I’ve read about TDD. I’ve done TDD. TDD is interesting. The only people who say TDD is better than testing are people who do not study testing. It’s nice that programmers want to produce better code. TDD helps with that. But white box testing is not necessarily “greater than” black box testing. It maybe less than. I try to avoid those misleading terms, actually.

    Please don’t use the phrase “the only way.” That’s a marketing mentality, not an engineering mentality. If you see me use that phrase, call me on it. Thanks.]

  3. Couldn’t agree more James.

    And whilst I agree that the rise and rise of TDD is an enormously good thing for development I also agree that it’s a very far fetched to state that it produces zero defect software.

    All it can ever do is confirm that the software works in the limited number of scenarios that the developer thought of up front before writing the code to pass the tests (the two things are so very far apart that I’m amazed someone talks about zero defects without the slightest hint of irony).

    [James’ Reply: You know, I used to be a zero-defects advocate. My excuse is that I was 21 years old. At that time I was also an Objectivist. I’ve learned a lot about complexity and human behavior since then.

    Anyone who implies that the answers to our problems are simple strikes me as a child, or as an adult intent on treating people like children.]

  4. @Nicholas – ‘Now when was the last time your computer froze?”

    Last night. It’s a near-daily occurrence, that hasn’t changed. Removing the mechanical “hard reset” button from the front of the computer case is the dumbest move of the last ten years. These days I have to reach behind and flip the power supply switch.

    You say you don’t get the BSODS and systems dying. I’m really glad for you, but I haven’t noticed much change. I still get them. Systems just refusing to respond are NORMAL and this has been the case without change since the IBM PC was invented. I find that I still need the electronic equivalent of “hit it over the head with a hammer” on a regular, near to daily, basis.

  5. Well here we go, another degeneration into an irrelevancy about which is better, this or that, old versus new etc. etc. etc.

    The fundamental issue is bigger than testing, technology, individual experience or even one person. The quality of the products and services we get today are a function of a wider malaise within the world we live in. The minutiae surrounding the individual merits of this versus that, are irrelevant as the whole encompassing view is one of poor, pi#$ poor, awful crap that we accept on a regular basis.

    I have worked in the business of software for nearly 20 years. In that time I have had the kind of experiences that would make unbelievable reading to those outside of our industry. I have done some things that have left a sour taste in my mouth and in other cases taken a principled stand. Without a doubt, the principle of Quality is the one most abused in whatever context you can place yourself. Quality within our industry is subjective, a measure that we take and define whatever way we want it to be, so that we can do what has to be done. I have only recently in my career taken the point of view that the only measure of Quality is what the customer will accept. I work to inform them, support their decision making process and help them understand the limitations that we and they labour under.

    At the end of the day, the customer will make their choice based on the minutiae people are talking about here. That and cost.

    Quality is not dead, we can rebuild it, we have the will 🙂

    @Mark Hughes, if you truly believe that there is some fast approaching Utopian horizon that yields the promise of Zero Defect software development, you are either a relative newcomer to this industry, a supremely arrogant developer or a fantasist. How many years has that mantra been taken out, dusted off and prostituted on the conference circuit? Answers on a postcard please! 🙂

    [James’ Reply: I sure appreciate your passion, Ivor.]

  6. It is hard to fathom what has gone wrong with young CS majors. Why do they write such awful code? Is it because they started with C++? Or is it just some collective character flaw in their generation?

    They were taught to code by teachers of our generation, and they grew up with the products that our generation developed. Entropy accelerates.

  7. @David,

    Ah, yes, you’re correct: I presented (and understood) Melkjug as the wrong piece of technology. That said, I appreciate your feedback and sent it on to the developers.

    PS: I went through the same process you did, remembered your post and was floored at the amount of discussion made in my absence. So ya, I feel your pain.

  8. Wow, what a load of BS!!! Quality is dead? Really? You rant about software being of poor quality and I’m sure I could add a few bugs to your list; why does the little mute light/button on my HP not stay in sync with the little speaker icon in my task tray. It’s probably because they didn’t test VISTA against a laptop that was released AFTER they released Vista. Is this a quality issue? Should HP or Microsoft have hired some testers from the future to test this combination?

    Most software bugs can be categorized into three categories; software/hardware or software/software incompatibility, user error, or stupid programmer error. The number of permutations of the first category would require years to release any new piece of software and STILL wouldn’t prove that your software would never break. The second category of user errors includes everything from not reading the prerequisites of the software, to doing something with the software that it was not designed to do. Most BUGS are from this category, software is getting better and better in not allowing user to shoot themselves in the foot, (when was the last time you heard of someone accidentally deleting their windows directory?) but you can’t always protect stupid. It’s the third category that always get the blame and in many cases it may be right to blame them, but… Was the Y2K issue a software bug?

    Outsourcing is here to stay, so get over it. Companies are outsource because management percieves that their developers are going to write crappy code anyway so they might as well outsource and get crappy code for cheaper. We as developers are responsible for this perception, we have historically not done a good enough job understanding what the customer wants, instead we give them what we would want if we were the customer. We are the worst customers on the planet.

    By the time we get to 2.0 of our software it is different enough form the original that we wind up trying to pound the square peg into the round hole. AM tries to solve this problem, stay tuned to see if it successful. Maybe someday software built using one of the AM’s will have a little sticker on the outside say “Build using eXtreme Programming”.

    Software quality is getting better as newer methods of development reduce the number of bugs that make it to QA. The downside of the “developers will test their own code” movement is that testing departments are getting slashed or off-shored. In this point you are right, but how many pieces of software do you have on your system that are pre-1.0 or beta software? Beta=you are the QA department.

    We as software users must demand that companies push the quality slider to the right.

    [James’ Reply: This comment is kind of all over the place. But I’m sure of one thing, you’ve offered no data. You say software is getting better. I would say that is not generally the case. I would also say quality is out of the hands of any one vendor, because of the flawed tools and frameworks they are using, the OS’s, and the heterogeneous computing environment.

    It bothers me that I also have little hard data beyond my own daily experience, except the Infoworld test that I have cited. I wish I had access to corporate bug databases. I believe I would find a vast increase in the incidence of intermittent bugs or bugs otherwise deemed unfixable.]

  9. In 1991, with System 7, I would not say that Apple’s software had qualitatively better quality. In 2009, with Mac OS X, I say that it does. Partially, this is because Apple has maintained a reasonable QA process while competitors have neutered theirs, and partially because the unix model of independent processes with protected memory has made automated crash logging much easier than it was in Classic Mac OS, making QA more effective.

    I am not claiming Apple’s software has no bugs. I am claiming that the majority of “mysterious” crashes and problems experienced on Mac OS X are caused by defective RAM. And users are accustomed to crappy software, so they just assume the problem is unfixable. Many many Mac users have a trouble-free experience. If your Mac experience isn’t trouble free, don’t just bitch, get the problem fixed. Specifically, if you upgraded your own RAM, pull out that module and run with just the stock RAM for a few days and see if your problem vanishes.

    BTW- I agree with your core thesis here. But the solution will not come from the vendors. It will come from users voting with their wallet. That’s one of the reasons I wish Apple would take every source of instability seriously, because customers can not distinguish the difference between a product with a 1% failure rate from one with a 0.001% rate. If it fails for them, or someone whose opinion they trust, it might as well be 100%. To differentiate by quality in the marketplace, it’s not good enough to be better, the product has to be so close to perfect that people assume any defect will be remedied.

    Incidentally, that’s the model customers expect for the iPhone. If they have apps randomly hanging on their iPhone, they walk into an Apple store and get it replaced. That’s a powerful financial incentive for Apple to maintain quality. And it’s worth noting that the iPhone is running the same Mac OS X as the desktop. The tools Apple uses to maintain software quality on the iPhone were developed first for the desktop.

    [James’ Reply: I have had some trouble with my iTouch and iTunes, though overall I am happy with its reliability. I am not happy with the draconian digital rights management which caused it to dump several hundred of my legally owned songs when I dared to sync it with a new computer.

    iPhone had some important problems, though, when it was released. Don’t you remember? It was all over the news.]

  10. Good day James,

    I agree that many companies and teams have given up on striving for quality. The problem that I have is the reason why. It takes more (in $$ and time) to deliver a quality product. I don’t agree with this premise. A team of experienced developers with an understanding and lean towards making quality software will do so in less time than any team of cheap developers or outsourced team that has longer feedback cycles with the customer. Agile teams can get better time to market and with integrity built in to a sufficient level.

    Lets teach more people to create quality software and more will be demonstrated in the marketplace. Stories will find their way out about the shortened time to market for these products and there successful subsequent releases. This will NOT solve the software industry issue around quality but it will create more quality in our profession and maybe little by little it will continue to grow. But it is a long road and there will always be less experienced and cheap labor around that will jump to create crappy products and we should be able to acknowledge their results, successful or not.

    [James’ Reply: Chris, I think you may have missed my point. Let me say it more bluntly: quality is impossible.

    Now that’s a simplification. Quality actually IS possible, in some or other context. We can play games with defining quality loosely or restricting use of products. But what I’m trying to say is that it doesn’t matter how good your developers are, if you release a product “into the wild” it will be glitchy and troublesome NO MATTER WHAT YOU DO because of THINGS BEYOND ANYONE’S CONTROL OR KNOWLEDGE.

    The entropy of natural operating environments, with many accidentally interacting components, dictates that this is so. And the result is that programmers shrug and users grin and bear the pain.]

  11. Well I think your looking at quality too one dimensionally take for examples all those processes that start up on windows yes they consume a ton of resources but imagine your techno phobe needs to start one of them that isn’t started yet, how is he going to figure t out how to do that? Windows starts everything because it figures that most users won’t be able to figure out how to start a process if they need it. There are other operating systems that are for more tech savvy people that make different trade off’s. Also quality is just one pillar, cost, schedule and scope at least in the short term and in the small they are tradeoffs. In the large and long term quality will impact all the other pillars.

    [James’ Reply: This doesn’t affect my hypothesis that quality is dead.]

    Mac versus windows is very interesting debate here is my take. An OS’s purpose is to run software. Windows runs more software than any other OS so in my books it is the best OS because it does it job better than any other.

    [James’ Reply: It’s the best because it’s the best because it’s the most?]

    There are software projects that have failed striving to hard for quality and there are projects that have imploded due to lack of quality. Good enough software is hard middle ground to find.

    Also if we were satisfied with simpler software we would have a lot more quality software but we want flashy easy to use guis, software that understands all the complicated tax laws out there etc etc. If you look at the best software you will find it to be a simple only doing one thing well. It is easy to get one thing right, now compare that to software we write with hundreds or thousands of requirements/features. Trying to be all things for all people just never works that well and probably never will.

    [James’ Reply: I agree. That’s a big reason why quality died.]

    As a person who writes software for a living the quality mantra gets old, people act as if we write bad software on purpose or that other fields do such a better job then we do when it comes to quality (this is total bull). Family, friends and I have been ripped off by enough by poor trades man and contractors to know better. Plenty cars have been recalled, bridges have collapse, planes have fallen out of the sky, space shuttles have blown up, toys with lead paint, every industry has quality issues and has had fiascoes.

    Also since you’re a tester I would expect you to have a rather bleak view on software quality try writing some software and you will notice that it is harder than it looks. Actually try any other profession then you own and you will notice no matter how simple it looked it is harder then you thought it be.

    [James’ Reply: Yeah, I was a developer before I became a tester. I know it’s hard. Did you think I was calling it easy? My hypothesis is that it is TOO hard to write software, these days.]

    Now I will agree with that Testing is extremely important I agree with Fred Brooks that testing should consume 50 % of the development cost and schedule in some form or another versus coding which should only consume 25 % effort. If your following a more archaic process it means you should have 2 testers for every 1 developer, I have never at the places I worked. In an agile setting this might come in writing as much or more test automation code then production code.

    NASA has 4 testers for every 1 developer but most companies can’t or won’t spend the money to achieve this kind of quality so if your unhappy with software quality then don’t buy it obviously despite your complaints you feel better off with this crap software then without otherwise you would not buy it. Or you can write your own custom software and make it is high quality as you are able to.

    Anyways interesting post.


    [James’ Reply: Thank you. My post was interesting. I hope you read it, sometime. From what you wrote, it seems you only read the title.]

  12. Emblematic anecdote:

    Today I awoke late (better rested than usual, but awake).

    I use a little alarm-app to play a 15-second mp3 of a gong, repeatedly, No such gong happened. Well, maybe I turned the sound down before I went to bed. Go and look.

    My XP system has BSOD’d (which is quite surprising to me). I reboot. Eventually, I find that XP wants to file a crash report, which I OK. Then I get a balloon help popup. It seems that my system has just accepted a mandatory update from Microsoft.

    BSOD on completion of system software patch? Hmmmmmmmmaybeeee….

    Wow. I lack the time or inclination to figure it out. I’m left with the faint free-floating hope that a BSOD won’t happen again soon.

    But The BSOD did include the helpful suggestion that I might want to remove any software *I* had recently installed. Not one clue that “it” (the computer-Microsoft collective) had done so.


    And incidentally, on reboot, my computer lately sets its sound level to zero. Not sure how long that has been going on, but I haven’t figured it out, either. How would I? So even if I had gotten a full reboot–I wouldn’t have heard the alarm go off.

    Froth at mouth, rinse, repeat.

    It’s this death by a thousand cuts that saps one’s strength.

  13. [James’ Reply: Thank you. My post was interesting. I hope you read it, sometime. From what you wrote, it seems you only read the title.]

    Sorry for failing to contribute anything meaningful and wasting your time.

    [James’ Reply: Okay, but I was serious. Read what I said was my hypothesis. Then attack that if you want.]

  14. Troy’s response was right on.

    ‘Quality’, in so much that it can be measured, costs. If software companies were to apply the same level of ‘quality’ to their software as is applied to say, the software on the mars rovers, it’d be prohibitively expensive for consumers. There’s a cost/benefit trade off, and customers are an equal part of that equation. To the average customer, a few blue screens every once and a while is a good trade off for paying $200 instead of $20,000 for an OS.

  15. I utterly disagree, JerseyGuy. There is no ‘cost/benefit trade off”. This is a completely dysfunctional market, and even if I’m fully willing and happy to pay more for quality, it isn’t available to me at any price. Customers are in NO sense any part of the equation.

    If a developer did create a reasonably solid operating system, he couldn’t reasonably anticipate selling it in enough quantity to be profitable. To the extent that it wasn’t compatible with the Windows monopoly, the market would reject it just as it has rejected Macintosh and *nix. To the extent that it was compatible, MS would sue it out of existence or buy it and bury it.

    Even then, there would be no applications for it. Mac users should be very familiar with this one. Why should I write an application for MacOS when I can write it for Windows and sell ten times the number of copies?

    Do not pretend that it’s the market’s fault when the market has never been allowed to function. There are those who WOULD pay $20,000 for a bulletproof OS, if they had that option. There are many who would pay a lot more for something that worked better than this. It’s not available. There are no price points and no options on this graph.

  16. I think one place that’s really starting to get it is, unlikely enough of all, Microsoft. Maybe not all groups, no, but their core OS folks get it. On the Engineering Windows 7 blog, they talk about their teams: usually 1 program manager per five developers and five testers. It shows with the core of Windows 7 in their beta: it’s a rock solid product. The outlying programs (Windows media Center, some of the other add-in programs) don’t seem to have this level of service. Windows has been one of the biggest eyesores in the PC market for years, but its almost exclusively made itself better over the years. (Regarding apps and drivers written for Windows, that’s a whole different matter… 😉 )

    There are some industries out there where ‘it has to work right’ and there’s no way around it. Mission critical embedded systems are a good example…well, most of the time. If the flight system in an F22 goes out, say, because they crossed a particular time zone at a particular time, no one will put up with that, ever. Inquiries, jail terms, and in the worst case deaths appear when people’s lives are on the line and the quality isn’t there.

    Here’s the kicker that I’m sure you understand but very few realize: quality in a product doesn’t increase it’s long term cost, it /reduces/ it. The less money you have to pay for support, the less rewriting you have to do, the fewer dependencies you rely on, the less time you spend tracking down issues from year old bugs, the simpler and the stronger your codebase is, the cheaper it gets. Problem is, it takes brains, guts, and time in the short term to move in that direction. Unfortunately, the easy way out is always present.

  17. I thought what I had to say was relevant but ultimately I am not the judge of what you feel is relvant.
    As long as there are people that care about quality, quality can’t and won’t die completely.
    Quality is doing the best job you can in the situation your in. I care about the code I write so it is of higher quality
    then if I did not care, I actively study my practices and the practices of others to improve myself.

    To me Quality is an attitude, if everyone said well I can’t make this perfect so I am not going to do it all there would be
    no value at all created in the world for anyone. The fact that software is valuable despite all the bugs and cost of producing it is self evident.

    In general my belief it is at least twice as hard to verify a product then to build it. I have never worked in an environment that recognized this in a put your money where your mouth is fashion. I have joined the test infected crowd and write unit tests for most of my code, I have come to apreciate the difficulty of writing good tests it is a skill set I need to improve upon. Hence why I know who you are since I am considering a career move into testing. Nice talk on becoming a software testing expert by the way loved it.

    I never bitch about the quality of products that I have the choice to avoid I think this is pointless I believe in caveat empor buyer beware. I can rant about some of the horrible pieces of software that I am forced to use at work and how they turn something I love to do (writing code) into an utter nightmare sometimes. In defense of these horrible tools (like Rhapsody and Clearcase) someone somewhere made a free market decision to buy these tools they perceive some value in them there perceptions remains a mystery to me and my co workers. Value and quality is in the eye of the beholder and the fact that people pay lots of money for something is the most objective measure of value and quality I can think off. Certaintly commercial success is a better indicator of quality then some quality assurance person giving
    their blessing.

    I was hoping to contribute to this blog because I felt I had a lot to say. If I have gone off on an unwelcome tangent I apologize again.


  18. I cannot help but draw a parallel about what you mention about falling levels of quality (or rather its death in your view) with the life we live today and its many ‘unsavoury’ accompaniments versus say 2-3 decades back when our forefathers were the flag bearers of a relatively less complex life veering towards simplicity and order.

    Do we not sense and realise that our lives are today more complicated, stinkier by the day, falling in value ans substance, cursed and step-by-step rotting away from the (uncontaminated) life that we knew existed say back in the 60s/before. Are we not now witness to scenes of crime/violence/hatred which becomes bloodier and more horrific than the previous one? Vietnam, 9/11, Mumbai terror attack, Winnenden school shooting, & so on…

    How do we counter this in our own individual microcosmic way? Do we give up on teaching good values to our children? Do we not stop the sibling fights and not abhor violence of any kind? In a bus, when I see an elderly woman struggling to keep her balance, do I not offer her a seat? When I hear a child speaking profanities do I not admonish even if she is not a blood relation?

    Of course I do.

    Quality of life is in our hands to a great extent and we must continue to encourage, inspire, instil and ensure that we influence its goodness to the extent possible.

    Likewise when a product we are not happy about is readied for release would we not raise a RED flag?

    Of course we will.

    We will be vehement in giving this information to the powers-that-be supported with defects information, tests undergone/not undergone, plan ahead for testing and so on… When we do this do we not try to protect the user who should better not use the product in its present shape. Do we do this because it is our job? Yes but is that all. I think the voice comes from within & it is equivalent to the scenario of a small boy uttering profanities.

    We do our best and we should.

    So I do not agree with your “Quality is Dead” completely. As much as the Project/Release Managers will push the buggy products out there would be test professionals (like us) standing in the door wearing and shouting ‘red’. The test professionals may be outnumbered and shooed away but the bugle would have been sounded. Sanity would prevail at times and it would not at others but death – No. We would not let it happen!!!

  19. I stumbled upon this when I was searching for more info on why the quality of Vista was so bad, which some have attributed to the fact that Microsoft has taken the view that their software quality depends on having it pass automated test scripts. It’s a lot easier for ABC manager to prove that a program has passed XYZ tests than to explain that the software has some sort of subjective quality problem. So, software projects are driving more towards the types of things that can be measured. IMHO quality may be subjective, but purchases are definitely binary. Showing that there is a connection between these things is how to convince companies to produce higher-quality software.

    I wanted to comment on the products you mentioned. I haven’t used Tivo, but I am a Dish subscriber and use the VIP 722. I think it’s an awesome machine, however I keep noticing bugs that confuse me and cause me to sometimes miss shows. The more of a power-user I become, the more bugs I see. I was pretty upset last week when the end of a basketball game I was recording got cut off.

    I’ve been a Mac user for the past 3 years, and I have little interest in using Windows unless some program requires it. Apple has had some real issues with hardware, although their software works pretty well. I think this idea that the hardware/software combo has been so thoroughly tested has a lot of merit. I did own an Ipod for a while until it crashed. Now I have a Sony Walkman that has a worse interface, but far superior hardware. I happen to be one of few people who is willing to put up with some clunkiness to get better sound quality. I realize I’m making a trade-off. I do still use Itunes because I love how easily it interfaces with podcasts.

    In fact I think our experiences with “pretty good” software like OSX has made everyone less patient with a poor user experience. So I don’t think quality is dead–I just think your company is dead if you aren’t figuring out how to make your users happy. It’s a complicated problem, and I hope product developers take the human aspect seriously.

  20. I got to wondering, James, why you don’t simply repudiate Vista? Just uninstall it and reinstall XP, get back in control of your system?

    [James’ Reply: 1) My computer came with Vista on it, so I don’t know if the drivers in XP will work with the chipset. 2) I don’t have a copy of XP that I can install, since Microsoft stopped selling it. 3) It’s a netbook with no built-in CD drive and I don’t know if the external CD I bought for it will work during installation of an operating system. 4) It’s an annoying use of my time.]

  21. Wow! And I thought it was just me. Nothing annoys me any more than paying big bucks for software that is buggy or just doesn’t work the way it should. Complain to the company and all you will get is a comment that they \think\ it must be that your computer doesn’t meet the operating requirements of the particular software package that you have purchased. It’s just too bad that you have spent X dollars for nothing.

  22. Hi James,

    It is interesting that you are mentioning financial industry, but not in the context of financial software development / testing. Well, as someone working in this industry for more than 10 years I can tell you that the management is very much aware of the consequences of software defects for the companies. Your hypothesis clearly does not apply here as the penalties for low quality of software are pretty steep. Yet in current economic conditions the financial companies behave very similar to those who (as you believe) just gave up on quality. They slash the cost by replacing trained, experienced, smart testers with junior offshore staff. I do believe that they behave rationally, but the explanation is different (and simpler) than yours. In hard times the companies just slow down their development efforts, and they simply don’t need the same level of testing staff for their IT departments. In fact they also cut application development staff albeit not that drastically. My explanation at least gives a glimmer of hope to us testers as the economy will recover sooner or later, and we just need to be properly prepared for that time. We need to sharpen our tools and methodologies, and learn to better articulate the value added by software testing. As new application development projects pick up the speed we should be able to deliver competitive advantage to the companies that care about quality, and by the way hire better software testing staff. If we can not deliver then I am afraid software testing will end as profession.

  23. Just curious James, why did you migrate from XP to Vista? I suppose there must be reasonable legit reasons to do so. But I can’t seem to think of any myself? (Perhaps I’m living in a bubble…) But I am curious. I have found XP to be quite manageable as apparently you did as well? (well quite manageable as long as I apply my standard methods, which I would describe as ‘utterly ruthless’) I have worked with Vista when absolutely required. It’s brain dead. (I have so far discovered several innovative features in Vista, well exactly three… 😉
    I would like to present the notion that even spectacular software would seem like crap when running on Vista?
    Clearly the same thing applies to the affect of all the worthless meaningless processes running on most machines today.
    The installation of update programs, app preload programs, endless unwanted and unnecessary system hacks (that usually provide crappy versions of components and such that Windows already provides) and on and on, ad nausia… creating this situation where you MUST use Autoruns and ProcExp just to use your machine! (as you mentioned, in so many words…)
    Point being that no amount of quality can overcome an OS that is choking on crap and/or an OS that is choking on it’s own crap.


    [James’ Reply: I got an HP Netbook thing that only comes with Vista installed. So I was stuck. The next system I got was XP, but in Swedish, and it turns out it’s not trivial to change the language to English!]

  24. I agree: Quality is dying. Or dead already.

    I remember a Technology Review article on the topic, maybe five years ago?, that came to the same conclusion for very similar reasons: Given the classic three-way tradeoff (quality, price, time), customers demand poor quality NOW and look for competition on price.

    I’ve lived through this problem for fifteen years now. I’ve tried to find companies to work at where quality is forced to matter because the customers demand high quality and look for competition on price and time. It’s been a long and generally unrewarding road. It’s probably safe to say that I’ve turned from a highly productive, ambitious, helpful tester to an angry, frustrated, morally corrupt smartass. I used to give a damn and I miss that. I’m far enough down this path that it’s going to take a big kick to either go another path or figure out how to make the path I’ve chosen bother me less.

    Please hurry up and get on with your ideas on the “what to do about it” part! 🙂


    [James’ Reply: Ah yes. I need to write the positive part!]

  25. I laughed OUT LOUD!! Thank you, for saying what I too have felt. I MOURN my tivo! why couldn’t the others get it right. Why couldn’t I keep my tivo! I had to choose, HD or Tivo, and believe me it was a hard choice. Thanks for letting me know I wasn’t out of line for feeling shafted.

  26. I have to admit, I only made it about 2/3 of the way through the comments before I gave up.

    I have to love the folks lecturing you about how you should use this or that platform, or saying you are ignorant of his or that magic bullet practice that will solve all the problems.

    Surely it’s proof that the quality of their reasoning is dead… without knowing you, your background and experience they jump to the conclusion that the problem is ‘YOU’ and not the software/os/etc. Lovely ‘blame the user’ mentality there, which I think is yet another aspect of the underlying problem. (reminds me of the advert slamming the cable companies. ‘we need better customers, ones who don’t want so much from us’.

    I’ll disagree on two points however.

    1) security isn’t worse.. it’s far better than it was.. However the landscape is far far worse than it was 15 years ago, by orders of magnitude. So the net result is that we are more likely to experience a problem. 15 years ago you could put a system up ‘naked’ without a firewall on the internet and be just fine. if you put that same system up there now, the ‘time to ownage’ could likely be measured in minutes.. (and I don’t care if it’s a windows system, or a mac or a unix/sun/*nix box, they would ALL get owned in fairly short order if they were running an OS from 15 years ago without any patches since that time)

    2) users are stupider.. well maybe that’s not fair, lets say ‘not technologists’ shall we? 15 years ago you had to be computer literate to use a PC of any variety, even mac’s of that era required some level of savvy from the user. The average user in those days was far more skilled than the average user today.. And the system did far less then, there was less to go wrong, fewer places for things to break. Now I’m not sure if that trend towards less literate users has accellerated faster than the improvements in usability and self diagnostics, but one thing is for sure, when something goes wrong, the average user of today is far less equipped to deal with the problem in terms of experience and troubleshooting skills, than the average user of 15 years ago.

    we’re also running more programs and a greater variety of hardware, all of which increases the potential for adverse interactions, so the potential for problems has increased (e.g. the pairwise test set would be substantially larger)

    there’s other trends in terms of landscape change that have affected us also, such as slimmer margins for computer makers that have forced them to seek additional revenue by preloading new systems with tons of crapware..

    but still I think your fundimental point remains true, it seems a lot of companies have either given up on quality, or drunk the cool-aid and fallen hook/line/sinker for some bogus silver bullet (automate all testing, offshore it all, replace testers with unit tests and tdd, etc).. There are anti-patterns to this, such as the agile notion that the entire team is responsible for quality, not just the QA guys at the end of the cycle, and I think largely they are headed in the right direction, but if the upper level management doesn’t give a hoot, and doesn’t fund time and energy for quality, those things are doomed to fail (imho anyway)


  27. MartinG, I’m sure James or I could also come up with a pretty long list of our infelicitous experiences with, say, Android phones, but the list would probably be of things almost all of which are only typically experienced by Android users.

  28. I’ve been involved with systems and software for some time. After some personal accounting it seems about 30% of my professional time has been spent trying to make something work or recovering from something that didn’t. Discussions with others come up with agreement.

    I recently spent two days to tracking down the reason the partitions on two disks in my home system disappeared. It was a driver that was part of a group of drivers. The partitions reappeared after uninstalling the driver. But not before a lot of frantic time I couldn’t afford was spent looking for other reasons.

    Drivers were not suspected initially because of weeks of booting and performance issues. This seemed to be a continuation of the problems. The updates were an attempt to address those problems. I was already blaming everything else. Bad power supply, bad disks, bad memory, a BIOS update done the month before in attempt to resolve those problems, and others.) It’s gotten to the point I have little faith in anything. Anything that presents even the smallest problem or makes a change that causes time be invested to adapt is out, unless there is no choice but to keep it. And I’m trying to avoid getting into that situation.

    I love my toys. I enjoy the type of work this world enables. New toys are great. I’m a person others ask about them. But I have been walking away from many without thought.

    For a while I was in the mode of doing research. That is just as much a waste of time, even though it can produce good results. Most of the time is a choice of the lessor evil. Even the old reliable companies seem to have gone to the dark side. Reputation is not a solid factor any longer.

    I may be looking at the toys I’ll have for a long time. And they may not be updated. I don’t know how precarious the balance is. It’s just not worth it.

  29. I did satellite installs for 10+ years, (for both Dish and DirecTV) and I met plenty of people who swore by TIVO. Great systems, so I’m in agreement there.

    The early Dish & DirecTV DVR systems were indeed just awful. Not so much now though. They are actually good now.

  30. Thank you James, i now understand completely and know to define ‘quality’. Its like getting out from the matrix and seeing the true ‘wild world’. I am taking this thought to academia on my research proposal. Yep i know its going to be hard.

Leave a Reply

Your email address will not be published. Required fields are marked *


This site uses Akismet to reduce spam. Learn how your comment data is processed.