Two months ago, I had a $732 charge on my Sprint phone bill for “telephone equipment.” I had not purchased any telephone equipment, so I called customer service to dispute the bill.

The service agent told me “we haven’t sold phone equipment for months, so this can’t be right. I’ll remove the charge.” (Notice how the human used inference to provide me a service that I can’t imagine any automated system providing as part of resolving a billing dispute).

The following month I received a notice of impending disconnect if I failed to pay the past due $732. So, I got back on the phone with Sprint, ready to breath fire on them. The friendly agent told me there were notations on my customer record about the problem, then put me on hold for 10 minutes or so while she researched it. When she got back on the line, she told me that the first agent had failed to fill out the proper forms to remove the charge. It turned out that filling out those forms takes several minutes, so she asked me to stay on the line with her “or else someone else will call and I may forget to do it.”

The interesting thing about the second agent is that she told me how the charge had occurred. Apparently, I bought a telephone from Sprint three years ago on installments. When I moved my business line to a new location, a couple months ago, the information about the long since paid for telephone was moved with it and somehow became refreshed as if it was a new transaction. Hence the new charge for equipment not even any longer sold by Sprint. (Notice that the second agent– her name is Virginia and she’s based in Sprint’s Orlando office– also behaved in a way wonderfully human, developing a causal theory of the problem instead of treating merely as One of Those Things).

What does this mean as far as possible bugs in Sprint’s software? I think any of the following:

• An obsolete feature remains active in their software. It could have been deactivated, or caught by some auditing process that knows no new telephone equipment charges are possible.
• Simply changing the address of an existing telephone line apparently means recreating the entire customer record, and the process of doing that apparently copies things that should not be copied, such as old records of purchases from years ago. This seems to be design flaw.
• A purchase order should require a specific authorizing action by a human agent. Either there is a usability problem in the software whereby the user accidentally signalled that I had purchased equipment, or the system automatically decided that I had purchased it without a specific authorizing step.

I’ve had a number of problems with Sprint; with their flaky website and billing systems. But I have not yet had a bad experience with their customer service people. Which only goes to show: quality is the whole relationship with the customer, not just what happens with the products. Virginia gave me her full name and direct line in case the problem comes back. I’m impressed.

And then there was Dell…

I no longer do business with Dell. Their customer service has been terrible. I once sent my malfunctioning laptop to Dell for service and they not only announced that they could not find any problem with it (when it came back to me the problem was even worse) but it came back with scratches on the screen. I could have sent it back to them with more specific instructions for making the problem happen, but by then I had so little respect for the ability of their tech support people I just found a workaround and suffered with it until I could afford to replace my laptop with a Toshiba.

But that’s not the bug I want to report. The bug has to do with paying off a computer lease I once had with Dell. Leasing a computer from Dell is just a bad idea all around. It’s very expensive for one thing. But the other thing is that their record keeping is shockingly bad.

I tried to pay off my lease early. And get this. They called me SEVEN times over the next several months to demand further payments, because their systems had no record of my final payment.

I had FedExed the payment to them. I had records of the FedEx being received and the check being cashed. Each time their collections people called I gave them the same evidence, and each time they accepted it, apparently making some note in my records. But every few weeks, their computer would once again kick out an alarm that I had not paid. These calls got increasingly farcical. On one of them, the guy called me demanding payment, then immediately announced that he could see from his system that I owed them nothing, apologized, and hung up. I don’t think I said a word other than “hello.” and “I know.”

During my various investigations of the problem, working with various agents, I discovered that there are three different departments that must all coordinate at Dell in order for Dell The Corporation to believe that a lease is paid off. This coordination was apparently not happening in my case. The reason for this was not known, or if known, I wasn’t told. The collections department is apparently yet another independent group, and is incapable of doing anything to solve such problems. Do not lease with Dell.

What’s the bug, there? I don’t know. It appears to be a completely inadequate system. I might complain about this to them, but what’s the point? I learned my lesson. I buy my computers strictly through my local computer store, now, which has amazing customer service (http://www.royaloakcomputers.com). I think they ship computers, too.

Dell seems to be trying to cut costs by automating more and training its people less. All the more need, then, to test test test those systems.

I recently used the United Airlines “self check-in” kiosk. It’s a touchscreen system at the airport that allows passengers to get a boarding pass for a flight without having to see a live ticket agent. United is keen for people to use them. The more popular the automation is, the less staff United needs for the ticket counters.

I’ve tried those kiosks twice, now, and twice they have failed me.

Here’s how they work. You touch the screen to start, and the system asks you for a credit card or any of various other forms of identification. After receiving that, the system conducts you through a series of screens to identify your flight and seat assignments. The boarding pass pops out and you’re done. Theoretically.

The first time I used one was just after they appeared at Washington Dulles airport, my home base. On my flight that day, I had applied for and received an upgrade to business class. A passenger can upgrade using paper coupons, electronic “coupons” stored in my frequent flyer account, or by spending frequent flyer miles. The self-check-in system correctly informed me that my upgrade had been granted, and then asked whether I wanted to provide paper upgrade coupons or use electronic ones from my frequent flyer account. My answer was “none of the above, I want to use miles” but there was no way to select that option. That option was missing. Since I had no paper coupons, and had no electronic coupons in my account, I was forced to abort the process and get in line for a human. The humans, it turned out, knew what to do.

If it is an ordinary operation to use miles for a service upgrade, why couldn’t the kiosk handle it?

For months after that incident I just ignored the kiosks. It’s too annoying to stand in line to use the automated system only to have to go stand in another line to see a human when the automation breaks down. I am a very frequent flyer. I often use my miles to upgrade. Whether the omission of that feature was an oversight or a calculated decision, I think United screwed up. Even if there’s some compelling reason to omit that capability, the system could still have displayed a message to the effect that the feature was coming soon, or that it was available from the human staff. The incident left me with not just disatisfied with the kiosks, but with the feeling that United is careless about the needs of its customers.

The next time I considered using the kiosks was on New Years Eve, 2003. The line for humans looked pretty long, but the self check-in kiosks seemed deserted. What the heck, I thought, I’ll try them again. I expected no trouble with my upgrade, since I was using coupons.

The system took me through all of the screens, then announced that the printer was broken and that I should try again with another kiosk or see a human agent. Grumbling, I stepped up to the kiosk next to it, which cheerfully informed me that I had already-checked-in-everything’s-fine-have-a-nice-flight-goodbye. I supposed it assumed that I already had my boarding passes. Meanwhile, the first kiosk I tried had reverted back to its welcome screen, luring unsuspecting travellers to the same time wasting trap.

What are the bugs, here? I think any of the following:

• The kiosk apparently does not specifically log whether or not a boarding pass was actually issued to the passenger. Or if it is logged, that function failed for me.
• The printer failure message directs passengers to use another kiosk, even though that is guaranteed never to help in the case of a printer failure.
• The kiosk assumes that passengers must have no need for its services if they’ve already checked in. But what if I’d lost my boarding pass, or wanted to change my seat, or wanted to un-upgrade or re-upgrade? The kiosk should not just kick me off if I’ve already checked-in.
• Even if the kiosk is intended only for initial check-ins, there could and should be a message directing passengers to see a human agent in the case of a lost boarding pass or a seat change request, etc.
• The kiosk continues to take new users even when it knows (after the first failure) that its printer is broken.

Many testers who work on a kiosk like this might think of testing the printer failed condition, but most of those, I suspect, would have stopped after seeing that the apparently “correct” error message appeared. But the bug I found is only revealed when you go beyond the simple function tests and look at the complete lifecycle of a passengers interaction with the system.

The Moral
The moral, here, might be “scenario testing gives you value that function testing does not.” Another moral might be “watch users use your system.” If I was United, I would be studying log files and surveillance video of users trying to figure out the kiosks. I would count the number of people who struck out at the kiosks and defected to seek an audience with a friendly and expensive human.

Oh, and that’s another moral: friendly and expensive humans are difficult to replace with computers, because humans can cope with things that go awry. That’s why pilotless aircraft will never ferry human passengers. We can accept humans making mistakes, but we can’t accept machines making a mistake that a human might have averted.

United’s brittle software has cost it real money, too. The reason I was flying to Las Vegas on New Years Eve was to get 4000 more miles to qualify for a special status as a United frequent flyer. I flew to Vegas and immediately got back on the very same plane to come home. I would have been happy to purchase a ticket and not flown anywhere, just to get the miles. United could sell the seat to someone else, and I could stay comfortably at home, confident in my exalted new status as an elite United customer. Everybody wins. For whatever reason, United can’t do that.

But when I got the airport, they announced that the flight was overbooked and asked for volunteers to be bumped from the flight. I considered volunteering, as long as I was credited with the mileage. The gate agent said that might be possible, but on reflection, I realized there was nothing the agent could say that would convince me that United’s computer was smart enough to give me the right kind of mileage credit. My lack of confidence in United therefore led directly to them kicking someone else off the flight and giving them a free ticket or travel voucher.

A Personal Appeal to United
Hire me. I’ll give you a discounted rate. I’ll work for miles (literally and figuratively). Let me help you test your kiosks so that your customers will have more confidence in you. Look, I can’t afford for you to go out of business. I’ve invested too much in you. Don’t make me start over with another airline.