Counterstrings: Self-Describing Test Data

I was at a conference some months ago when Danny Faught showed me a Perl package for manipulating the Windows clipboard. I turned it into a little tool for helping me test text fields.

It’s called PerlClip. Feel free to download it. You don’t need Perl to run it.

One of the things PerlClip does is allow you to produce what I call “counterstrings”. A counterstring is a graduated string of arbitrary length. No matter where you are in the string, you always know the character position. This comes in handy when you are pasting huge strings into fields and they get truncated at a certain point. You want to know how many characters that is.

Here is a 35 character counterstring:

Each asterisk in the string occurs at a position specified by the immediately preceding number. Thus, the asterisk following the 29 is the 29th character in that string. So, you can chop the end of the string anywhere, and you know exactly where it was cut. Without having to count, you know that the string “2*4*6*8*11*14*17*2” has exactly 18 characters in it. This saves some effort when you’re dealing with a half million characters. I pasted a 4000 character counterstring into the address field of Explorer and it was truncated at “2045*20”, meaning that 2047 characters were pasted.

I realize this is may not be a very interesting sort of testing, except perhaps for security purposes or when you’re first getting to know the app. But security is an increasingly important issue in our field, and sometimes when no one tells you the limits and dynamics of text fields, this can come in handy.

Software Quality Lessons From Borders Bookstore

The in-store lookup system at Borders bookstore is pretty clean; relatively free of UI glitches. Still, I try to make it fail every time I go book browsing. It’s provided some interesting food for thought.

Case #1: If you press numlock, the keyboard gets remapped to put a numeric keypad in the middle of the keyboard. This means, among other things, that the L key becomes a 3. This functionality is standard stuff. So standard that when I noticed it, I thought absolutely nothing of it. Certainly not a bug. Then one day after I had played with one kiosk for a while and moved on to another one, a lady walked over and said “Those computers don’t work.”

“Oh, they do work,” I replied, “I’ve been trying to make it stop working, actually, and I’ve failed so far.”

“No.” she insisted, “If you type an L you get a 3. The keyboards are broken.”

Turns out she had tried using the other kiosk where I had pressed numlock. The lesson for us, here, is that it’s easy to overlook bugs because of our familiarity with computers. I saw the numlock bug but did not “see” it. Because the developers of the Borders software had not disabled the numlock key, at least one customer is now spreading false rumors about the quality of their product.

Here’s the funny part. When I explained about the numlock key, she denied that could have been the culprit. She seemed attached to the belief that the software was broken. After her denial, as I was offering to show her that the terminal did indeed work, she abruptly walked away. Well, maybe she was late for something.

Case #2: At the Manassas, VA Borders, I found that doing a search for a book on the in-store terminal, then pressing and holding down the backspace key for, oh, five to seven seconds, put the software into a funny state where endless error messages appear (the same script error no matter what you do) and all functions are disabled. I could find no way to recover from that mode through the keyboard. I didn’t play with it for very long, but I suspect that the system may have an exploitable security hole.

I reproduced the problem on three other terminals in the same store, then tried to report the problem to a manager. She told me she didn’t think that was something anyone was likely to do, and made irritated sighing noises when I told her that Borders IT people might be interested to hear of it, because of potential security concerns. Also there is the simple fact that any prankster like me can disable their terminals and watch (as I did watch) customers try to use those disabled terminals, only to walk away in frustration.

The shift manager told me that the computers weren’t networked, so that security wasn’t an issue. When I pointed out that the computers seemed able to access a inventory database that spanned all the Borders stores, she replied that yes, of course they are “connected to the home office” but that they are not “on the Internet”. Well, even in the fabulously unlikely event that they do not traffic on the Internet, the issue is moot. Private networks can also be hacked.

To be fair, I suspect the manager I talked to was pre-occupied with some other trouble. Her shortness with me seemed a little exaggerated. Still, it’s interesting to realize that a lot of customer facing technology, these days, is served by potentially irritatable people making $9.50 an hour and maybe not motivated to report problems back to the development team. We know about latent bugs, but there is also something you might call a latent bug report, which is a problem that keeps happening to people, who recognize it as a problem and yet never report it.

There are at least two interesting testing problems, here. One is that we have to go to special pains to build bridges with our users in the field so that we discover what is annoying them. The other is that third party technology such as Internet Explorer is brimming with obscure little features that must be explicitly turned off, or else will result in security holes. A quick search on Google turned up several lists of keyboard shortcuts in Explorer, most of which I hadn’t known about. I want to go back to Borders and try them all.

By the way, I did try to report the problem to Borders online. They sent me a nice note apologizing for the trouble I had, but so far they have not followed up with me to find out the details of the bug. Of course, if the incredibly rare event that someone would hold down backspace after doing a search suddenly became less rare, because someone wrote about it on his blog, that might alter their risk calculations…