Tester Pilot

Richard drove up to the hangar just as I was checking the oil on the Husky, his prized baby float plane. Nuts. He was right on time. I was late. I’m supposed to have the plane ready to go when he arrives.

“Hey Dad, looks like a good day for flying. I’m just in the middle of the pre-flight.”

“Where are we going today?” He asked.

“I haven’t been up for a few months, so I figured just a sightseeing tour around the islands and then some pattern work at Friday Harbor.” I hate pattern work: landing and taking off while talking on the radio to the other pilots. That’s exactly why I need to do it, though. I must get over my nerves; must become a safe pilot. It’s a lesson from testing: focus on the risk.

“How much fuel do we need for that?”

“There’s about 18 or 20 gallons on board. That’s actually enough, but I figure it would be better to bring it to 35, just in case.”

“How much will you put in each tank, then?”

“7 gallons.”

“7 plus 7 plus 18 doesn’t add up to 35. Decide on the fuel you want and get that. If you’re going to fudge, fudge toward having more fuel. What are the four most useless things in the world?”

Oh I know this… “Um, altitude above you… runway behind you… gas in the gas truck, and… Um–”

“–and a tenth of a second ago” he finished. “But you remembered the important one. Gas. We don’t want that terrible feeling when they close the airport for 30 minutes to clean something up on the runway, and we don’t have the fuel to divert.”

I could have quibbled with him. What we would actually do in that situation is land 10 minutes away at Friday Harbor airport, or heck, anywhere, because we’re a float plane in the midst of an archipelago. But that’s not the point. The point was the habit of precision; of being conscious and calculated about the risks I take, wherever reasonable. Again this relates to testing. When I’m testing, the habit of precision comes out in considering and controlling the states of my test platforms and test data, and of knowing what to look for in test results.

Dad called the flight center for a briefing. He already knew what they’d say, since he always checked the weather before he left home, but Richard Bach is an especially fastidious pilot. He’s not exactly “by the book.” It’s more that he prides himself on safety. Nothing surprises him without itself being surprised at how prepared he is for it. Yes he knew you were coming; here’s your cake.

Dad and me

The Tester’s Attitude in the Air

My father’s philosophy dovetails perfectly with the tester’s mindset. We expect things to go wrong. We know we will be surprised at times, but we anticipate the kinds of surprises that might occur. That means we perform checks that might seem unnecessary at times. And we keep our eyes open.

I was almost done with the walkaround when he got off the phone.

“Three knots at zero six zero over at Friday,” he announced.

I paused to visualize, then tried to sound authoritative. “That’s a crosswind favoring runway three four.”

“Yes. We have the same conditions here.”

Cool. I got it right. I’m supposed to pretend to be the pilot. Officially, Dad is the pilot-in-command, but I do everything he would do, while he supervises and is ready to take over in case there’s a problem. While I’m doing the preflight, he’s doing it too, but not telling me anything– unless I miss something important. Each time we fly, I’m determined to find a problem with the aircraft that he hasn’t noticed, first. I haven’t yet succeeded.

“Dad, what’s this rust colored streak coming out of this bolt?” Yay, I found something! “There’s one of each side of the elevator.”

“Just a little bit of rust.” He smiled and materialized a can of WD-40 and blasted the bolts with it. This airplane is pristine, so even a little blemish of rust really stands out.

“Were you flying recently?”

“Yeah, I went out last week and splashed around at Lake Watcom.”

“That explains the streaks. Water spray on the tail. Did you pump out the floats afterward?”

“No, but I doubt there’s more than a pint of water in there.”

“Let’s see about that.” I retrieved the hand pump while he popped out the drain plugs. He was right again, I couldn’t suck out more than a cup of water from the floats, total, from all the compartments.

But there was something odd about the last one.

“This water is PINK, Dad!”

Now he was not smiling.

“Unless you landed at the rainbow lake on Unicorn Planet, there may be a hydraulic leak in there.”

He put his fingers in the residue and sniffed it like a bush guide on the trail of a white tiger. “Yeah, that’s what it is. Let’s pop the hatch and take a look.”

Testing With Open Expectations

This is an example of why good testing is inherently an open investigation. Yes, I had definite ideas of what I was testing for: leaky floats. Yes, I had a specific method in mind for performing that test. Had I not a specific method, I would have developed one on the fly. That’s the testing discipline. My oracles were judging the amount of water I was able to pump out of the floats compared to other occasions, and I also tasted the water a couple of times to detect if it was salty. It shouldn’t be, because it had been several flights since we had landed in salt water, but I check just in case there was a previously undetected leak from before. If salt water gets in there, we could have a serious corrosion problem.

I had no conscious intent to check the color of the water. But in testing we take every anomaly seriously. We take data in and ask ourselves, does this make sense? In that way, we are like secret service agents, trained to deal with known threats and to have a good chance to discern fresh and unusual threats, too.

The question “Can I explain everything that I see?” is a good heuristic to keep in mind while testing.

But if I were to have automated my float pump tests, I would never have found this problem. Because unlike a human, a program can’t look at something and just try to explain it. It must be told exactly what to look for.

I got an email, today…

Geoff confirmed the hydraulic leak at the connection in the left float, and will be sealing it, probably tomorrow.  He’ll move the Husky to the big hangar to do the work. Nice, that you decided to pump the floats!