Ever since I got into the testing field, almost 20 years ago, it’s been a truism that military software development is moribund. It’s not that they love process, it’s that they love bad process. Documentation? Bad documentation. Who can look upon 2167A or Mil-Std-499 without dismay? I’ll tell you who: paper mills and people paid to arrange the ink on all that paper. It’s just a scam and a travesty.
I was asked, in 1998, to analyze two military software software test plans, each for a different major weapons system. I told them up front that I was neither interested nor qualified to assess the test plans against DoD standards, such as Mil-Std-499. I was told, no problem, assess them against best commercial practice. Interpreting “best commercial practice” as what I would generally recommend doing for a life-critical or otherwise high stakes project in the commercial sector, I created a model for analyzing test plans (now published on my website and also in the appendices of Lessons Learned in Software Testing). I then applied the model to the test plans I was given. What was immediately apparent is that the military test documentation had very little information density. It looked like the 75-page documents had been automatically generated by a set of macros operating on a short bullet list.
I made a bunch of suggestions, including showing how the same information could productively be packaged in 5 pages or so. That way, at least we could wage war on something other than trees. They replied, after two months of silence, that my analysis was not useful to them. Why? Because my ideas about test plan documentation were not consistent with Mil-Std-499. That was the only feedback I received about that work. Way to squander taxpayer money, guys! Hoowaa!
A New Hope
The defense department may be waking up to the problem, at long last. See the NDIA Top Issues Report. Notice that two issues I like to harp about are in the top five challenges: skills and testing. Notice that the military is now officially concerned about wasteful documentation and low-skilled workers.
Maybe not coincidentally, I recently taught my class at Eglin Air Force base, with F-15s thundering regularly overhead. I was surprised that they would invite me to teach there. I was a bit more surprised that they were quite receptive to the concept of skilled and agile software testing, wherein our documentation is produce only because and only to the extent that it actually serves a useful purpose.