The Next Step In “Test Automation” is Pure Bullshitting

I defy any responsible, sober technical professional to visit this website and discover what the “MABL” tool is and does without reaching out to the company to beg for actual details. It has an introduction video, for instance, that conveys no information whatsoever about the product. Yes, it is teeming with sentences that definitely contain words. But the words representing irresponsible, hyperbolic summarizing that could be applied, equally irresponsibly and hyperbolically to lots of different tools.

My favorite moments in the video:

0:33 “write tests… just like a really smart QA engineer would.” Huh. I would like to see a QA engineer go on the video and say “I’m really smart QA engineer, and MABL does just what I do.” I would like to interview such a person.

0:44 “She uses machine intelligence to…” Yes, the talking man is using the female pronoun to imply that “MABL” has the tacit knowledge of a female human engineer. Isn’t that nice? He speaks with a straight face and an even tone. He must have a lot of respect for this imaginary woman he is marketing. (Note: no human women speak on the video, but there is one in a non-speaking role for about a half-second.)

Ultimately, I am left not knowing what specific functionalities their tool has that they are lying about. Yes, lying. Because their claims cannot possibly be true, and they cannot possibly believe they are true– kind of like one of those infomercials about 18-year-old girls in your area that would love to talk to you. Except in this case, her name is MABL and she wants to test your product.

What is really going on?

Apparently the industry has reached a point where testing services can be sold the same way miracle weight loss programs or anti-aging face creams (with micro-beads!) are sold. This can only happen in an industry that holds testing craftsmanship in utter contempt. The testing industry is like a failed state ruled by roving gangs.

Maybe this MABL tool does something interesting, but it seems they don’t want us to worry our pretty little heads about it. And that is something that should worry us all.

My War: Agency vs. Algorithm

In a recent Twitter conversation, yet again someone who should know better claimed that testing improves quality, and yet again Michael Bolton and I rose to speak against that notion. Our correspondent wants to reduce it to a simple matter of probability:

My answer to this question is to reject it. It rests on certain false premises:

  • That we sufficiently agree about what quality is.
  • That we sufficiently agree about what testing is.
  • That we agree on what is typical.
  • That we agree that discussing human choices in terms of brute causality is helpful in this context.

I’m not saying that questions like this are necessarily bad. I’m sure I have used a similar construction in some other argument. The problem here is that the writer of the Tweet arrived at this simplifying formula after dismissing, as irrelevant, my social and ethical objections to his entire line of reasoning.

I say that testing does not improve quality, first, because it is obviously true that learning facts about code does not change code. Second, and much more importantly, I say it because it is both practical and ethical for testers to respect the agency of the people who control the code. Those people may not fix the bugs I report. It is critical to my discipline as a tester to understand that. Otherwise, I risk losing my credibility and influence. I risk adopting an attitude that desensitizes me to the kinds of problems my clients need me to find.

Agency. It’s dawning on me that all of my projects are connected by this thread.

  • I am opposed to “best practices” because that phrase is just a ploy to avoid responsibility for decisions about how to work.
  • I am opposed to professional certification programs, such as the ISTQB, because it is just a ploy to profit from the fear and ignorance of testers and managers; insidious manipulation via snake oil.
  • I am opposed to “standards” that are crafted by consulting companies to justify expensive, useless services.
  • I don’t use the phrase “test automation” because that encourages thinking about testing as a set of mechanical actions instead of a set of choices, interpretations, and explorations.
  • I insist on distinguishing testing from mere fact checking. Checking can be done by machines that make no choices, but testing requires socially competent judgments and a whole raft of choices which must be made and to which testers must be accountable.
  • I am opposed to self-driving cars not because they are unsafe, but because they aim to change without deliberation or consent the social contract that humans have with each other about the uses of public roads and accountability for what happens on them.
  • Although I am loud and opinionated, I tell my students that my goal is to turn them into colleagues, not followers. People who parrot what I say are no use to me.
  • My son was born at home, because my wife wanted control over the entire birth process and was not satisfied that the hospital would respect that. So, I read a few books on midwifery, hired illegal midwives to help, and made plans in case of emergency. (To my surprise, I found research which showed that home births were safer for low-risk pregnancies.)
  • I could not tolerate school. I was assigned tasks without my consent.
  • I am opposed to forced schooling, because good education is a personal journey of self development. (My son dropped out after 6th grade.)
  • The biggest conflict of my marriage centered around money. I solved it (after a few years to get my pride under control) by giving my wife control of our finances and giving her 70% of the company I started. I got complete creative freedom and she has cheerfully been my assistant for 19 years and counting. Lesson: giving away one kind of power can give you back another kind of power.

Agency is the capacity of an actor to act; the ability to make choices. I have a lot of intellectual attitudes and arguments about testing. But my emotion and motivation about testing comes from my feelings about agency.

I am a tester because I want to set people free. I am a teacher because I want to set people free. I am a husband and father because I want to keep my family free.

Fight the algorithm.