This is a tutorial on creating multiple choice questions, framed by Haladyna’s heuristics for test design and Anderson & Krathwohl’s update to Bloom’s taxonomy. My interest in computer-gradable test questions is to support teaching and learning rather than high-stakes examination. Some of the design heuristics are probably different for this case. For example, which is the more desirable attribute for a test question:

1. defensibility (you can defend its fairness and appropriateness to a critic) or
2. potential to help a student gain insight?

In high-stakes exams, (a) [defensibility] is clearly more important, but as a support for learning, I’d rather have (b) [support for insight].

This tutorial’s examples are from software engineering, but from my perspective as someone who has also taught psychology and law, I think the ideas are applicable across many disciplines.

The tutorial’s advice and examples specifically target three projects:

* In the Black Box Software Testing Course [some course materials here], students take the multiple choice tests while they watch the video lectures or work through the assigned readings [research description here].
* We are following the same structure for learning units for graduate student instruction in software engineering ethics.
* In the Open Certification Project for Software Testing we are creating a public database of questions, with peer commentary/criticism. Anyone can review the questions, including people preparing for the exam. For the rationale behind this approach, see this paper by Kaner and Tim Coulter.

This year’s Workshop on Teaching Software Testing (WTST) will be January 18-20 in Melbourne, Florida.
WTST is concerned with the practical aspects of teaching university-caliber software testing courses to academic or commercial students.
This year, we are particularly interested in teaching testing online. How can we help students develop testing skills and foster higher-order thinking in online [...]

Summary: With some new NSF funding, we are researching and revising BBST to make it more available and more useful to more people around the world. The course materials will continue to be available for free. If you are interesting in joining an advisory board that helps us set direction for the course and the [...]