Recapitulation and introduction

The invention of short multiple-choice test items provided an enormous technical and practical advantage for test developers; certainly, the items could be scored easily, but that was just one of the reasons for their popular adoption in the early part of the 20th century. A more important reason was the potential for an increase in validity because of the speed with which such items could be answered. This meant that a broad range of content specifications could be addressed, and hence an examinee no longer needed to be heavily penalized because of the unfortunate choice of a question requiring a longer-to-answer constructed response. These advantages, as well as many others (see Anastasi, 1976, pp. 415–417), led the multiple-choice format to become, by far, the dominant form used in large-scale standardized mental testing for the past century. Nevertheless, this breakthrough in test construction, dominant at least since the days of Army alpha (1917), is currently being reconsidered.

Critics of tests that are made up of large numbers of short questions suggest that de-contextualized items yield a task that is abstracted too far from the domain of inference for many potential uses, and hence would have low predictive validity for most practical issues. For several reasons, only one of them as a response to this criticism, variations in test theory were considered that would allow the retention of the short-answer format while eliminating the shortcomings expressed by those critics. One of these variations was the development of item response theory (IRT), an analytic breakthrough in test scoring.