Educational design in general is a largely underestimated and unexplored area of design, and its relationship with educational research can be characterized as somewhat less than satisfying. The design of assessments is often seen as an afterthought. And it shows.

Of course, there is a wealth of publications on assessment, but quite often these focus on psychometric concerns or preparation for high-stakes tests (a very profitable industry). What is lacking is a tight linkage between research findings and the creation of mathematically rich and revealing tasks for productive classroom use. The report “Inside the black box” [Black andWiliam 1998], which looks in depth at current research, shows clearly that we should not only invest more in classroom assessment in mathematics, but also that the rewards will be high if we do so.

To provoke some discussion, and to invite the reader to reflect on the items and make a judgment, I start the examples with an item from the Third International Mathematics and Science Study (TIMSS). According to the TIMSS web site http://www.timss.org, TIMSS was “the largest and most ambitious international study of student achievement ever conducted. In 1994–95, it was conducted at five grade levels in more than 40 countries (the third, fourth, seventh, and eighth grades, and the final year of secondary school).” Mathematics coverage on TIMSS was, in essence, international consensus coverage of the traditional curriculum.

The item in question, given to students about 14 years old in the mid-1990s, involved a simplified portion of a regional road map, replicated on the next page. The scale is reinforced by a sentence above the map:

One centimeter on the map represents 8 kilometers on the land.

The problem seems simple, actually too simple, for the intended age group. The multiple-choice format makes it even easier for the students to find the correct answer, 35 km. There is nothing wrong with easy questions as such; the surprise lies in the “intended performance category,” which is “using complex procedures.” The consequence of this classification is that students who have chosen answer C are considered able to use complex mathematical procedures. This seems somewhat far-fetched.