Application exercises

A. Examine a language program that you are familiar with. What types of decisions are made or should be made in the program with Criterion-referenced tests? What approaches to validity would you use to defend the validity of these tests? Content validity? Construct validity? More to the point, how would you use logical arguments? Expert judgments? Intervention studies? Differential-groups studies? Hierarchical-structure studies? Which would you use? And, how would you set them up?

B. How would Messick’s (1988) notions of the evidential and consequential bases of test interpretation and use be important and useful in thinking about the validity of the tests in question A? What about Cronbach’s ideas about the functional, political, economic, and explanatory perspectives on testing?

C. Once again, look at the language program that you examined in question A above. What types of decisions are made or should be made in the program (both norm-referenced and Criterion-referenced)? Which standards setting methods discussed in this chapter would you use to set each of the cut-points for those decisions? Sketch an outline for the setting of all standards. Be sure to include information about how the standards will be set, who will do it, and how often the standards should be revised.

D. Look at the syllabus for a particular course in some language program with which you are familiar.

1. What decisions need to be made in that course on the basis of Criterion-referenced tests?

2. How would you decide the cut-points for any such decisions?

3. How do you think those cut-points will affect the dependability and validity of the tests?

4. How would could you use item analysis to improve the dependability and validity of the tests?