Technical Quality of Alternative Assessments

Because of the major differences between traditional and alternative assessments, some educators argue that we must rethink the technical quality criteria used to judge the quality of alternative assessments. Linn, Baker, and Dunbar (1991) suggest the following technical quality criteria:

Consequences. Does the assessment have a positive impact on educational practice?
Fairness. Do performance ratings reflect the examinee's true capabilities rather than the perceptions and biases of the persons evaluating the performance?
Transfer and Generalizability. Does performance on the specific performance tasks in the alternative assessment generalize to the broader domain of achievement from which the tasks are sampled?
Cognitive Complexity. Does the task require problem solving, critical thinking, comprehension, reasoning, and metacognitive processes, or can the student perform well simply by memorizing a process without understanding the underlying concept?
Content Quality. Is the content consistent with the best current understanding of the field and at the same reflect what are judged to be aspects of quality that will stand the test of time?
Content Coverage. Does the assessment adequately sample the breadth of possible important content?
Meaningfulness. Does the assessment give students the opportunity to deal with meaningful problems that provide worthwhile educational experiences?
Cost and Efficiency. Is the cost reasonable and are data collection designs and scoring procedures efficient?

"In summary, serious validation of alternative assessments needs to include evidence regarding the intended and unintended consequences, the degree to which performance on specific assessment tasks transfers, and the fairness of the assessments. Evidence is also needed regarding the cognitive complexity of the processes students employ in solving assessment problems and the meaningfulness of the problems for students and teachers. In addition, a basis for judging both the content quality and the comprehensiveness of the content coverage needs to be provided. Finally, the cost of the assessment must be justified." (Linn, Baker, & Dunbar, 1991, p. 20)

References.