Rating accuracy and cognitive load associated with the distributional assessment model
Date
2011
Authors
Vanhove, Adam, author
Gibbons, Alyssa Mitchell, advisor
Clegg, Benjamin A., committee member
Henle, Chris A., committee member
Thornton, George C. III, committee member
Journal Title
Journal ISSN
Volume Title
Abstract
This study examined both the interrater agreement and true score accuracy associated two different types of response formats, one using the traditional assessment (TA) approach, and the other using the distributional assessment (DA) approach. In addition, proponents of the DA response format have proposed that DA users experience less cognitive load than TA users (e.g., Kane, 2000), however, this has not been empirically examined until now. Findings suggest 1) greater interrater agreement among DA users, 2) higher true score accuracy for DA users despite minimal practical significance, and 3) DA users actually experienced significantly more cognitive load than TA users. Finally, a mediational hypothesis was tested to examine whether response format led to experienced cognitive load which, in turn, led to differences in true score accuracy. No evidence was found for this mediational hypothesis.
Description
Rights Access
Subject
performance appraisal
mediation
rater accuracy
cognitive load
distributional assessment