Repository logo

Rating accuracy and cognitive load associated with the distributional assessment model




Vanhove, Adam, author
Gibbons, Alyssa Mitchell, advisor
Clegg, Benjamin A., committee member
Henle, Chris A., committee member
Thornton, George C. III, committee member

Journal Title

Journal ISSN

Volume Title


This study examined both the interrater agreement and true score accuracy associated two different types of response formats, one using the traditional assessment (TA) approach, and the other using the distributional assessment (DA) approach. In addition, proponents of the DA response format have proposed that DA users experience less cognitive load than TA users (e.g., Kane, 2000), however, this has not been empirically examined until now. Findings suggest 1) greater interrater agreement among DA users, 2) higher true score accuracy for DA users despite minimal practical significance, and 3) DA users actually experienced significantly more cognitive load than TA users. Finally, a mediational hypothesis was tested to examine whether response format led to experienced cognitive load which, in turn, led to differences in true score accuracy. No evidence was found for this mediational hypothesis.


Rights Access


performance appraisal
rater accuracy
cognitive load
distributional assessment


Associated Publications