Psychometric properties of the clinical practice and reasoning assessment

This research had the following two objectives: to describe the development and administration processes of the Clinical Practice and Reasoning Assessment (CPRA), an extended, performance-based examination of clinical practice and reasoning skills of medical students; to determine the extent to which the assessment yields data that are psychometrically reliable and valid. To meet the first objective, the development and administration of the CPRA was described in detail. To meet the second, the CPRA was administered to 126 third year medical students at the University of Virginia. The CPRA combined eight standardized patient encounters followed by a unique clinical reasoning post-encounter exercise, developed to measure the student's ability to synthesize the information gained during the patient encounter in formulating a diagnosis, selecting and interpreting appropriate tests, and proposing efficacious management.

Guided by 14 research questions, multiple methods of analyses were conducted to document the reliability, and ecological, content, criterion, and construct validity of the CPRA. Overall patient encounter checklist reliability was moderate (generalizability coefficient= .69) and individual skill component reliability ranged from .56 to .70. Faculty and resident evaluations strongly supported the ecological and content validity of the CPRA. Evidence for predictive validity was mixed — the correlation between the clinical reasoning exercise scores and the United States Medical Licensing Exam, Step II, was significant (r= .48, p = < .05); however, the correlation between the overall patient encounters and Step II was not significant (r = .11, p =ns). Evidence for construct validity was compelling in that 85% of the predictions made about the relationships between the CPRA skill components and external criteria were accurate. Although room for improvement is evident, the accumulation of evidence supports the merit of the CPRA as a valid and reliable assessment tool.

Note: Abstract extracted from PDF file via OCR.