Purpose: Standardized patients participate in clinical performance examinations not only to simulate case scenarios but also to evaluate the performance of students using a checklist. The accuracy in checking off checklist items is one of the most important factors determining the reliability of this examination. The purposes of this study were to determine the SP's overall accuracy in recording checklist items, and whether their accuracy was affected by certain characteristics of checklist items. Methods: Three professors, who have been fully involved in scenario development and SP training, reviewed videotapes of the examination and evaluated the performance of the students using the same checklist. SP's checklists were marked on this ‘correct checklist’. The checklists and checklist guidelines of the items marked under the score of 50 out of 100 were analyzed. Results: Results showed that the accuracy of the SP's in recording checklist items was 86.9% and was affected by certain characteristics, such as complexity or ambiguity of checklists and checklist guidelines. Conclusion: In this study, the SP's accuracy in recording checklist items was good to very good, and the result suggested that the accuracy could be improved by the elaboration of checklists and checklist guidelines.

Purpose: Standardized patients participate in clinical performance examinations not only to simulate case scenarios but also to evaluate the performance of students using a checklist. The accuracy in checking off checklist items is one of the most important factors determining the reliability of this examination. The purposes of this study were to determine the SP's overall accuracy in recording checklist items, and whether their accuracy was affected by certain characteristics of checklist items. Methods: Three professors, who have been fully involved in scenario development and SP training, reviewed videotapes of the examination and evaluated the performance of the students using the same checklist. SP's checklists were marked on this ‘correct checklist’. The checklists and checklist guidelines of the items marked under the score of 50 out of 100 were analyzed. Results: Results showed that the accuracy of the SP's in recording checklist items was 86.9% and was affected by certain characteristics, such as complexity or ambiguity of checklists and checklist guidelines. Conclusion: In this study, the SP's accuracy in recording checklist items was good to very good, and the result suggested that the accuracy could be improved by the elaboration of checklists and checklist guidelines.

Clinical performance, Observer variation, Educational measurement