earticle

영어

“Does Raters’ Rating Experience Influence English-speaking Test Ratings?” Studies in English Language & Literature. 38.3 (2012): 243-263. In second language performance tests, rater effect has been a major concern in that it is a potential source of measurement error. The purpose of the study is to examine if raters' rating experience affects their English-speaking ratings. Two experienced and two less experienced native English-speaking raters scored 116 Korean college test-takers' English-speaking test responses. The data were analyzed with the GENOVA program and independent samples t-tests. The results revealed that both experienced and less experienced raters yielded almost identical dependability indices, showing that they scored with consistency the English-speaking samples within their group. The findings of this study also indicate that there is no significant rater effect on combined scores between experienced and less experienced raters. However, there is a significant rater experience effect on some analytic rating components such as pronunciation. Based on the findings, theoretical and practical implications are discussed. (Mokwon University)