초록 열기/닫기 버튼
Korean as a foreign language has experienced significant growth rates in the past few decades due in part to the worldwide popularity of Korean pop and entertainment culture, as well as its strengthening economic status. This new demand has led to a significant increase in the number of non-native Korean-speaking teachers. Despite their competence, many of these instructors report gaps in their confidence and abilities when it comes to evaluating students. Rater biases stemming from features present or absent in their linguistic and cultural backgrounds are of particular concern and may be more prevalent in assessments or oral proficiency. Despite the fact that rater bias in oral assessment has been explored extensively in the English-as-a-Foreign-Language context, there exists no relevant study of rater bias on speaking assessments in the Korean-as-a-foreign-language context. This paper reports the findings of an empirical study, which investigated possible systematic differences in evaluation of native Korean-speaking and non-native raters on speaking samples of learners of Korean and the effects of devices commonly used to enhance inter-rater reliability, such as detailed rubrics and rater training. The data were derived from four native and five non-native teachers of Korean, who were asked to evaluate five speech samples using both holistic and analytic rating scales. In the first round, the participants evaluated the samples without any rubrics. After being provided with detailed rubrics and rater training, the participants again engaged in the second round of rating using the same speech samples. The results, analyzed by many-facet Rasch measurement, revealed that the rating patterns of the two rater groups were significantly different in terms of severity, especially for holistic rating and accuracy and range features for analytic rating, while both groups maintained the acceptable level of rating consistency. The analysis also showed that rating criteria and rater training substantially settled the difference in rating severity between the two rater groups. The paper concludes with implications of the study on NNS raters’ rating pattern and future directions for rater training for Korean oral proficiency test.
