This paper aimed to find what English teachers at secondary schools in Korea did during their rating assessment and to suggest what effective rater training session/programme for them should be like. Three English teachers at various secondary schools participated as raters for this study, and they were asked to rate six pieces of writing samples written by secondary school students, using FCE rating scale for writing assessment. They were also supposed to do think-aloud while rating so as to reveal their rating behaviour, and then their protocols recorded on cassette tapes were transcribed. The results showed that; the raters did not prevent themselves from relying on their subjective criteria; they failed to stick to the scale throughout their scoring; they were likely to rate a sample, comparing with preceeding samples; they often had difficulties in understanding some descriptors within the scale; the very first impression which a sample made on a rater often affected on his/her rating; they sometimes showed halo effect in their rating. From these findings it can be suggested that rater training session or programme for English teachers should be devised, reflecting their rating behaviour and help them improve test validity, and further intra- and inter-rater reliability.

This paper aimed to find what English teachers at secondary schools in Korea did during their rating assessment and to suggest what effective rater training session/programme for them should be like. Three English teachers at various secondary schools participated as raters for this study, and they were asked to rate six pieces of writing samples written by secondary school students, using FCE rating scale for writing assessment. They were also supposed to do think-aloud while rating so as to reveal their rating behaviour, and then their protocols recorded on cassette tapes were transcribed. The results showed that; the raters did not prevent themselves from relying on their subjective criteria; they failed to stick to the scale throughout their scoring; they were likely to rate a sample, comparing with preceeding samples; they often had difficulties in understanding some descriptors within the scale; the very first impression which a sample made on a rater often affected on his/her rating; they sometimes showed halo effect in their rating. From these findings it can be suggested that rater training session or programme for English teachers should be devised, reflecting their rating behaviour and help them improve test validity, and further intra- and inter-rater reliability.

쓰기 평가, 채점, 채점자 훈련