초록 열기/닫기 버튼
.
This study aims to determine whether generative AI has grammatical skills at the level of native Korean speakers. For this purpose, a grammatical understanding ability test and a production ability test were conducted. In Chapter 2, we reviewed previous research related to language ability evaluation of generative AI and presented an evaluation method. In Chapter 3, the grammatical ability of generative AI for ‘endings’ and ‘negative sentences’ was tested in terms of understanding and production. This work could improve the performance of domestic and foreign language models. This is because it summarizes areas where Korean language study can contribute, such as fine-tuning specific errors in terms of Korean grammatical ability. And conversely, it can be said that the possibility of using generative AI services in Korean language education and research was also discussed. The results of this study can be summarized into the following three points. First, these models generally showed a much higher percentage of correct answers in the grammaticality judgment test than in the correction. This is natural, assuming that generative AI is a Korean language learner. This provides an important clue to show that the current level of Korean language proficiency of generative AI has a long way to go and to gauge the direction for raising the level of Korean proficiency of generative AI to that of a native language speaker. Second, at least for the questions evaluated in this study, CLOVA X, developed by Naver, a domestic company, showed the highest accuracy rate. However, even CLOVA X had a very low accuracy rate in evaluating grammatical production ability for endings. This will be an important clue in assessing and improving the level of Korean language proficiency of current generative AI. Third, although ChatGPT 4.0 is the only paid version among generative AIs, it is overall superior to ChatGPT 3.5 in evaluating Korean grammar skills, but the difference is not noticeable.
키워드열기/닫기 버튼
generative AI, grammaticality, endings, negative sentences, grammatical competence, understanding ability, production ability
