earticle

영어

The most recent language model in NLP, known as BERT, has numerous advantages over other language models. It ‘understands’ human language in a subword unit and can recognize rare words and out-of-vocabulary words. Furthermore, it considers syntax and context during the text-understanding process. We applied the evaluation metric “BERTscore” using this BERT model to evaluate human translation (HT). 120 translated texts were evaluated with five evaluation metrics: BLEU, METEOR, emBLEU, emMETEOR, as well as BERTscore and the result was compared with professional translators’ evaluation. The comparison examines the validity and reliability of these metrics, particularly the BERTscore, for future application for HT evaluation. BERTscore demonstrated a stable performance, taking first place in scores, and third in ranks. The validity of metrics of word2vec models, especially that of emBLEU, was somewhat disappointing, probably owing to the domain difference between the training corpus and test corpus.