Residential College | false |
Status | 已發表Published |
Difficulty-Aware Machine Translation Evaluation | |
Runzhe Zhan![]() ![]() ![]() ![]() | |
2021 | |
Conference Name | The Joint 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL-IJCNLP 2021 |
Source Publication | ACL-IJCNLP 2021 - 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Proceedings of the Conference
![]() |
Volume | 2 |
Pages | 26-32 |
Conference Date | AUG 01-06, 2021 |
Conference Place | Virtual |
Publisher | Association for Computational Linguistics |
Abstract | The high-quality translation results produced by machine translation (MT) systems still pose a huge challenge for automatic evaluation. Current MT evaluation pays the same attention to each sentence component, while the questions of real-world examinations (e.g., university examinations) have different difficulties and weightings. In this paper, we propose a novel difficulty-aware MT evaluation metric, expanding the evaluation dimension by taking translation difficulty into consideration. A translation that fails to be predicted by most MT systems will be treated as a difficult one and assigned a large weight in the final score function, and conversely. Experimental results on the WMT19 English$German Metrics shared tasks show that our proposed method outperforms commonly-used MT metrics in terms of human correlation. In particular, our proposed method performs well even when all the MT systems are very competitive, which is when most existing metrics fail to distinguish between them. |
DOI | 10.18653/v1/2021.acl-short.5 |
URL | View the original |
Indexed By | CPCI-S ; CPCI-SSH |
Language | 英語English |
WOS Research Area | Computer Science ; Linguistics |
WOS Subject | Computer Science, Artificial Intelligence ; Computer Science, Interdisciplinary Applications ; Linguistics |
WOS ID | WOS:000694699200005 |
Scopus ID | 2-s2.0-85119568324 |
Fulltext Access | |
Citation statistics | |
Document Type | Conference paper |
Collection | DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Corresponding Author | Derek F. Wong |
Affiliation | NLP2CT Lab, Department of Computer and Information Science, University of Macau, |
First Author Affilication | University of Macau |
Corresponding Author Affilication | University of Macau |
Recommended Citation GB/T 7714 | Runzhe Zhan,Xuebo Liu,Derek F. Wong,et al. Difficulty-Aware Machine Translation Evaluation[C]:Association for Computational Linguistics, 2021, 26-32. |
APA | Runzhe Zhan., Xuebo Liu., Derek F. Wong., & Lidia S. Chao (2021). Difficulty-Aware Machine Translation Evaluation. ACL-IJCNLP 2021 - 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Proceedings of the Conference, 2, 26-32. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment