Residential Collegefalse
Status已發表Published
Difficulty-Aware Machine Translation Evaluation
Runzhe Zhan; Xuebo Liu; Derek F. Wong; Lidia S. Chao
2021
Conference NameThe Joint 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL-IJCNLP 2021
Source PublicationACL-IJCNLP 2021 - 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Proceedings of the Conference
Volume2
Pages26-32
Conference DateAUG 01-06, 2021
Conference PlaceVirtual
PublisherAssociation for Computational Linguistics
Abstract

The high-quality translation results produced by machine translation (MT) systems still pose a huge challenge for automatic evaluation. Current MT evaluation pays the same attention to each sentence component, while the questions of real-world examinations (e.g., university examinations) have different difficulties and weightings. In this paper, we propose a novel difficulty-aware MT evaluation metric, expanding the evaluation dimension by taking translation difficulty into consideration. A translation that fails to be predicted by most MT systems will be treated as a difficult one and assigned a large weight in the final score function, and conversely. Experimental results on the WMT19 English$German Metrics shared tasks show that our proposed method outperforms commonly-used MT metrics in terms of human correlation. In particular, our proposed method performs well even when all the MT systems are very competitive, which is when most existing metrics fail to distinguish between them.

DOI10.18653/v1/2021.acl-short.5
URLView the original
Indexed ByCPCI-S ; CPCI-SSH
Language英語English
WOS Research AreaComputer Science ; Linguistics
WOS SubjectComputer Science, Artificial Intelligence ; Computer Science, Interdisciplinary Applications ; Linguistics
WOS IDWOS:000694699200005
Scopus ID2-s2.0-85119568324
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionDEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorDerek F. Wong
AffiliationNLP2CT Lab, Department of Computer and Information Science, University of Macau,
First Author AffilicationUniversity of Macau
Corresponding Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Runzhe Zhan,Xuebo Liu,Derek F. Wong,et al. Difficulty-Aware Machine Translation Evaluation[C]:Association for Computational Linguistics, 2021, 26-32.
APA Runzhe Zhan., Xuebo Liu., Derek F. Wong., & Lidia S. Chao (2021). Difficulty-Aware Machine Translation Evaluation. ACL-IJCNLP 2021 - 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Proceedings of the Conference, 2, 26-32.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Runzhe Zhan]'s Articles
[Xuebo Liu]'s Articles
[Derek F. Wong]'s Articles
Baidu academic
Similar articles in Baidu academic
[Runzhe Zhan]'s Articles
[Xuebo Liu]'s Articles
[Derek F. Wong]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Runzhe Zhan]'s Articles
[Xuebo Liu]'s Articles
[Derek F. Wong]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.