| Status | 已發表Published |
| Variance-Aware Machine Translation Test Sets | |
Zhan, R. ; Liu, X.; Wong, F.  ; Chao, L.
| |
| 2021-12-06 | |
| Source Publication | Thirty-fifth Conference on Neural Information Processing Systems
 |
| Abstract | We release 70 small and discriminative test sets for machine translation (MT)evaluation called variance-aware test sets(VAT), covering 35 translation directions from WMT16 to WMT20 competitions. VAT is automatically created by a novel variance-aware filtering method that filters the indiscriminative test in-stances of the current MT test sets without any human labor. Experimental results show that VAT outperforms the original WMT test sets in terms of the correlation with human judgment across mainstream language pairs and test sets. Further analysis on the properties of VAT reveals the challenging linguistic features(e.g., translation of low-frequency words and proper nouns) for competitive MTsystems, providing guidance for constructing future MT test sets. The test sets and the code for preparing variance-aware MT test sets are freely available athttps://github.com/NLP2CT/Variance-Aware-MT-Test-Sets. |
| Keyword | Neural Machine Translation Evaluation |
| Language | 英語English |
| The Source to Article | PB_Publication |
| PUB ID | 58019 |
| Document Type | Conference paper |
| Collection | DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
| Corresponding Author | Wong, F. |
| Recommended Citation GB/T 7714 | Zhan, R.,Liu, X.,Wong, F.,et al. Variance-Aware Machine Translation Test Sets[C], 2021. |
| APA | Zhan, R.., Liu, X.., Wong, F.., & Chao, L. (2021). Variance-Aware Machine Translation Test Sets. Thirty-fifth Conference on Neural Information Processing Systems. |
| Files in This Item: | There are no files associated with this item. | |||||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment