Residential College | false |
Status | 已發表Published |
A Two-Stage Prediction-Aware Contrastive Learning Framework for Multi-Intent NLU | |
Chen, Guanhua; Yao, Yutong; Wong, Derek F.; Chao, Lidia S. | |
2024 | |
Conference Name | The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) |
Source Publication | 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings |
Pages | 1778-1788 |
Conference Date | 20-05-2024 |
Conference Place | Torino |
Country | Italy |
Abstract | Multi-intent natural language understanding (NLU) presents a formidable challenge due to the model confusion arising from multiple intents within a single utterance. While previous works train the model contrastively to increase the margin between different multi-intent labels, they are less suited to the nuances of multi-intent NLU. They ignore the rich information between the shared intents, which is beneficial to constructing a better embedding space, especially in low-data scenarios. We introduce a two-stage Prediction-Aware Contrastive Learning (PACL) framework for multi-intent NLU to harness this valuable knowledge. Our approach capitalizes on shared intent information by integrating word-level pre-training and prediction-aware contrastive fine-tuning. We construct a pre-training dataset using a word-level data augmentation strategy. Subsequently, our framework dynamically assigns roles to instances during contrastive fine-tuning while introducing a prediction-aware contrastive loss to maximize the impact of contrastive learning. We present experimental results and empirical analysis conducted on three widely used datasets, demonstrating that our method surpasses the performance of three prominent baselines on both low-data and full-data scenarios. |
Keyword | Conversational Systems Natural Language Understanding Prediction-aware Contrastive Learning |
URL | View the original |
Language | 英語English |
Scopus ID | 2-s2.0-85195913365 |
Fulltext Access | |
Citation statistics | |
Document Type | Conference paper |
Collection | DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Corresponding Author | Wong, Derek F. |
Affiliation | NLP2CT Lab, Department of Computer and Information Science, University of Macau, Macao |
First Author Affilication | University of Macau |
Corresponding Author Affilication | University of Macau |
Recommended Citation GB/T 7714 | Chen, Guanhua,Yao, Yutong,Wong, Derek F.,et al. A Two-Stage Prediction-Aware Contrastive Learning Framework for Multi-Intent NLU[C], 2024, 1778-1788. |
APA | Chen, Guanhua., Yao, Yutong., Wong, Derek F.., & Chao, Lidia S. (2024). A Two-Stage Prediction-Aware Contrastive Learning Framework for Multi-Intent NLU. 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings, 1778-1788. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment