Multi-Turn Generation-Based Conversational Agents in Open Domains
Doctoral thesis
Permanent lenke
https://hdl.handle.net/11250/3121195Utgivelsesdato
2024Metadata
Vis full innførselSamlinger
Sammendrag
This PhD thesis focuses on open-domain generation-based conversational agents, which are chatbots that generate responses to any input or question using natural language processing and deep learning techniques. The thesis identifies three major challenges faced by these conversational agents.
(1) Generating appropriate responses for a wide range of topics and domains. Current studies have focused on single-corpus training, which limits the model's ability to generate relevant responses for certain topics.
(2) Improving a model's performance of context attention distribution in multi-turn settings. The ability to distribute attention and assign importance to relevant information is necessary to generate appropriate responses. However, most existing works have treated multi-turn conversations as one-turn contexts, limiting the performance of the agents.
(3) Integrating knowledge under the conversational question-answering task perspective. There is a gap in research on integrating extractive question-answering techniques with instruction-based tuning and prompt-based tuning. The thesis proposes several approaches to address these challenges.
For (1), the thesis proposes Document-specific Frequency (DF) as an evaluation metric and proposes several methods for balancing multiple corpora. The best method, which integrates DF with the training, achieves an improvement by 34.1% on F1 performance and at least 20.0% on DF. A thorough human evaluation shows a highly significant (p < 0.001) improvement in all of our proposed methods.
For (2), the thesis proposes Distracting Attention Score ratio (DAS ratio) as an evaluation metric and employs self-contained negative samples and summarization techniques to improve a system's performance on context attention distribution. The proposed self-contained negative samples are applied as a training strategy, resulting in about 10% better DAS ratio. The best summarization technique setting with ORACLE gains a 23% improvement on the DAS ratio.
For (3), the thesis explores various settings of integrating extractive question answering with instruction-based tuning, prompt-based tuning, and multi-task learning. When combining prompt-based tuning with either instruction-based tuning or multi-task learning, the F1 performance is improved by about 18% over the baseline.
Together, these techniques have improved the overall performance of multi-turn conversational agents on open domains.
Består av
Paper 1: Xing, Yujie; Cai, Jinglun; Barlaug, Nils; Liu, Peng; Gulla, Jon Atle. Balancing Multi-Domain Corpora Learning for Open-Domain Response Generation. Findings of the Association for Computational Linguistics: NAACL 2022Paper 2: Xing, Yujie; Gulla, Jon Atle. Evaluating and Improving Context Attention Distribution on Multi-Turn Response Generation using Self-Contained Distractions. arXiv:2211.04943[cs.CL]
Paper 3: Xing, Yujie; Gulla, Jon Atle. Improving Context-Awareness on Multi-Turn Dialogue Modeling with Extractive Summarization Techniques. Lecture Notes in Computer Science (LNCS) 2023 ;Volum 13913. s. 478-488 https://doi.org/10.1007/978-3-031-35320-8_35
Paper 4: Xing, Yujie; Liu, Peng. Prompt and instruction-based tuning for response generation in conversational quesition answering. Lecture Notes in Computer Science (LNCS) 2023 ;Volum 13913. s. 156-169 https://doi.org/10.1007/978-3-031-35320-8_11