Neural generation of open-ended text and dialogue
Abstract/Contents
- Abstract
- Advances in Deep Learning have enabled more fluent and flexible Natural Language Generation (NLG). While these neural generative systems achieved early success in machine translation, they encounter problems — such as repetition, incoherence, and uncontrollability — when applied to more open-ended tasks such as abstractive summarization, story generation and chitchat dialogue. Furthermore, open-ended neural generative models tend to be evaluated by crowdworkers in carefully-controlled environments; it is less well-understood how they behave in realistic environments with real-life users. This thesis analyzes and improves neural generative systems performing several open-ended tasks; in the case of dialogue, the systems are evaluated in their full social context. First, for abstractive summarization, I present a pointer-generator model to improve copying accuracy and a coverage mechanism to reduce repetition in the generated summaries. Next, for chitchat dialogue, I present a large-scale detailed human evaluation to reveal the relationship between bot behaviors (such as repetition, specificity, staying on topic, and question-asking) and human quality judgments, and show that by controlling these bot behaviors, we can improve user experience. Third, for story generation, I characterize the effect of large-scale pretraining, and of the decoding algorithm, on several syntactic, semantic, structural, and stylistic aspects of the generated text. Lastly, I present a study of a neural generative chitchat model in deployment as part of the Alexa Prize, talking to real, intrinsically-motivated users. By analysing bot-user interactions, I identify the bot's main error types, and how they relate to user dissatisfaction. Furthermore, I demonstrate a semi-supervised method to learn from dissatisfaction and thus improve the dialogue system.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2021; ©2021 |
| Publication date | 2021; 2021 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | See, Abigail Elizabeth | |
|---|---|---|
| Degree supervisor | Manning, Christopher D | |
| Thesis advisor | Manning, Christopher D | |
| Thesis advisor | Hashimoto, Tatsunori | |
| Thesis advisor | Jurafsky, Dan, 1962- | |
| Degree committee member | Hashimoto, Tatsunori | |
| Degree committee member | Jurafsky, Dan, 1962- | |
| Associated with | Stanford University, Computer Science Department |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Abigail See. |
|---|---|
| Note | Submitted to the Computer Science Department. |
| Thesis | Thesis Ph.D. Stanford University 2021. |
| Location | https://purl.stanford.edu/hw190jq4736 |
Access conditions
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
- Copyright
- © Copyright 2021 by Abigail Elizabeth See
Also listed in
Loading usage metrics...