Modular and compositional learning for natural language understanding : implications of the transfer-interference trade-off
Abstract/Contents
- Abstract
- Core to human cognition are the lifelong abilities, developed continuously and effortlessly, to learn new expressions from a small number of examples, adapt to new types of input, and generalize creatively. However, current machine learning algorithms struggle to adapt to novel inputs while retaining existing memories, often resulting in a lack of generalization. In this thesis, this challenge is addressed by introducing a novel characterization of the problem as a trade-off between two effects: on one hand, the ability to transfer knowledge from previously learned skills to the skill in the process of being learned; on the other, the effect of interference where previously learned skills are forgotten when acquiring new ones. Using the trade-off between transfer and interference as a guide, we introduce novel deep reinforcement learning architectures involving memory and routing networks that model cognitive modularity by specializing parts of their memory to different inputs. These new architectures can learn in more realistic scenarios when combined with novel optimization techniques, and can learn end-to-end with a modular and compositional inductive bias well suited to Natural Language Processing applications. These models can be flexibly incorporated in a variety of existing architectures, and can effectively leverage external information about the examples to be successful at fine-grained semantic tasks.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2020; ©2020 |
| Publication date | 2020; 2020 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Cases Martin, Juan Ignacio |
|---|---|
| Degree supervisor | Jurafsky, Dan, 1962- |
| Degree supervisor | Potts, Christopher, 1977- |
| Thesis advisor | Jurafsky, Dan, 1962- |
| Thesis advisor | Potts, Christopher, 1977- |
| Thesis advisor | Greene, Joshua |
| Thesis advisor | Lassiter, Daniel |
| Degree committee member | Greene, Joshua |
| Degree committee member | Lassiter, Daniel |
| Associated with | Stanford University, Department of Linguistics. |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Juan Ignacio Cases Martin. |
|---|---|
| Note | Submitted to the Department of Linguistics. |
| Thesis | Thesis Ph.D. Stanford University 2020. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2020 by Juan Ignacio Cases Martin
Also listed in
Loading usage metrics...