Generating structures by editing prototypes
Abstract/Contents
- Abstract
- Methods for structured prediction underlie many successful applications of machine learning, including machine translation, speech synthesis, image generation, protein structure prediction and many other problems. However, producing high-quality structures is challenging because the individual components of a structure (e.g., the words in a sentence or the pixels in an image) constrain and depend on each other in complex ways. A wide variety of approaches have been proposed to tackle this problem. At one extreme, retrieval-based methods sidestep the difficulty of modeling the internal consistency of structures by simply selecting from a pre-populated repository of ``good'' structures. However, this may sacrifice flexibility and coverage, as the repository could fail to contain every possible structure that might be required. At the other extreme, generation-based methods synthesize structures from scratch, often building them up one unit at a time. This offers extreme flexibility, but faces the difficult problem of modeling the complex dependencies that exist between units, as well as the challenge of searching over all possible structures for good configurations. We aim to combine the best of both worlds using a new approach called retrieve-then-edit: first, a structure is retrieved from a repository of high quality candidates (as in retrieval-based methods), and then it is edited into a new structure using a synthesis-based editor. The retrieval step ensures that we start in the neighborhood of an already high quality structure, while the editing step gives us the flexibility to customize the structure in a fine-grained way. We demonstrate that this approach can be successfully applied across diverse structured prediction problems spanning text generation, executable code generation, and reinforcement learning to interact with a web browser (where an agent's sequence of actions is treated as a structure).
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2018; ©2018 |
| Publication date | 2018; 2018 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Guu, Kelvin |
|---|---|
| Degree supervisor | Liang, Percy |
| Thesis advisor | Liang, Percy |
| Thesis advisor | Mackey, Lester |
| Thesis advisor | Manning, Christopher D |
| Thesis advisor | Wong, Wing Hung |
| Degree committee member | Mackey, Lester |
| Degree committee member | Manning, Christopher D |
| Degree committee member | Wong, Wing Hung |
| Associated with | Stanford University, Department of Statistics. |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Kelvin Guu. |
|---|---|
| Note | Submitted to the Department of Statistics. |
| Thesis | Thesis Ph.D. Stanford University 2018. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2018 by Kelvin Gu
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...