Probabilistic models of pragmatics for natural language
Abstract/Contents
- Abstract
- Grice (1975) puts forward a view of linguistic meaning in which conversational agents enrich the semantic interpretation of linguistic expressions by recourse to pragmatic reasoning about their interlocutors and world knowledge. As a simple example, on hearing my friend tell me that she read some of War and Peace, I reason that, had she read all of it, she would have said as much, and accordingly that she read only part. It turns out that this perspective is well suited to a probabilistic formalization. In these terms, linguistic meaning is fully characterized by a joint probability distribution P(W; U) between states of the world W and linguistic expressions U. The Gricean perspective described above corresponds to a factoring of this enormously complex distribution into a semantics [[u]](w) : U -> (W -> {0, 1}, world knowledge P(W) and a pair of agents which reason about each other on the assumption that both are cooperative and have access to a commonly known semantics. This third component, of back and forth reasoning between agents, originates in work in game-theory (Franke, 2009; Lewis, 1969) and has been formalized in probabilistic terms by a class of models often collectively referred to as the Rational Speech Acts (RSA) framework (Frank and Goodman, 2012). By allowing for the construction of models which explain in precise terms how Gricean pressures like informativity and relevance interact with a semantics, this framework allows us to take an intuitive theory and explore its predictions beyond the limits of intuition. But it should be more than a theoretical tool. To the extent that its characterization of meaning is correct, it should allow for the construction of computational systems capable of reproducing the dynamics of opendomain natural language. For instance, on the assumption that humans produce language pragmatically, one would expect systems which generate natural language to most faithfully reproduce human behavior when aiming to be not only truthful, but also informative to a hypothetical interlocutor. Likewise, systems which interpret language in a human-like way should perform best when they model language as being generated by an informative speaker. Despite this, standard approaches to many natural language processing (NLP) tasks, like image captioning (Farhadi et al., 2010; Vinyals et al., 2015), translation (Brown et al., 1990; Bahdanau et al., 2014) and metaphor interpretation (Shutova et al., 2013), only incorporate pragmatic reasoning implicitly (in the sense that a supervised model trained on human data may learn to replicate pragmatic behavior). The approach of this dissertation is to take models which capture dynamics of pragmatic language use and apply them to open-domain settings. In this respect, my work builds on research in this vein for referential expression generation (Monroe and Potts, 2015; Andreas and Klein, 2016a), image captioning (Vedantam et al., 2017) and instruction following (Fried et al., 2017), as well as work using neural networks as generative models in Bayesian cognitive architectures (Wu et al., 2015; Liu et al., 2018). The content of the dissertation divides into two parts. The first (chapter 2) focuses on the interpretation of language (particularly non-literal language) using a model of non-literal language previously applied to hyperbole and metaphor interpretation in a setting with a hand-specified and idealized semantics. Here, the goal is to instantiate the same model, but with a semantics derived from a vector space model of word meaning. In this setting, the model remains unchanged, but states are points in an abstract word embedding space - a central computational linguistic representation of meaning (Mikolov et al., 2013; Pennington et al., 2014). The core idea here is that points in the space can be viewed as a continuous analogue of possible worlds, and that linear projections of a vector space are a natural way to represent the aspect of the world that is relevant in a conversation. The second part of the dissertation (chapters 3 and 4) focuses on the production of language, in settings where the length of utterances (and consequently the set of all possible utterances) is unbounded. The core idea here is that pragmatic reasoning can take place incrementally, that is, midway through the saying or hearing of an utterance. This incremental approach is applied to neural language generation tasks, producing informative image captions and translations. The result of these investigations is far from a complete picture, but nevertheless a substantial step towards Bayesian models of semantics and pragmatics which can handle the full richness of natural language, and by doing so provide both explanatory models of meaning and computational systems for producing and interpreting language
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2020; ©2020 |
| Publication date | 2020; 2020 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Cohn-Gordon, Reuben Harry |
|---|---|
| Degree supervisor | Potts, Christopher, 1977- |
| Thesis advisor | Potts, Christopher, 1977- |
| Thesis advisor | Bergen, Leon |
| Thesis advisor | Jurafsky, Dan, 1962- |
| Thesis advisor | Lassiter, Daniel |
| Degree committee member | Bergen, Leon |
| Degree committee member | Jurafsky, Dan, 1962- |
| Degree committee member | Lassiter, Daniel |
| Associated with | Stanford University, Department of Linguistics. |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Reuben Cohn-Gordon |
|---|---|
| Note | Submitted to the Department of Linguistics |
| Thesis | Thesis Ph.D. Stanford University 2020 |
| Location | electronic resource |
Access conditions
- Copyright
- © 2020 by Reuben Harry Cohn-Gordon
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...