Extensions and applications of deep probabilistic inference for generative models
Abstract/Contents
- Abstract
- Despite the growth of data size, many applications for which we would like to apply learning algorithms to are limited by data quantity and quality. Generative models propose a framework to naturally combine prior beliefs with real world data. Core to the generative approach is the challenge of probabilistic inference, or estimating latent variables given observations. This challenge has led to a rich field of research spanning many statistical techniques. More recently, deep learning methods have been used to solve inference queries, aptly named deep inference. In my dissertation I will explore extensions to deep inference in response to real world challenges of sparsity and efficiency. I will present case studies of practical applications where deep inference achieves considerable improvements upon prior work. This dissertation is centered around three parts. We present the background for generative models and deep inference with an emphasis on modern variational methods. The first part will present new algorithms for generalizing inference to be robust to different notions of sparsity, such as multimodal data, missing data, or computational constraints. Second, we study meta-amortized inference, or "inferring how to infer". A doubly-amortized inference algorithm would be cheaply able to solve inference queries for a novel generative model. We will show a new algorithm to re-purpose masked language modeling to do just this. Third, we present two real-world applications of deep inference in education: (a) estimating student abilities under item response theory and related psychometric models, and (b) inferring educational feedback for students learning to solve programming questions. Together, these contributions showcase the richness and utility of deep inference in education, and more broadly in real world contexts.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2022; ©2022 |
| Publication date | 2022; 2022 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Wu, Mike |
|---|---|
| Degree supervisor | Goodman, Noah (Noah D.) |
| Thesis advisor | Goodman, Noah (Noah D.) |
| Thesis advisor | Ermon, Stefano |
| Thesis advisor | Finn, Chelsea |
| Degree committee member | Ermon, Stefano |
| Degree committee member | Finn, Chelsea |
| Associated with | Stanford University, Computer Science Department |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Mike Hongfei Wu. |
|---|---|
| Note | Submitted to the Computer Science Department. |
| Thesis | Thesis Ph.D. Stanford University 2022. |
| Location | https://purl.stanford.edu/yg258mp8049 |
Access conditions
- Copyright
- © 2022 by Mike Wu
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...