Methods for scalable Markov chain Monte Carlo inference
Abstract/Contents
- Abstract
- This dissertation contains three methods aiming at improving the scalability of Markov Chain Monte Carlo (MCMC) sampling. The first method addresses the computation bottleneck of Metropolis-Hastings in Bayesian models with a large number of observations. As an extension to the first method, we have developed a new MCMC algorithm to handle multimodal high-dimensional posterior distributions with a large number of observations. Our objective with the third method is to improve the efficiency of MH in high dimension, by designing a novel local MH move to make an effective use of the gradient information of the log target density. In the first part, we propose a general framework of performing MCMC with only a mini-batch of data. We show that by estimating the MH ratio with only a subset of data, one can sample from the true posterior raised to a known temperature. We show by experiments that our algorithm, ``Mini-batch Tempered MCMC'', can efficiently explore the landscape of a multimodal posterior distribution. In addition, based on the Equi-Energy sampler \citep{kou2006discussion}, we propose a new MCMC algorithm, which enables exact sampling from high-dimensional multimodal posteriors with well-separated modes. In the second part, we introduce a novel local MH move which we shall refer to as the ``Cone Move''. Our goal is to address a crucial limitation of Langevin Dynamics, namely its inherent inefficiency in utilizing the gradient information in exploring the target distribution. Compared with Langevin Dynamics, Cone Move makes better use of the gradient information and thus generates more targeted moves along the shape of the target distribution. We propose a new MCMC algorithm, the Cone Sampler, which adapts to the geometry of the target density by alternating between Cone Move and Langevin Dynamics. We demonstrate the effectiveness of Cone Sampler by testing its performance on challenging sampling problems in high dimensions.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2018; ©2018 |
| Publication date | 2018; 2018 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Li, Dangna |
|---|---|
| Degree supervisor | Wong, Wing Hung |
| Thesis advisor | Wong, Wing Hung |
| Thesis advisor | Lai, T. L |
| Thesis advisor | Sabatti, Chiara |
| Degree committee member | Lai, T. L |
| Degree committee member | Sabatti, Chiara |
| Associated with | Stanford University, Department of Computational and Mathematical Engineering. |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Dangna Li. |
|---|---|
| Note | Submitted to the Department of Computational and Mathematical Engineering. |
| Thesis | Thesis Ph.D. Stanford University 2018. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2018 by Dangna Li
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...