Statistical inference of properties of distributions
Abstract/Contents
- Abstract
- Modern data science applications---ranging from graphical model learning to image registration to inference of gene regulatory networks---frequently involve pipelines of exploratory analysis requiring accurate inference of a property of the distribution governing the data rather than the distribution itself. Notable examples of properties include mutual information, Kullback--Leibler divergence, total variation distance, the entropy rate, among others. This thesis makes contributions to the performance, structure, and deployment of minimax rate-optimal estimators for a large variety of properties in high-dimensional and nonparametric settings. We present general methods for constructing information theoretically near-optimal estimators, and identify the corresponding limits in terms of the parameter dimension, the mixing rate (for processes with memory), and smoothness of the underlying density (in the nonparametric setting). We employ our schemes on the Google 1 Billion Word Dataset to estimate the fundamental limit of perplexity in language modeling, and to improve graphical model learning. The estimators are efficiently computable and exhibit a ``sample size enlargement'' phenomenon, i.e., they attain with $n$ samples what prior methods would have needed $n\log n$ samples to achieve. We provide a brief survey on our utilization and development of relevant tools from approximation theory, probability theory, and functional analysis.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2018; ©2018 |
| Publication date | 2018; 2018 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Jiao, Jiantao |
|---|---|
| Degree supervisor | Weissman, Tsachy |
| Thesis advisor | Weissman, Tsachy |
| Thesis advisor | Montanari, Andrea |
| Thesis advisor | Tse, David |
| Degree committee member | Montanari, Andrea |
| Degree committee member | Tse, David |
| Associated with | Stanford University, Department of Electrical Engineering. |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Jiantao Jiao. |
|---|---|
| Note | Submitted to the Department of Electrical Engineering. |
| Thesis | Thesis Ph.D. Stanford University 2018. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2018 by Jiantao Jiao
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...