Information theoretic approaches to statistical learning problems
Abstract/Contents
- Abstract
- Machine learning algorithms have achieved super-human scores in several benchmark tasks from various areas such as computer vision, natural language processing, and speech processing. These algorithms, however, have been observed to lack sufficient robustness and generalizability necessary for their safe application to high-risk tasks such as designing self-driving cars and healthcare systems. A state-of-the-art learning algorithm demonstrates great generalization behavior over one particular domain of data, while the same algorithm can generalize poorly in other practical domains or under a slight domain shift such as adding minor additive noise to its input. Resolving such issues and improving robustness and generalization properties of learning algorithms require a thorough theoretical analysis of statistical learning settings, raising the need for novel theoretical approaches to analyze machine learning problems. This doctoral dissertation introduces several new theoretical frameworks for analyzing generalization and robustness in statistical learning settings. The proposed frameworks are inspired by existing tools and ideas from the following fields of study: information theory, convex analysis, Fourier analysis, and optimal transport theory. More specifically, this dissertation is divided into three parts. The first part focuses on developing information theoretic approaches to supervised learning problems. We consider and optimize well-known information measures such as entropy, mutual information, and maximal correlation to design robust supervised learning and feature selection algorithms. The second part discusses novel information theoretic frameworks for generative adversarial nets (GANs) and group-structured adversarial learning problems. We use convex analysis tools as well as optimal transport costs to develop those frameworks. The third part is dedicated to understanding generalization in various machine learning contexts such as classic kernel learning and modern adversarial deep learning. We introduce novel Fourier-based and spectral approaches to better understand generalization in different learning contexts and propose several regularization tools for improving the generalization performance in those learning settings.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2019; ©2019 |
| Publication date | 2019; 2019 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Farnia, Farzan |
|---|---|
| Degree supervisor | Tse, David |
| Thesis advisor | Tse, David |
| Thesis advisor | Boyd, Stephen P |
| Thesis advisor | Montanari, Andrea |
| Thesis advisor | Weissman, Tsachy |
| Degree committee member | Boyd, Stephen P |
| Degree committee member | Montanari, Andrea |
| Degree committee member | Weissman, Tsachy |
| Associated with | Stanford University, Department of Electrical Engineering. |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Farzan Farnia. |
|---|---|
| Note | Submitted to the Department of Electrical Engineering. |
| Thesis | Thesis Ph.D. Stanford University 2019. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2019 by Farzan Farnia
- License
- This work is licensed under a Creative Commons Attribution 3.0 Unported license (CC BY).
Also listed in
Loading usage metrics...