Advancing optimization for modern machine learning
Abstract/Contents
- Abstract
- Machine learning is a transformative computational tool on its way to revolutionizing a number of technologies and scientific applications. However, recent successes in artificial intelligence and machine learning, and the resulting imminent widespread deployment of models have transformed the classical machine learning pipeline. First of, the sheer scale of available data---in both quantity and dimensionality---has exploded. Furthermore, modern machine learning architectures come with an exponential number of design choices and hyperparameters, yet they are all optimized using generic stochastic gradient methods. This highlights the need for adaptive gradient methods that perform adequately without prior knowledge of the instance they will be given. Institutions then deploy these models in the wild and expect them to provide good predictions even on out-of-distribution inputs---this emphasizes the need for robust models. Finally, as we collect evermore user data, we wish that models trained on this data do not compromise the privacy of individuals present in the training set as we release these models to the public. In this thesis, we show that solving these emerging problems require fundamental advances in optimization. More specifically, we first present new theoretical results on understanding the optimality of adaptive gradient algorithms and show a practical use case of adaptive methods in the context of gradient-based samplers. We then present scalable methods for min-max optimization with the goal of efficiently solving robust objectives. We conclude by developing private optimization methods that optimally learn under more stringent privacy requirements, as well as adaptive methods that add the "right amount of noise" and significantly decrease the price of privacy on easy instances.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2021; ©2021 |
| Publication date | 2021; 2021 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Levy, Daniel Asher Nathan |
|---|---|
| Degree supervisor | Duchi, John |
| Degree supervisor | Liang, Percy |
| Thesis advisor | Duchi, John |
| Thesis advisor | Liang, Percy |
| Thesis advisor | Ré, Christopher |
| Thesis advisor | Sidford, Aaron |
| Degree committee member | Ré, Christopher |
| Degree committee member | Sidford, Aaron |
| Associated with | Stanford University, Computer Science Department |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Daniel Levy. |
|---|---|
| Note | Submitted to the Computer Science Department. |
| Thesis | Thesis Ph.D. Stanford University 2021. |
| Location | https://purl.stanford.edu/nv133bt5905 |
Access conditions
- Copyright
- © 2021 by Daniel Asher Nathan Levy
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...