New problems and perspectives on learning with limited memory
Abstract/Contents
- Abstract
- This thesis explores various new questions around the role of memory in learning. These questions are motivated by the scale of modern machine learning models and datasets which have made memory a bottleneck in many practical settings, but at their core they are fundamental questions about the interaction of data, computation and learning. Following are some of the questions we consider: 1) What role does memory play in learning and optimization? Are there inherent trade-offs between the amount of memory required for learning or optimization, and the amount of data or computation required? 2) How do we learn concepts and make accurate predictions with small memory? When can we solve learning problems on high-dimensional data with memory much less than the dimensionality of the data? 3) How do we consolidate and reference memories to make predictions about future observations? When is it necessary to leverage long-term memory of observations from the distant past to make accurate predictions? In this thesis, we distill down these questions and formalize them to enable a systematic study. We undertake both a theoretical and an experimental study---showing new inherent limitations on learning with limited memory, and also developing new algorithms which can solve many problems of interest with a small memory footprint in practice. These results illuminate new perspectives on the role of memory in learning, and also suggest several directions of future research
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2020; ©2020 |
| Publication date | 2020; 2020 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Sharan, Vatsal |
|---|---|
| Degree supervisor | Valiant, Gregory |
| Thesis advisor | Valiant, Gregory |
| Thesis advisor | Duchi, John |
| Thesis advisor | Wootters, Mary |
| Degree committee member | Duchi, John |
| Degree committee member | Wootters, Mary |
| Associated with | Stanford University, Department of Electrical Engineering |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Vatsal Sharan |
|---|---|
| Note | Submitted to the Department of Electrical Engineering |
| Thesis | Thesis Ph.D. Stanford University 2020 |
| Location | electronic resource |
Access conditions
- Copyright
- © 2020 by Vatsal Sharan
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...