Nature of learning and learning of nature
Abstract/Contents
- Abstract
- This thesis explores questions surrounding the foundations of intelligence, both artificial and natural. The first part focuses on the algorithmic and statistical underpinnings of modern machine learning systems. First, we discuss a clean framework for investigating the surprising ability of large language models to learn in-context: the apparent ability to solve new tasks given just a text prompt that provides examples. Further, motivated by concerns around the insatiable data appetite of modern machine learning systems, we discuss the problem of "sample amplification", where we formalize the seemingly naive question of how hard it is to create new data and contrast the hardness of this task to that of learning the data-generating distribution. The second part considers the algorithmic basis of intelligence in nature, specifically in ant colonies and the brain. We examine how arboreal turtle ants solve variants of the shortest path problem without any central control and with minimal computational resources. In the context of the brain, we study how it manages to train its neural network despite its structural limitations. Specifically, we investigate a biologically plausible learning algorithm and contrast it with gradient descent, arguably the only known algorithm for training large-scale artificial neural networks.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2023; ©2023 |
| Publication date | 2023; 2023 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Garg, Shivam |
|---|---|
| Degree supervisor | Valiant, Gregory |
| Thesis advisor | Valiant, Gregory |
| Thesis advisor | Charikar, Moses |
| Thesis advisor | Tan, Li-Yang |
| Degree committee member | Charikar, Moses |
| Degree committee member | Tan, Li-Yang |
| Associated with | Stanford University, School of Engineering |
| Associated with | Stanford University, Computer Science Department |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Shivam Garg. |
|---|---|
| Note | Submitted to the Computer Science Department. |
| Thesis | Thesis Ph.D. Stanford University 2023. |
| Location | https://purl.stanford.edu/tc258nv3060 |
Access conditions
- Copyright
- © 2023 by Shivam Garg
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...