Structure and function in neural networks
Abstract/Contents
- Abstract
- Biology is unique among the natural sciences for its study of systems which exist, in some sense, to solve problems. Neuroscience, in turn, is unique within biology for the kinds of complex, multi-step, and high-dimensional information processing problems it confronts: visual object recognition, spatial reasoning and navigation, linguistic communication, and social behavior, to name a few. On its face, neuroscience is about discovering the brain's solutions to these problems. In many cases, however, this is likely to be extremely difficult without first understanding the tasks themselves, at least well enough that we could recognize the brain's solution if we saw it. In this dissertation, I present results from three sets of projects exemplifying a task-based approach to neuroscience. In one set of projects, we study recurrent neural networks trained to solve a navigation task, which were recently shown to spontaneously develop periodic firing patterns like those of entorhinal grid cells. Through detailed virtual neurophysiology and connectomics, we discover the network mechanisms responsible for grid firing in the network. Further, an analytic study of the objective function used during training reveals that hexagonal grids are indeed optimal for the task, furnishing a normative explanation for grid firing in both artificial and biological neural networks. The second set of projects is motivated by the idea that the functions implemented by biological circuits -- e.g. visual object recognition -- are, in some sense, aligned to the statistical structure of the input data, rather than random. We study a very simple form of alignment between input data and task, deriving exact formulas for the generalization error of both a linear model and the random feature model as a function of alignment. We prove that aligned tasks are indeed easier to learn. In the process, we uncover a rich mathematical structure relating to the spectrum of kernel matrices, predict, and then confirm, a novel empirical phenomenon known as multiple sample-wise descent, and demonstrate the application of recently developed mathematical techniques from the field of free probability to problems in learning theory. The third and final set of projects concerns the important practical issue of when one can accurately detect and decode low-dimensional task structure in neural recordings. Using a combination of techniques from statistical physics, random matrix theory, and free probability, we derive formulas for the detectability/decodability of low-dimensional signals in noisy data and demonstrate the predictions of the theory in the newly coined \emph{extensive spike model}. Finally, I conclude and briefly discuss directions for future research.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2024; ©2024 |
| Publication date | 2024; 2024 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Mel, Gabriel Carreira |
|---|---|
| Degree supervisor | Ganguli, Surya |
| Thesis advisor | Ganguli, Surya |
| Thesis advisor | Baccus, Stephen |
| Thesis advisor | Druckmann, Shaul |
| Thesis advisor | Giocomo, Lisa |
| Degree committee member | Baccus, Stephen |
| Degree committee member | Druckmann, Shaul |
| Degree committee member | Giocomo, Lisa |
| Associated with | Stanford University, School of Medicine |
| Associated with | Stanford University, Neurosciences Program |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Gabriel Carreira Mel de Fontenay. |
|---|---|
| Note | Submitted to the Neurosciences Program. |
| Thesis | Thesis Ph.D. Stanford University 2024. |
| Location | https://purl.stanford.edu/kj403pf9573 |
Access conditions
- Copyright
- © 2024 by Gabriel Carreira Mel
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...