Modularity and coordination for planning and reinforcement learning
Abstract/Contents
- Abstract
- The foundational objective of the field of artificial intelligence is to build autonomous systems that can perceive their environment and take actions that maximize their ability to achieve their goals. Decision making under uncertainty is a fundamental requirement for such intelligent behavior. Various real world problems of interest like autonomous driving, virtual assistants, and disaster response are sequential decision making problems. Planning and reinforcement learning are abstractions for studying optimal sequential decision making in natural and artificial systems. Combining these ideas with deep neural network function approximation (*"deep reinforcement learning"*) has allowed scaling these abstractions to a variety of complex problems and has led to super-human performance, especially in game playing. These successes are still limited to virtual worlds with fast simulators where massive amounts of training data can be generated given enough computational resources. However, decision making in the real world requires solutions that are data efficient, capable of utilizing domain knowledge when available, and generalize to related problems. Moreover, often decision making requires decentralized execution for scalability. The concept of modularity has proven effective in a large number of fields to deal with complex systems. The key ideas driving a modular system are 1) information encapsulation and 2) coordination for integrated function. Modularity allows breaking down a complex problem into manageable units. This dissertation explores how, as designers of complex decision making systems, the principles of modular design can allow us to provide structural inductive biases and define appropriate coordination mechanisms. In the first part, we explore the concept of functional modularity in the form of agents, and how they can inform the design of large multi-agent decision making systems. In the second part, we explore the concept of temporal modularity in the form of subtasks in complicated tasks and how we can learn decomposed solutions that show improved transfer performance to related tasks. Finally, in the last part, we explore the concept of architectural modularity; how known physics can inform our neural network models of mechanical systems allowing reliable planning and efficient reinforcement learning. We find that these design principles lead to enormous data efficiency improvements and lower costs for learning and inference. Moreover, we find solutions that generalize better to related problems
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2020; ©2020 |
| Publication date | 2020; 2020 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Gupta, Jayesh Kumar |
|---|---|
| Degree supervisor | Ermon, Stefano |
| Degree supervisor | Kochenderfer, Mykel J, 1980- |
| Thesis advisor | Ermon, Stefano |
| Thesis advisor | Kochenderfer, Mykel J, 1980- |
| Thesis advisor | Brunskill, Emma |
| Degree committee member | Brunskill, Emma |
| Associated with | Stanford University, Computer Science Department |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Jayesh Kumar Gupta |
|---|---|
| Note | Submitted to the Computer Science Department |
| Thesis | Thesis Ph.D. Stanford University 2020 |
| Location | electronic resource |
Access conditions
- Copyright
- © 2020 by Jayesh Kumar Gupta
- License
- This work is licensed under a Creative Commons Attribution 3.0 Unported license (CC BY).
Also listed in
Loading usage metrics...