Deep reinforcement learning methods for autonomous driving safety and interactivity
Abstract/Contents
- Abstract
- To drive a vehicle fully autonomously, an intelligent system needs to be capable of having accurate perception and comprehensive understanding of the surroundings, making reasonable predictions of the progressing of the scenario, and executing safe, comfortable, as well as efficient control actions. Currently, these requirements are mostly fulfilled by the intelligence of human drivers. During past decades, with the development of machine learning and computer science, artificial intelligence starts to show better-than-human performance on more and more practical applications, while autonomous driving is still one of the most attractive and difficult unconquered challenges. This thesis studies the challenges of autonomous driving on its safety and interaction with the surrounding vehicles, and how deep reinforcement learning methods could help address these challenges. Reinforcement learning (RL) is an important paradigm of machine learning which focuses on learning sequential decision-making policies which interact with the task environment. Combining with deep neural networks, the recent development of deep reinforcement learning has shown promising results on control and decision-making tasks with high dimensional observations and complex strategies. The capability and achievements of deep reinforcement learning indicate a wide range of potential applications in autonomous driving. Focusing on autonomous driving safety and interactivity, this thesis presents novel contributions on topics including safe and robust reinforcement learning, reinforcement learning-based safety test, human driver modeling, and multi-agent reinforcement learning. This thesis begins with the study of deep reinforcement learning methods on autonomous driving safety, which is the most critical concern for all autonomous driving systems. We study the autonomous driving safety problem from two points of view: the first is the risk caused by the reinforcement learning control policies due to the mismatch between simulations and the real world; the second is the deep reinforcement learning-based safety test. In both problems, we explore the usage of adversary reinforcement learning agents on finding failures of the system with different focuses: on the first problem, the RL adversary is trained and applied at the learning stage of the control policy to guide it to learn more robust behaviors; on the second problem, the RL adversary is used at the test stage to find the most likely failures in the system. Different learning approaches are proposed and studied for the two problems. Another fundamental challenge for autonomous driving is the interaction between the autonomous vehicle and its surrounding vehicles, which requires accurate modeling of the behavior of surrounding drivers. In the second and third parts of the thesis, we study the surrounding driver modeling problem on three different levels: the action distribution level, the latent state level, and the reasoning level. On the action distribution level, we explore advanced policy representations for modeling the complex distribution of driver's control actions. On the latent state level, we study how to efficiently infer the latent states of surrounding drivers like their driving characteristics and intentions, and how it could be combined with the learning of autonomous driving decision-making policies. On the reasoning level, we investigate the reasoning process between multiple interacting agents and use this to build their behavior models through multi-agent reinforcement learning.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2021; ©2021 |
| Publication date | 2021; 2021 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Ma, Xiaobai |
|---|---|
| Degree supervisor | Kochenderfer, Mykel J, 1980- |
| Thesis advisor | Kochenderfer, Mykel J, 1980- |
| Thesis advisor | Schwager, Mac |
| Degree committee member | Schwager, Mac |
| Associated with | Stanford University, Department of Aeronautics and Astronautics |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Xiaobai Ma. |
|---|---|
| Note | Submitted to the Department of Aeronautics and Astronautics. |
| Thesis | Thesis Ph.D. Stanford University 2021. |
| Location | https://purl.stanford.edu/mz221hz7239 |
Access conditions
- Copyright
- © 2021 by Xiaobai Ma
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...