Algorithms for black-box safety validation
Abstract/Contents
- Abstract
- Autonomous systems have the potential to improve safety and efficiency in safety-critical domains such as transportation, and medicine. Since human lives are at risk in these applications, we require rigorous safety validation before deployment. Traditional safety validation approaches such as real-world testing and scenario-based testing in simulation are not scalable to complex systems and environments and may miss unforeseen failures. Formal verification techniques also lack the scalability required for large scale autonomy. The thesis address the safety validation problem with black-box sampling techniques, which assume no knowledge of the design of the autonomous system. The system takes actions in a stochastic environment and failures are discovered by sampling environmental disturbances. The black-box assumption allows for better scalability to complex autonomous systems and sampling can be combined with machine learning to discover unforeseen failures. Previous black-box safety validation approaches have been based on optimization, path-planning, reinforcement learning and importance sampling. Although successful for many safety validation applications, existing algorithms may have poor interpretabiliy, scalability, and efficiency. Black-box sampling approaches can provide example failure trajectories but do not provide a high-level description of failures, as scenario-based approaches do. We present a new technique for generating failure descriptions in the form of signal temporal logic specifications on the environment disturbances. The specifications are optimized with genetic programming to produce failure examples and can used to gain insight into why a failure occurred. A key contribution of this thesis is the proposal and analysis of a state-dependent sampling distribution to approximate the distribution over failures. The use of the state of the environment produces a more efficient sampling distribution than baseline importance sampling approaches, but may be limited by the size of the state space. To improve scalability, we propose a decomposition technique for multi-agent validation tasks. Each subproblem is solved independently and the results are combined for better performance than learning from scratch. During the design of an autonomous system, safety validation is performed repeatedly, requiring a large computational expense. We propose a transfer learning technique that can reduce the number of required samples and lead to better performance. Knowledge from previous validation tasks is transferred to new tasks in the form of value functions that are combined using a learned set of attention weights. Results show improved knowledge transfer between tasks compared to baseline techniques. The safety validation algorithms presented in this work are tested on two gridworld scenarios and two driving scenarios. A simple gridworld scenario is used to illustrate important safety validation concepts while a gridworld with multiple adversaries is used as a test case for multi-agent validation. A rules-based autonomous driving policy is tested in a crosswalk scenario with a pedestrian and a T-intersection scenario with multiple vehicles. It is shown that the presented algorithms can improve the interpretability, scalability, and efficiency of safety validation.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2021; ©2021 |
| Publication date | 2021; 2021 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Corso, Anthony Louis |
|---|---|
| Degree supervisor | Kochenderfer, Mykel J, 1980- |
| Thesis advisor | Kochenderfer, Mykel J, 1980- |
| Thesis advisor | Pavone, Marco, 1980- |
| Thesis advisor | Sadigh, Dorsa |
| Degree committee member | Pavone, Marco, 1980- |
| Degree committee member | Sadigh, Dorsa |
| Associated with | Stanford University, Department of Aeronautics and Astronautics |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Anthony Corso. |
|---|---|
| Note | Submitted to the Department of Aeronautics and Astronautics. |
| Thesis | Thesis Ph.D. Stanford University 2021. |
| Location | https://purl.stanford.edu/xw289mz1340 |
Access conditions
- Copyright
- © 2021 by Anthony Louis Corso
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...