Long-horizon prediction for human-robot collaboration
Abstract/Contents
- Abstract
- Robots are mainly seen as tools that offer passive assistance to humans or operate independently in close proximity to them. However, we are still far from witnessing robots that can collaborate with humans on tasks wherein both agents' actions and dynamics are interdependent and achieve synergistic results. As an example, and one that serves as the central motivation of my work, robots carrying a table towards a goal location and avoiding obstacles must be able to predict their partners' actions and mutually adapt to changing strategies during collaboration. Key challenges have been accounting for diverse, multimodal human behaviors over both short and long horizons, and producing joint actions that are temporally consistent. To address these issues, I leverage generative modeling for accurate sequence prediction of human-like motion and behaviors. I focus on two approaches to model interaction: first, by predicting team subgoals and executing cooperative actions; and second, by predicting partner actions and executing collaborative actions that would account for the partner's efforts. In the first part of this thesis, I develop a Variational Recurrent Neural Network-based planner trained with human-human demonstrations that learns to sample sequences of future waypoints autoregressively to achieve successful synergy with a human-in-the-loop on the table-carrying task. Next, I show that mixture density models can capture multimodality in human-human demonstrations, but fail to predict action sequences accurately for the cooperative carrying task. Finally, I show that a co-policy developed with a Transformer-based diffusion model conditioning on past human actions can not only plan action sequences with real humans-in-the-loop to achieve high success rates, but also display compelling collaborative behaviors in novel, out-of-training-distribution settings.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2023; ©2023 |
| Publication date | 2023; 2023 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Ng, Eley |
|---|---|
| Degree supervisor | Kennedy, Monroe |
| Thesis advisor | Kennedy, Monroe |
| Thesis advisor | Sadigh, Dorsa |
| Thesis advisor | Schwager, Mac |
| Degree committee member | Sadigh, Dorsa |
| Degree committee member | Schwager, Mac |
| Associated with | Stanford University, School of Engineering |
| Associated with | Stanford University, Department of Mechanical Engineering |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Eley Ng. |
|---|---|
| Note | Submitted to the Department of Mechanical Engineering. |
| Thesis | Thesis Ph.D. Stanford University 2023. |
| Location | https://purl.stanford.edu/nm281rs5069 |
Access conditions
- Copyright
- © 2023 by Eley Ng
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...