Self-supervised scene representation learning
Abstract/Contents
- Abstract
- Unsupervised learning with generative models has the potential of discovering rich representations of 3D scenes. Such Neural Scene Representations may subsequently support a wide variety of downstream tasks, ranging from robotics to computer graphics to medical imaging. However, existing methods ignore one of the most fundamental properties of scenes: their three-dimensional structure. In this work, we make the case for equipping Neural Scene Representations with an inductive bias for 3D structure. We demonstrate how this inductive bias enables the unsupervised discovery of geometry and appearance, given only posed 2D images. By learning a distribution over a set of such 3D-structure aware neural representations, we can perform joint reconstruction of 3D shape and appearance given only a single 2D observation. We show that the features learned in this process enable 3D semantic segmentation of a whole class of objects, trained with as few as 30 labeled examples, demonstrating a strong link between 3D shape, appearance, and semantic segmentation. Finally, we reflect on the nature and potential role of scene representation learning in computer vision itself, and discuss promising avenues for future work
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2020; ©2020 |
| Publication date | 2020; 2020 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Sitzmann, Vincent Simon |
|---|---|
| Degree supervisor | Wetzstein, Gordon |
| Thesis advisor | Wetzstein, Gordon |
| Thesis advisor | Guibas, Leonidas J |
| Thesis advisor | Wandell, Brian A |
| Degree committee member | Guibas, Leonidas J |
| Degree committee member | Wandell, Brian A |
| Associated with | Stanford University, Department of Electrical Engineering. |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Vincent Sitzmann |
|---|---|
| Note | Submitted to the Department of Electrical Engineering |
| Thesis | Thesis Ph.D. Stanford University 2020 |
| Location | electronic resource |
Access conditions
- Copyright
- © 2020 by Vincent Simon Sitzmann
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...