Visual content creation and editing via structural and functional hierarchies
Abstract/Contents
- Abstract
- Starting from low-level perceptual input, humans can automatically infer higher-level semantically meaningful structures in the visual world. In this thesis, I present my work on systems that extract and leverage such hierarchies for visual content creation and editing. First, I present Motion Programs, a hierarchical motion representation for human actions that captures both low-level motion and its high-level description as motion concepts. This representation enables human motion description, interactive editing, and controlled synthesis of novel video sequences within a single framework. Next, I present ProGRIP, a novel shape program representation that uses implicit functions to represent parts. ProGRIP outperforms existing structured representations in shape reconstruction fidelity and semantic segmentation and enables interactive editing. Finally, I present a model for realistically inserting people into scenes. Our model can infer the set of realistic poses given the scene context, re-pose the reference person, and harmonize the composition. We train a large-scale diffusion model to perform this task. At inference time, our model can also be prompted differently to perform several different auxiliary tasks, such as person hallucination, scene hallucination, and cloth swapping. Quantitative and qualitative evaluations show that our method synthesizes more realistic human appearance and more natural human-scene interactions than prior work.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2023; ©2023 |
| Publication date | 2023; 2023 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Kulal, Sumith |
|---|---|
| Degree supervisor | Aiken, Alexander |
| Degree supervisor | Wu, Jiajun, (Computer scientist) |
| Thesis advisor | Aiken, Alexander |
| Thesis advisor | Wu, Jiajun, (Computer scientist) |
| Thesis advisor | Niebles Duque, Juan Carlos, 1980- |
| Degree committee member | Niebles Duque, Juan Carlos, 1980- |
| Associated with | Stanford University, School of Engineering |
| Associated with | Stanford University, Computer Science Department |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Sumith Kulal. |
|---|---|
| Note | Submitted to the Computer Science Department. |
| Thesis | Thesis Ph.D. Stanford University 2023. |
| Location | https://purl.stanford.edu/gk508gg0730 |
Access conditions
- Copyright
- © 2023 by Sumith Kulal
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...