Leveraging deepfakes to close the domain gap between real and synthetic images in facial capture pipelines
Abstract/Contents
- Abstract
- From a technical standpoint, this thesis presents a framework for personalized appearance and motion capture that utilizes unsupervised deep learning in the form of deepfake technology. Notably, only a minimal amount of person-specific in-the-wild imagery is required to drive these pipelines, as we were able to effectively leverage synthetically created ground truth data during network training. Such personalized facial capture pipelines bypass issues with data diversity and bias that plague large-scale data collection and large pre-trained monolithic models, and has potential benefits beyond appearance/motion capture and retargeting. From a more general standpoint, our methods reliance on only subject-specific in-the-wild data could be leveraged to parametrize and analyze the facial motion present in videos, and be subsequently used to detect frame based video forgery (e.g. as created by deepfakes, ) which is actually the original motivation for this line of research. To paint a more complete picture of this research, I present technical details of our methodology and results, but also provide some more high-level context on deepfake detection, on our initial efforts on this work, and on some brief musings on ``lean AI as an alternative to large networks trained on massive datasets.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2022; ©2022 |
| Publication date | 2022; 2022 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Lin, Peng-Wen |
|---|---|
| Degree supervisor | Fedkiw, Ronald P, 1968- |
| Thesis advisor | Fedkiw, Ronald P, 1968- |
| Thesis advisor | Liu, Cheng-Yun Karen, 1977- |
| Thesis advisor | Yeung, Serena |
| Degree committee member | Liu, Cheng-Yun Karen, 1977- |
| Degree committee member | Yeung, Serena |
| Associated with | Stanford University, Computer Science Department |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Winnie Lin. |
|---|---|
| Note | Submitted to the Computer Science Department. |
| Thesis | Thesis Ph.D. Stanford University 2022. |
| Location | https://purl.stanford.edu/fm636jn5413 |
Access conditions
- Copyright
- © 2022 by Peng-Wen Lin
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...