Deep optics : optimizing integrated imaging and computer vision systems
Abstract/Contents
- Abstract
- Over the past decade, computational imaging has transformed the expectations we have for our cameras. At the fingertips of anyone with an iPhone X, for example, are panoramas, high dynamic range (HDR) photography, and Portrait mode---all classic examples of computational imaging. This integration of optical design and image processing has also enabled us to see around corners and behind walls, observe molecular dynamics during chemical reactions, and even to piece together the first image of a black hole. For each application though, the system needs to be reformulated and designed, and it is difficult to claim that the final solution is actually optimal. Concurrently, a tremendous amount of progress has been made in the field of deep learning. Convolutional neural networks (CNNs) have found success in a wide variety of computer vision tasks, including image classification, object detection, image generation, and more. However, this high performance comes at a high computational cost, and it remains difficult to deploy CNNs in embedded systems due to tight power budgets. Through earlier projects discussed in Section I, we gain an understanding of the ways computational imaging and deep learning could potentially enhance and benefit from the other. In section II, we present the concept of deep optics, in which the optical model is jointly optimized with the task-specific algorithm. Deep optics capitalizes on the recent successes of deep learning for computer vision and uses the same data-driven approach to optimize optical parameters together with subsequent neural network parameters. Not only could this improve computer vision performance over basic or heuristically designed cameras, but it could also relieve some of the computational cost by partially pre-processing the data in optics. We begin by building a hybrid optical-electronic convolutional neural network for image classification, specifically focusing on the concept of an optimizable optical convolutional layer to replace the standard electronic convolutional layer. Finally, we apply deep optics to the problems of monocular depth estimation and 3D object detection, demonstrating how these common computer vision tasks that typically ignore the image capture process can be improved by proper modeling and optimization of the optics.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2019; ©2019 |
| Publication date | 2019; 2019 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Chang, Julie |
|---|---|
| Degree supervisor | Wetzstein, Gordon |
| Thesis advisor | Wetzstein, Gordon |
| Thesis advisor | Lee, Jin Hyung |
| Thesis advisor | Liphardt, Jan |
| Degree committee member | Lee, Jin Hyung |
| Degree committee member | Liphardt, Jan |
| Associated with | Stanford University, Department of Bioengineering. |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Julie Chang. |
|---|---|
| Note | Submitted to the Department of Bioengineering. |
| Thesis | Thesis Ph.D. Stanford University 2019. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2019 by Julie Chang
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...