Local linear learned method for image and reflectance estimation
Abstract/Contents
- Abstract
- The recent increase in the number of megapixels in digital cameras has resulted in images that have a higher spatial resolution than is required by most imaging applications. This excess of pixels offers an opportunity to redesign imaging sensors to achieve improvements in other aspects of photography instead of spatial resolution. Improved low light sensitivity, expanded dynamic range, and improved color estimation are possible by altering the sensor's color filter array (CFA). The CFA is a mosaic of optical filters overlaying each photosensitive site on the sensor that controls the spectral sensitivity of the pixel. A few novel CFA designs are proposed to achieve the improvements stated above. In addition to the above potential improvements to images for human perception, novel CFAs can enable multispectral imaging where the full spectrum of incoming light is estimated. There is a rich field of multispectral applications including computer vision, remote sensing, and medicine. Unfortunately expensive equipment is typically required to acquire multispectral measurements, which could be eliminated using new CFAs with a small number of color filters. For cameras with new CFAs, there is a great challenge in designing the image processing pipeline, the series of calculations that transforms the raw output from the sensor into a desirable image. In general, one needs to perform demosaicking to estimate a full color image from the sensor's image where only a single band is measured at each pixel. Also denoising is required to suppress photon shot noise and noise from imperfections in the sensor electronics. Finally, a transformation is needed from the input color space defined by the sensor's spectral sensitivities to a desired output color space. The Local Linear Learned (L3) pipeline is presented that simultaneously performs demosaicking, denoising, and color conversion for an arbitrary CFA to an arbitrary output color space. Although there exist numerous image processing algorithms for camera pipelines, very few have the ability to operate on any CFA, estimate images in color spaces with any number or shape of bands, or perform any of the pipeline calculations simultaneously. The L3 algorithm learns from a training set of images a method of clustering and adaptive linear filtering for each cluster. As a result, the algorithm can adapt to a specific dataset, CFA, and desired output color space that are appropriate for a particular imaging application. After training, the pipeline is applied to a pixel from a sensor image by identifying the appropriate cluster and applying that cluster's pre-computed filters. The algorithm enables rapid design and testing of future CFAs by automatically generating the image processing pipeline. To address the spectral design of the CFA and potential multispectral applications, the problem of estimating a surface's reflectance is analyzed. The reflectance, which describes how much light a surface reflects at each wavelength, is estimated using a small number of colorband measurements along with knowledge of the spectrum of the light and camera sensitivities. The L3 reflectance estimation method uses local filtering based on training reflectances that have similar color measurements to a given test point. The L3 method and the optimal Bayesian estimator have similar performance. The localization of the estimate compared to a global linear estimator is most advantageous for constrained datasets that may appear in specific applications.
Description
| Type of resource | text |
|---|---|
| Form | electronic; electronic resource; remote |
| Extent | 1 online resource. |
| Publication date | 2011 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Associated with | Lansel, Steven Paul |
|---|---|
| Associated with | Stanford University, Department of Electrical Engineering |
| Primary advisor | Wandell, Brian A |
| Thesis advisor | Wandell, Brian A |
| Thesis advisor | Levoy, Marc |
| Thesis advisor | Weissman, Tsachy |
| Advisor | Levoy, Marc |
| Advisor | Weissman, Tsachy |
Subjects
| Genre | Theses |
|---|
Bibliographic information
| Statement of responsibility | Steven Paul Lansel. |
|---|---|
| Note | Submitted to the Department of Electrical Engineering. |
| Thesis | Thesis (Ph.D.)--Stanford University, 2011. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2011 by Steven Paul Lansel
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...