Performance limits for motion deblurring cameras

doi:10.1017/CBO9781107360181.014

13 - Performance limits for motion deblurring cameras

Published online by Cambridge University Press: 05 June 2014

Oliver Cossairt and

Mohit Gupta

Edited by

A. N. Rajagopalan and

Rama Chellappa

Show author details

Oliver Cossairt: Affiliation:
Northwestern University
Mohit Gupta: Affiliation:
Mitsubishi Electric Research Labs (MERL), USA
A. N. Rajagopalan: Affiliation:
Indian Institute of Technology, Madras
Rama Chellappa: Affiliation:
University of Maryland, College Park

Book contents

Get access

Summary

Introduction

A number of computational imaging (CI) based motion deblurring techniques have been introduced to improve image quality. These techniques use optical coding to measure a stronger signal level instead of a noisy short exposure image. However, the performance of these techniques is limited by the decoding step, which amplifies noise. While it is well understood that optical coding can increase performance at low light levels, little is known about the quantitative performance advantage of computational imaging in general settings.

In this chapter, we derive the performance bounds for various computational imaging-based motion deblurring techniques. We then discuss the implications of these bounds for several real-world scenarios. The scenarios are defined in terms of real-world lighting (e.g. moonlit night or cloudy day, indoor or outdoor), scene properties (albedo, object velocities), and sensor characteristics. The results show that computational imaging techniques do not provide a significant performance advantage when imaging with illumination brighter than typical indoor lighting. This is illustrated in Figure 13.1. These results can be readily used by practitioners to decide whether to use CI and, if so, to design the imaging system. We also study the role of image priors on the decoding steps. Our empirical results show that the use of priors reduces the performance advantage of CI techniques even further.

Scope

The analysis in this chapter focuses on techniques that use optical coding to preserve high frequencies in the blur kernels so that deblurring becomes a well-conditioned problem. These techniques assume that the blur kernel is known a priori. The analysis is limited to techniques that acquire a single image and follow a linear imaging model.

Information

Type: Chapter
Information: Motion Deblurring
Algorithms and Systems
, pp. 258 - 282

DOI: https://doi.org/10.1017/CBO9781107360181.014 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2014

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Agrawal, A. & Raskar, R. (2009). Optimal single image capture for motion deblurring. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2560–7.

Agrawal, A., Xu, Y. & Raskar, R. (2009). Invertible motion blur in video. In ACM Special Interest Group on Graphics and Interactive Techniques, 28(3), 95:1–8.Google Scholar

Baer, R., Holland, W., Holm, J. & Vora, P. (1999). A comparison of primary and complementary color filters for CCD-based digital photography. In SPIE Electronic Imaging Conference. Citeseer, pp. 16-25.

Bioucas-Dias, J. & Figueiredo, M. (2007). Anew twist: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing, 16(12), 2992–3004.Google Scholar

Cai, J., Ji, H., Liu, C. & Shen, Z. (2009). Blind motion deblurring using multiple images. Journal of Computational Physics, 228(14), 5057–71.CrossRef Google Scholar

Chakrabarti, A. & Zickler, T. (2011). Statistics of real-world hyperspectral images. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 193–200.

Chambolle, A. (2004). An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision, 20(1), 89-97.Google Scholar

Chi, W. & George, N. (2001). Electronic imaging using a logarithmic asphere. Optics Letters, 26(12), pp. 875–7.Google Scholar

Cho, T., Levin, A., Durand, F. & Freeman, W. (2010). Motion blur removal with orthogonal parabolic exposures. In IEEE International Conference on Computational Photography, pp. 1-8.

Clark, R. N. (2013). Digital camera sensor performance summary. “http://www.clarkvision.com/articles/digital.sensor.performance.summary/#model”.

Cossairt, O. (2011). Tradeoffs and Limits in Computational Imaging. Ph.D. Thesis. Technical report, Department of Computer Science, Columbia University.

Cossairt, O., Gupta, M. & Nayar, S. (2012). Whendoes computational imaging improve performance?IEEE Transactions on Image Processing, 22(2), 447–58.Google Scholar

Cossairt, O. & Nayar, S. K. (2010). Spectral focal sweep: extended depth of field from chromatic aberrations. In IEEE International Conference on Computational Photography, pp. 1-8.

Cossairt, O., Zhou, C. & Nayar, S. K. (2010). Diffusion coding photography for extended depth of field. In ACM Special Interest Group on Graphics and Interactive Techniques, 29(4), 31:1–10.Google Scholar

Dabov, K., Foi, A., Katkovnik, V. & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–95.Google Scholar

DeCusatis, C. (1997). Handbook of Applied Photometry. New York: AIP-Press.

Dowski, E. R. Jr. & Cathey, W. T. (1995). Extended depth of field through wave-front coding. Applied Optics, 34(11), 1859–66.CrossRef Google Scholar

Fei-Fei, L., Fergus, R. & Perona, P. (2004). Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-9.

Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T. & Freeman, W. T. (2006). Removing camera shake from a single image. In ACM Special Interest Group on Graphics and Interactive Techniques, 25(3), 787–94.Google Scholar

Gaubatz, M. (2013). MeTriX MuX visual quality assessment package. “http://foulard.ece.cornell.edu/gaubatz/metrix_mux/”.

Guichard, F., Nguyen, H., Tessières, R., Pyanet, M., Tarchouna, I. & Cao, F. (2009). Extended depth-of-field using sharpness transport across color channels. In Digital Photography V, vol. 7250, SPIE.

Hanley, Q., Verveer, P. & Jovin, T. (1999). Spectral imaging in a programmable array microscope by Hadamard transform fluorescence spectroscopy. Applied Spectro scopy, 53(1), 1-10.Google Scholar

Harwit, M. & Sloane, N. (1979). Hadamard Transform Optics. New York: Academic Press.

Hasinoff, S., Kutulakos, K., Durand, F. & Freeman, W. (2009). Time-constrained photography. In IEEE International Conference on Computer Vision, pp. 1-8.

Häusler, G. (1972). A method to increase the depth of focus by two step image processing. Optics Communications, 6(1), 38-42.Google Scholar

Horn, B. (1986). Robot Vision. MIT Press.

Horstmeyer, R., Euliss, G. W., Athale, R. A. & Levoy, M. (2009). Flexible multimodal camera using a light field architecture. In IEEE International Conference on Computational Photography, pp. 1-8.

Ihrke, I., Wetzstein, G. & Heidrich, W. (2010). A theory of plenoptic multiplexing. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 483–90.

Joshi, N., Kang, S. B., Zitnick, C. L. & Szeliski, R. (2010). Image deblurring using inertial measurement sensors. In ACM Special Interest Group on Graphics and Interactive Techniques, 29(4), 30:1–9.Google Scholar

Krishnan, D. & Fergus, R. (2009). Dark flash photography. In ACM Special Interest Group on Graphics and Interactive Techniques, 28(3), 96:1–11.Google Scholar

Kuthirummal, S., Nagahara, H., Zhou, C. & Nayar, S. K. (2010). Flexible depth of field photography. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1), 58-71.CrossRef Google Scholar

Lanman, D., Raskar, R., Agrawal, A. & Taubin, G. (2008). Shield fields: modeling and capturing 3D occluders. In ACM Special Interest Group on Graphics and Interactive Techniques, 27(5), 131:1–10.Google Scholar

Levin, A., Fergus, R., Durand, F. & Freeman, W. T. (2007). Image and depth from a conventional camera with a coded aperture. In ACM Special Interest Group on Graphics and Interactive Techniques, 26(3), 70:1–9.Google Scholar

Levin, A., Sand, P., Cho, T., Durand, F. & Freeman, W. (2008). Motion-invariant photography. In ACM Special Interest Group on Graphics and Interactive Techniques, 27(3), 71:1–9.Google Scholar

Liang, C., Lin, T., Wong, B., Liu, C. & Chen, H. (2008). Programmable aperture photography: multiplexed light field acquisition. In ACM Special Interest Group on Graphics and Interactive Techniques, 27(3), 55:1–10.Google Scholar

Ojeda-Castaneda, J., Landgrave, J. E. A. & Escamilla, H. M. (2005). Annular phase-only mask for high focal depth. Optics Letters, 30(13), 1647–9.Google Scholar

Raskar, R., Agrawal, A. & Tumblin, J. (2006). Coded exposure photography: motion deblurring using fluttered shutter. In ACM Special Interest Group on Graphics and Interactive Techniques, 25(3), 795–804.Google Scholar

Ratner, N. & Schechner, Y. (2007). Illumination multiplexing within fundamental limits. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8.

Ratner, N., Schechner, Y. & Goldberg, F. (2007). Optimal multiplexed sensing: bounds, conditions and a graph theory link. Optics Express, 15, 17072–92.Google Scholar

Schechner, Y., Nayar, S. & Belhumeur, P. (2007). Multiplexing for optimal lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(8), 1339–54.CrossRef Google Scholar

Shan, Q., Jia, J. & Agarwala, A. (2008). High-quality motion deblurring from a single image, In ACM Special Interest Group on Graphics and Interactive Techniques, 27(3), 73:1–10.Google Scholar

Sheikh, H. & Bovik, A. (2006). Image information and visual quality. IEEE Transactions on Image Processing, 15(2), 430–44.Google Scholar

Tendero, Y. (2012). Mathematical theory of the flutter shutter. Ph.D. thesis, École normale supérieure de Cachan.

Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A. & Tumblin, J. (2007). Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing. In ACM Special Interest Group on Graphics and Interactive Techniques, 26(3), 69:1–11.

Wang, Z. & Bovik, A. (2002). A universal image quality index. Signal Processing Letters, 9(3), 81–4.Google Scholar

Wang, Z., Bovik, A., Sheikh, H. & Simoncelli, E. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–12.CrossRef Google Scholar

Wikipedia (2013a). Lux “http://en.wikipedia.org/wiki/Lux”.

Wikipedia (2013b). Projection slice theorem. “http://en.wikipedia.org/wiki/Projection-slice_theorem”.

Yuan, L., Sun, J., Quan, L. & Shum, H.-Y. (2007). Image deblurring with blurred/noisy image pairs. In ACM Special Interest Group on Graphics and Interactive Techniques, 26(3), 1:1–10.Google Scholar

Zhang, L., Deshpande, A. & Chen, X. (2010). Denoising versus deblurring: HDR techniques using moving cameras. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 522–9.

Zhou, C., Lin, S. & Nayar, S. (2011). Coded aperture pairs for depth from defocus and defocus deblurring. International Journal of Computer Vision, 93(1), 53-72.Google Scholar

Zhou, C. & Nayar, S. (2009). What are good apertures for defocus deblurring? In IEEE International Conference on Computational Photography, pp. 1-8.

Accessibility standard: Unknown

Why this information is here

This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.

Accessibility Information

Accessibility compliance for the PDF of this chapter is currently unknown and may be updated in the future.