Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-p9bg8 Total loading time: 0 Render date: 2025-01-05T15:12:19.686Z Has data issue: false hasContentIssue false

13 - Performance limits for motion deblurring cameras

Published online by Cambridge University Press:  05 June 2014

By
Oliver Cossairt  and
Mohit Gupta
Edited by
A. N. Rajagopalan  and
Rama Chellappa
Oliver Cossairt
Affiliation:
Northwestern University
Mohit Gupta
Affiliation:
Mitsubishi Electric Research Labs (MERL), USA
A. N. Rajagopalan
Affiliation:
Indian Institute of Technology, Madras
Rama Chellappa
Affiliation:
University of Maryland, College Park
Get access

Summary

Introduction

A number of computational imaging (CI) based motion deblurring techniques have been introduced to improve image quality. These techniques use optical coding to measure a stronger signal level instead of a noisy short exposure image. However, the performance of these techniques is limited by the decoding step, which amplifies noise. While it is well understood that optical coding can increase performance at low light levels, little is known about the quantitative performance advantage of computational imaging in general settings.

In this chapter, we derive the performance bounds for various computational imaging-based motion deblurring techniques. We then discuss the implications of these bounds for several real-world scenarios. The scenarios are defined in terms of real-world lighting (e.g. moonlit night or cloudy day, indoor or outdoor), scene properties (albedo, object velocities), and sensor characteristics. The results show that computational imaging techniques do not provide a significant performance advantage when imaging with illumination brighter than typical indoor lighting. This is illustrated in Figure 13.1. These results can be readily used by practitioners to decide whether to use CI and, if so, to design the imaging system. We also study the role of image priors on the decoding steps. Our empirical results show that the use of priors reduces the performance advantage of CI techniques even further.

Scope

The analysis in this chapter focuses on techniques that use optical coding to preserve high frequencies in the blur kernels so that deblurring becomes a well-conditioned problem. These techniques assume that the blur kernel is known a priori. The analysis is limited to techniques that acquire a single image and follow a linear imaging model.

Type
Chapter
Information
Motion Deblurring
Algorithms and Systems
 , pp. 258 - 282
Publisher: Cambridge University Press
Print publication year: 2014

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Agrawal, A. & Raskar, R. (2009). Optimal single image capture for motion deblurring. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2560–7.
Agrawal, A., Xu, Y. & Raskar, R. (2009). Invertible motion blur in video. In ACM Special Interest Group on Graphics and Interactive Techniques, 28(3), 95:1–8.Google Scholar
Baer, R., Holland, W., Holm, J. & Vora, P. (1999). A comparison of primary and complementary color filters for CCD-based digital photography. In SPIE Electronic Imaging Conference. Citeseer, pp. 16-25.
Bioucas-Dias, J. & Figueiredo, M. (2007). Anew twist: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing, 16(12), 2992–3004.Google Scholar
Cai, J., Ji, H., Liu, C. & Shen, Z. (2009). Blind motion deblurring using multiple images. Journal of Computational Physics, 228(14), 5057–71.CrossRefGoogle Scholar
Chakrabarti, A. & Zickler, T. (2011). Statistics of real-world hyperspectral images. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 193–200.
Chambolle, A. (2004). An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision, 20(1), 89-97.Google Scholar
Chi, W. & George, N. (2001). Electronic imaging using a logarithmic asphere. Optics Letters, 26(12), pp. 875–7.Google Scholar
Cho, T., Levin, A., Durand, F. & Freeman, W. (2010). Motion blur removal with orthogonal parabolic exposures. In IEEE International Conference on Computational Photography, pp. 1-8.
Clark, R. N. (2013). Digital camera sensor performance summary. “http://www.clarkvision.com/articles/digital.sensor.performance.summary/#model”.
Cossairt, O. (2011). Tradeoffs and Limits in Computational Imaging. Ph.D. Thesis. Technical report, Department of Computer Science, Columbia University.
Cossairt, O., Gupta, M. & Nayar, S. (2012). Whendoes computational imaging improve performance?IEEE Transactions on Image Processing, 22(2), 447–58.Google Scholar
Cossairt, O. & Nayar, S. K. (2010). Spectral focal sweep: extended depth of field from chromatic aberrations. In IEEE International Conference on Computational Photography, pp. 1-8.
Cossairt, O., Zhou, C. & Nayar, S. K. (2010). Diffusion coding photography for extended depth of field. In ACM Special Interest Group on Graphics and Interactive Techniques, 29(4), 31:1–10.Google Scholar
Dabov, K., Foi, A., Katkovnik, V. & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–95.Google Scholar
DeCusatis, C. (1997). Handbook of Applied Photometry. New York: AIP-Press.
Dowski, E. R. Jr. & Cathey, W. T. (1995). Extended depth of field through wave-front coding. Applied Optics, 34(11), 1859–66.CrossRefGoogle Scholar
Fei-Fei, L., Fergus, R. & Perona, P. (2004). Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-9.
Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T. & Freeman, W. T. (2006). Removing camera shake from a single image. In ACM Special Interest Group on Graphics and Interactive Techniques, 25(3), 787–94.Google Scholar
Gaubatz, M. (2013). MeTriX MuX visual quality assessment package. “http://foulard.ece.cornell.edu/gaubatz/metrix_mux/”.
Guichard, F., Nguyen, H., Tessières, R., Pyanet, M., Tarchouna, I. & Cao, F. (2009). Extended depth-of-field using sharpness transport across color channels. In Digital Photography V, vol. 7250, SPIE.
Hanley, Q., Verveer, P. & Jovin, T. (1999). Spectral imaging in a programmable array microscope by Hadamard transform fluorescence spectroscopy. Applied Spectro scopy, 53(1), 1-10.Google Scholar
Harwit, M. & Sloane, N. (1979). Hadamard Transform Optics. New York: Academic Press.
Hasinoff, S., Kutulakos, K., Durand, F. & Freeman, W. (2009). Time-constrained photography. In IEEE International Conference on Computer Vision, pp. 1-8.
Häusler, G. (1972). A method to increase the depth of focus by two step image processing. Optics Communications, 6(1), 38-42.Google Scholar
Horn, B. (1986). Robot Vision. MIT Press.
Horstmeyer, R., Euliss, G. W., Athale, R. A. & Levoy, M. (2009). Flexible multimodal camera using a light field architecture. In IEEE International Conference on Computational Photography, pp. 1-8.
Ihrke, I., Wetzstein, G. & Heidrich, W. (2010). A theory of plenoptic multiplexing. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 483–90.
Joshi, N., Kang, S. B., Zitnick, C. L. & Szeliski, R. (2010). Image deblurring using inertial measurement sensors. In ACM Special Interest Group on Graphics and Interactive Techniques, 29(4), 30:1–9.Google Scholar
Krishnan, D. & Fergus, R. (2009). Dark flash photography. In ACM Special Interest Group on Graphics and Interactive Techniques, 28(3), 96:1–11.Google Scholar
Kuthirummal, S., Nagahara, H., Zhou, C. & Nayar, S. K. (2010). Flexible depth of field photography. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1), 58-71.CrossRefGoogle Scholar
Lanman, D., Raskar, R., Agrawal, A. & Taubin, G. (2008). Shield fields: modeling and capturing 3D occluders. In ACM Special Interest Group on Graphics and Interactive Techniques, 27(5), 131:1–10.Google Scholar
Levin, A., Fergus, R., Durand, F. & Freeman, W. T. (2007). Image and depth from a conventional camera with a coded aperture. In ACM Special Interest Group on Graphics and Interactive Techniques, 26(3), 70:1–9.Google Scholar
Levin, A., Sand, P., Cho, T., Durand, F. & Freeman, W. (2008). Motion-invariant photography. In ACM Special Interest Group on Graphics and Interactive Techniques, 27(3), 71:1–9.Google Scholar
Liang, C., Lin, T., Wong, B., Liu, C. & Chen, H. (2008). Programmable aperture photography: multiplexed light field acquisition. In ACM Special Interest Group on Graphics and Interactive Techniques, 27(3), 55:1–10.Google Scholar
Ojeda-Castaneda, J., Landgrave, J. E. A. & Escamilla, H. M. (2005). Annular phase-only mask for high focal depth. Optics Letters, 30(13), 1647–9.Google Scholar
Raskar, R., Agrawal, A. & Tumblin, J. (2006). Coded exposure photography: motion deblurring using fluttered shutter. In ACM Special Interest Group on Graphics and Interactive Techniques, 25(3), 795–804.Google Scholar
Ratner, N. & Schechner, Y. (2007). Illumination multiplexing within fundamental limits. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8.
Ratner, N., Schechner, Y. & Goldberg, F. (2007). Optimal multiplexed sensing: bounds, conditions and a graph theory link. Optics Express, 15, 17072–92.Google Scholar
Schechner, Y., Nayar, S. & Belhumeur, P. (2007). Multiplexing for optimal lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(8), 1339–54.CrossRefGoogle Scholar
Shan, Q., Jia, J. & Agarwala, A. (2008). High-quality motion deblurring from a single image, In ACM Special Interest Group on Graphics and Interactive Techniques, 27(3), 73:1–10.Google Scholar
Sheikh, H. & Bovik, A. (2006). Image information and visual quality. IEEE Transactions on Image Processing, 15(2), 430–44.Google Scholar
Tendero, Y. (2012). Mathematical theory of the flutter shutter. Ph.D. thesis, École normale supérieure de Cachan.
Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A. & Tumblin, J. (2007). Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing. In ACM Special Interest Group on Graphics and Interactive Techniques, 26(3), 69:1–11.
Wang, Z. & Bovik, A. (2002). A universal image quality index. Signal Processing Letters, 9(3), 81–4.Google Scholar
Wang, Z., Bovik, A., Sheikh, H. & Simoncelli, E. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–12.CrossRefGoogle Scholar
Wikipedia (2013a). Lux “http://en.wikipedia.org/wiki/Lux”.
Wikipedia (2013b). Projection slice theorem. “http://en.wikipedia.org/wiki/Projection-slice_theorem”.
Yuan, L., Sun, J., Quan, L. & Shum, H.-Y. (2007). Image deblurring with blurred/noisy image pairs. In ACM Special Interest Group on Graphics and Interactive Techniques, 26(3), 1:1–10.Google Scholar
Zhang, L., Deshpande, A. & Chen, X. (2010). Denoising versus deblurring: HDR techniques using moving cameras. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 522–9.
Zhou, C., Lin, S. & Nayar, S. (2011). Coded aperture pairs for depth from defocus and defocus deblurring. International Journal of Computer Vision, 93(1), 53-72.Google Scholar
Zhou, C. & Nayar, S. (2009). What are good apertures for defocus deblurring? In IEEE International Conference on Computational Photography, pp. 1-8.

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×