Skip to main content Accessibility help
×
Hostname: page-component-78c5997874-lj6df Total loading time: 0 Render date: 2024-11-06T00:58:21.651Z Has data issue: false hasContentIssue false

10 - Compressive video sensing to tackle motion blur

Published online by Cambridge University Press:  05 June 2014

Ashok Veeraraghavan
Affiliation:
Rice University
Dikpal Reddy
Affiliation:
Rice University
A. N. Rajagopalan
Affiliation:
Indian Institute of Technology, Madras
Rama Chellappa
Affiliation:
University of Maryland, College Park
Get access

Summary

Introduction

Spatial resolution of imaging devices is steadily increasing: mobile phone cameras have 5–10MP while point-and-shoot cameras have 12–18MP. As the spatial resolution of imaging devices increases, the effect of either camera or subject motion on the captured images is magnified, resulting in the acquisition of heavily blurred images. Since the motion-blur kernel is a low pass kernel, the high frequencies in the image are heavily attenuated. Deblurring the effects of such low pass blur kernels results in the introduction of significant high frequency artifacts. When the size of the blur kernel is small, these artifacts can be mitigated by the use of appropriate image regularizers that allow for the hallucination of the high frequency detail. Unfortunately there are several scenarios in which such a technique, relying primarily on image regularization, cannot be directly applied. They fall into the following three main categories.

1. Large motion

The resolutions of image sensors are rapidly increasing, while the hands of photographers are not becoming any steadier. This results in ever larger sizes of blur kernels. While image regularization allows us to handle blur kernels of a moderate size (say about 10–15 pixels), larger blur kernels test the ability of even modern deblurring algorithms, especially in regions of high texture. Such large motion blur kernels necessitate active approaches that shape the blur kernel to be invertible in order to handle these robustly.

2. Object motion

Object motion causes blur that is spatially variant. In particular, the blur kernel depends on the local velocity of the objects.

Type
Chapter
Information
Motion Deblurring
Algorithms and Systems
, pp. 207 - 221
Publisher: Cambridge University Press
Print publication year: 2014

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Agrawal, A., Gupta, M., Veeraraghavan, A. & Narasimhan, S. (2010). Optimal coded sampling for temporal super-resolution. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 599–606.
Baker, S. & Kanade, T. (2000). Limits on super-resolution and how to break them. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2372–9.
Ben-Ezra, M. & Nayar, S. K. (2004). Motion-based motion deblurring. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 689–98.Google Scholar
Borman, S. & Stevenson, R. (1998). Super-resolution from image sequences – a review. In IEEE Proceedings of the Midwest Symposium on Circuits and Systems, pp. 374–8.
Bub, G., Tecza, M., Helmes, M., Lee, P. & Kohl, P. (2010). Temporal pixel multiplexing for simultaneous high-speed, high-resolution imaging. Nature Methods, 7(3), 209–11.Google Scholar
Duarte, M., Davenport, M., Takhar, D., Laska, J., Sun, T., Kelly, K. & Baraniuk, R. (2008). Single-pixel imaging via compressive sampling. IEEE Signal Processing Magazine, 25(2), 83-91.Google Scholar
Fattal, R. (2007). Image upsampling via imposed edge statistics. ACM Transactions on Graphics, 26(3), 95:1–8.Google Scholar
Fergus, R., Singh, B., Hertzmann, A., Roweis, S. & Freeman, W. (2006). Removing camera shake from a single photograph. In ACM Special Interest Group on Graphics and Interactive Techniques, 25(3), 787–94.Google Scholar
Freeman, W., Jones, T. & Pasztor, E. (2002). Example-based super-resolution. IEEE Computer Graphics and Applications, 22(2), 56-65.Google Scholar
Glasner, D., Bagon, S. & Irani, M. (2010). Super-resolution from a single image. In IEEE 12th International Conference on Computer Vision, pp. 349–56.
Gu, J., Hitomi, Y., Mitsunaga, T. & Nayar, S. (2010). Coded rolling shutter photography: flexible space-time sampling. In IEEE International Conference on Computational Photography, pp. 1-8.
Gupta, M., Agrawal, A., Veeraraghavan, A. & Narasimhan, S. (2010). Flexible Voxels for Motion-Aware Videography. In European Conference on Computer Vision, pp. 100–14.
Hale, E., Yin, W. & Zhang, Y. (2007). A Fixed-Point Continuation Method for l1-Regularized Minimization with Applications to Compressed Sensing. CAAM Technical Report TR07-07, Rice University, Houston, Texas, USA.
Hitomi, Y., Gu, J., Gupta, M., Mitsunaga, T. & Nayar, S. K. (2011). Video from a single coded exposure photograph using a learned over-complete dictionary. In IEEE International Conference on Computer Vision, pp. 287–94.
Holloway, J., Sankaranarayanan, A., Veeraraghavan, A. & Tambe, S. (2012). Flutter shutter video camera for compressive sensing of videos. In IEEE International Conference on Computational Photography, pp. 1-9.
Levin, A., Sand, P., Cho, T., Durand, F. & Freeman, W. (2008). Motion-invariant photography. ACM Special Interest Group on Graphics and Interactive Techniques, 27(3), 71:1–9.Google Scholar
Lin, Z. & Shum, H. (2004). Fundamental limits of reconstruction-based super-resolution algorithms under local translation. In IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 83-97.
Liu, C. (2009). Beyond pixels: exploring new representations and applications for motion analysis, PhD thesis, Massachusetts Institute of Technology, Cambridge, MA, USA.
Marcia, R. F. & Willett, R. M. (2008). Compressive coded aperture video reconstruction. In Proceedings of the European Signal Processing Conference, Lausanne, Switzerland, pp. 1-5.
Park, J. & Wakin, M. (2009). A multiscale framework for compressive sensing of video. In IEEE Picture Coding Symposium, pp. 1-4.
Park, S., Park, M. & Kang, M. (2003). Super-resolution image reconstruction: a technical overview. IEEE Signal Processing Magazine, 20(3), 21-36.Google Scholar
Raskar, R., Agrawal, A. & Tumblin, J. (2006). Coded exposure photography: motion deblurring using fluttered shutter. In ACM Special Interest Group on Graphics and Interactive Techniques, 25(3), 795–804.Google Scholar
Reddy, D., Veeraraghavan, A. & Chellappa, R. (2011). P2C2: Programmable pixel compressive camera for high speed imaging. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 329–36.
Sankaranarayanan, A., Turaga, P., Baraniuk, R. & Chellappa, R. (2010). Compressive acquisition of dynamic scenes. In European Conference on Computer Vision, pp. 129–42.
Shan, Q., Jia, J. & Agarwala, A. (2008). High-quality motion deblurring from a single image. ACM Special Interest Group on Graphics and Interactive Techniques, 27(3), 73:1–10.Google Scholar
Shechtman, E., Caspi, Y. & Irani, M. (2005). Space-time super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 531–45.Google Scholar
Sun, J., Xu, Z. & Shum, H. (2008). Image super-resolution using gradient profile prior. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8.
Takeda, H. & Milanfar, P. (2011). Removing motion blur with space-time processing. IEEE Transactions on Image Processing, 20(10), 2990–3000.Google Scholar
Vaswani, N. (2008). Kalman filtered compressed sensing. In IEEE International Conference on Image Processing, pp. 893–6.
Veeraraghavan, A., Reddy, D. & Raskar, R. (2011). Coded strobing photography: compressive sensing of high speed periodic videos. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4), 671–86.Google Scholar
Wilburn, B., Joshi, N., Vaish, V., Talvala, E., Antunez, E., Barth, A., Adams, A., Horowitz, M. & Levoy, M. (2005). High performance imaging using large camera arrays. ACM Special Interest Group on Graphics and Interactive Techniques, 24(3), 765–76.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×