Computational Photography

Spring 2011

Course Information

Title: COMS 6734: Computational Photography, Spring 2011

Time: Wednesday, 2:10 pm -- 4:00 pm

Location: 825 Mudd

Credits: 3 credits

Pre-requisites: Linear Algebra, Calculus

Instructor

Prof. Peter N. Belhumeur

http://www.cs.columbia.edu/~belhumeur

Email: belhumeur@cs.columbia.edu

Office: 623 CEPSR

Phone: 212-939-7087

Office Hours: Mon, 8:30-10:00am

TA

Oliver Cossairt

Email: ollie@cs.columbia.edu

Office: 6LW4 CEPSR

Office Hours: Mon 10am-11:30am

Overview

In recent years, the fields of computer graphics, computer vision and photography have converged to give rise to a new and very active area of research -- Computational Photography. The goal in this work is to redefine the camera by using computational techniques to produce an new level of images and visual representations. This course will be seminar, offered to all students with knowledge in any of the three core areas: computer vision, computer graphics, or photography. There will be no midterm or final exam.

Topics

         Introduction to the Camera

         HDR Imaging

         Feature Matching Using Ransac

         Image Mosaics, Image Stitching, and Dynamosaics

         Image-Based Rendering, Environment Matting and Compositing

         Image Refocusing

         Motion Magnification

         Removing Camera Shake

         Camera Lens Arrays

         Bluescreening

         Programmable Lighting

         Computational Flash Photography

         Light Fields

         Photo Pop-Up

         Schematic Storyboarding

         Face Detection

         Face Modeling

         Texture Synthesis

         Video Textures

         View Synthesis

         Motion Estimation and Warping

         Single and Multi-View Geometry

         Photo Tourism

Grading

         Class Presentations and Discussion: 25%. Some simple tips about presentations

         Paper Review: 10%

         Assignments: 25% Assignment 1 Assignment 2

         Project: 40% Project Description(In-class Presentation: April 29th)

Paper Review Form: TXT

Syllabus + Assigned Reading Material (to be updated throughout the course)

Lecture 1. Introduction

         Syllabus Review + Computational Photography Examples:

o   Photo Tourism Movie

o   Campanile Movie, webpage

o   Gigapan demo of inauguration

Lecture 2. Basics of Imaging, Cameras, and Radiometry (Resolution and HDR)PPT + PPT

Readings

         E. Trucco and A. Verri Chapter 2 in Introductory Techniques for 3-D Computer Vision, pp. 15--26, Prentice Hall.

         F. X. Sillion and C. Puech Chapters 1 and 2 on Radiosity and Global Illumination, Morgan Kaufmann Publishers Inc. 1994.,

         F. E. Nicodemus, J.C. Richmond and J.J. Hsia, Geometrical Considerations and Nomenclature for Reflectance, Institute of Basic Standards, National Bureau of Standards, October 1977.

         Debevec and Malik, Recovering High Dynamic Range Radiance Maps from Photographs, SIGGRAPH 1997.PDF

         Radiometry and photometry FAQ

Lecture 3. Plenoptic Function and Light Field Rendering PPT

Presentations

         Isaksen et al., Dynamically Reparameterized Light Field, SIGGRAPH 2000.Assigned to

         Levoy and Hanrahan, Light Field Rendering , SIGGRAPH, 1996. Assigned to

Readings

         Gortler et al., The Lumigraph, SIGGRAPH, pp 43--54, 1996.

         Adelson and Bergen, The plenoptic function and the elements of early vision.. Computational Models of Visual Processing, MIT Press, Cambridge MA, 1991.

         Chen, Quicktime VR - an image-based approach to virtual environment navigation, SIGGRAPH 1995.

         Ng et al. Light Field Photography with a Hand-Held Plenoptic Camera, Stanford Tech. Report, 2005.

         Andrew Jones, et al. Rendering for an Interactive 360?Light Field Display, SIGGRAPH 2007.

Lecture 4. Image Deblurring PPT

Presentations

         Levin et al.. Motion Invariant Photography, SIGGRAPH 2008.  Assigned to Guangyu Sun and Zhiwei Wang

         Nagahara et al. Flexible Depth of Field. SIGGRAPH 2008.  Assigned to Erbo Li and Pranay Prabhakar

         Cossairt et al. Spectral Focal Sweep. ICCP 2010.  Assigned to

Lecture 5. Flash Photography

Presentations

         Petschnigg et al. Digital Photography with Flash and No-flash Image Pairs. SIGGRAPH 2004.  Assigned to Delia Wang and Astha Malik

         Raskar et al. Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering using Multi-Flash Imaging. SIGGRAPH 2004.  Assigned to Pei-Lun Hsieh and Wenxin Zhu

         Nayar, et al. Fast Separation of Direct and Global Components of a Scene using High Frequency Illumination, SIGGRAPH 2006.  Assigned to

Readings

         Kundur and Hatzinakos. Blind Image Deconvolution. IEEE Transactions on Signal Processing.

Lecture 6. Recent Work on Photograph Enhancement

Presentations

         Raanan Fattal. Single Image Dehazing. SIGGRAPH 2008 Assigned to Yifu Zheng and Chun-Kang Chen

         Kaiming He, et al. Single Image Haze Removal using Dark channel Prior. CVPR 2009.

         Eugene Hsu, et al. Light Mixture Estimation for Spatially Varying White Balance. SIGGRAPH 2008. Assigned to ziyi zhang and Erbo Li

Readings

         Eino-Ville Talvala, et al. Veiling Glare in High Dynamic Range Imaging. SIGGRAPH 2007.

         Johannes Kopf, et al. Deep Photo: Model-Based Photograph Enhancement and Viewing. SIGGRAPH 2008 Asia. Assigned to Yi Zhang and Hang Li

Lecture 7. Image and Video Matting

Presentations

         J. Sun et al. Flash Matting. SIGGRAPH 2006. Assigned to Joshua Cataldo and Shih-Hao Tsai

         Bando et al. Extracting Depth and Matte using a Color-Filtered Aperture. SIGGRAPH ASIA 2008. Assigned to Pushkaraj Bhirud and Siddharth Patil

Readings

         Morgan McGuire, et al. Defocus Video Matting. SIGGRAPH 2005.

         Y.-Y. Chuang, et al. A Bayesian Approach to Digital Matting. CVPR 2001.

         Y.-Y. Chuang, et al. Video Matting of Complex Scenes. SIGGRAPH 2002.

         J. Wang and M. Cohen. An iterative optimization approach for unified image segmentation and matting. ICCV 2005

Lecture 8. Large Photo Collections (March 23)

Presentations

Levin, D. Lischinski, and Y. Weiss. A Closed form Solution to Natural Image Matting. PAMI 2008. Assigned to Nikhil Mhatre and Kaushik Kaul

Zeng et al. From Image Parsing to Painterly Rendering, TOG 2009. Assigned to Neerja Pancholi and Pei-Lun Hsieh


Readings

Hiroya Tanaka, PhotoWalker

Michael Goesele, et al. Multi-View Stereo for Community Photo Collections. ICCV 2007

Torralba, et al. 80 Million Tiny Images: a large dataset for non-parameteric object and scene recognition.

Lecture 9. Image Editting (March 30)

Presentations

Shai Avidan and Ariel Shamir. Seam Carving. SIGGRAPH 2007. Assigned to Katherine Scott and Chun-Kang Chen

P. Perez. Possion Image Editing, SIGGRAPH 2003. Assigned to Austin Reiter and Joshua Cataldo

Readings

A. Hertzman et al. Image Analogies, SIGGRAPH 2001.

Liu et al. Content-Preserving Warps for 3D Video Stabilization, Siggraph 2009.

Antonio Criminisi, et al. Object Removal by Exemplar-based Inpainting, CVPR 2003.

Yu-Shuen Wang, et al. Optimized Scale-and-Stretch for Image Resizing. SIGGRAPH Asia 2008.

Michael Rubinstein, et al. Improved Seam Carving for Video Retargeting, SIGGRAPH 2008.

Lecture 10. Computational Photography for Faces (April 6)

Presentations

V. Blanz and T. Vetter, A Morphable Model for the Synthesis of 3D Faces, SIGGRAPH 1999. Assigned to Zhiwei Wang and ziyi zhang

Nguyen et al. Image-based Shaving. Eurographics 2008. Assigned to Shih-Hao Tsai and Yifu Zheng

Readings

       Dmitri Bitouk, et al. Face Swapping: Automatic Face Replacement in Photographs. SIGGRAPH 2008.

Viola and Jones, Robust Real-time Face Detection, IJCV 2004.

Lecture 11. Image Relighting (April 13)

Presentations

N. Kumar, et al. FaceTracer: A Search Engine for Large Collections of Images with Faces. ECCV 2008. Assigned to Hang Li and Yi Zhang

Nishino and Nayar. Eyes for Relighting, SIGGRAPH 2004.  Assigned to Pranay Prabhakar and Austin Reiter

Readings

Debevec et al. Performance Relighting and Reflectance Transformation with Time-Multiplexed Illumination, SIGGRAPH 2005. 

Lalonde et al. Estimating Natural Illumination from a Single Outdoor Image, ICCV2009.

Debevec. Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography,SIGGRAPH 1998.

Lecture 12. Image Matching and Stitching (April 20)

Presentations

Hoiem et al. Automatic Photo Pop-up, SIGGRAPH 2005. Assigned to Katherine Scott and Delia Wang

Y. Nomura, et al. Scene Collages and Flexible Camera Arrays. EGSR 2007. Assigned to Siddharth Patil and Pushkaraj Bhirud

Readings

M. Fischler et al. RANSAC, Comm. of ACM, 1981.

Agarwala et al. Photographing Long Scenes with Multi-Viewpoint Panoramas, 2006.

Russell et al. Segmenting Scenes by Matching Image Composites. NIPS 2009.

R. Szeliski, Image Alignment and Stitching: A Tutorial.

D. Lowe, Object Recognition from Local Scale-Invariant Features, ICCV 1999.

Lecture 13. (In-class Presentations for Final Projects) (April 27)

 

 

 

 

 

Last Modified: