Rendering: Appearance Modeling in Vision and Graphics

Spring 2006

Course Information

Title : E6161-01: Rendering: Appearance Modeling in Vision and Graphics, Spring 2006

Time : Wednesdays, 1:10 pm -- 3:00 pm

Location : 620 CEPSR, Schapiro Building

Credits : 3 credits

Pre-requisites : Background in Computer Science, Engineering, or Applied Math

Web page : http://www.cs.columbia.edu/~belhumeur/appearance/

Instructor

Prof. Peter N. Belhumeur

http://www.cs.columbia.edu/~belhumeur

Email: belhumeur@cs.columbia.edu

Office: 623 CEPSR, Schapiro Building

Phone: 212-939-7087

Teaching Assistant

Jinwei Gu

Email: jwgu@cs.columbia.edu
Phone: 212-939-7091

Office: 617 CEPSR
Office Hours: Monday, 9:30am-10:30am and Thursday, 2:00pm-3:00pm

Overview

This is a reading course on topics in computer rendering, focusing on appearance modeling in vision and graphics. The course is intended for advanced undergraduate or graduate students. We assume students have a background in computer science, engineering, physics, or applied math. There will be no homework assignments other than readings, class presentations, and a final project. There will be no exams. The readings will come from various texts and journals and will be assigned as the course progresses. In addition we will take a mid-semester trip to the Metropolitan Museum to view important works of painting and sculpture.

Topics

Fundamentals of Appearance

Cameras, Imaging, and Optics
Reflection, Refraction, Diffraction
Surface Reflection Mechanisms
Signal Processing framework for Reflection
Light Transport: Reflection, Radiosity, and Global Illumination
Modeling Illumination
3-D Geometry
Textures and Spatially Varying BRDFs: BTF
Subsurface Scattering BSSRDFS
(Interreflections)
(Scattering and Volumetric Light Transport)
(Fluids)

Algorithms and Applications

Photometric 'Shape-from-X' algorithms
(Appearances of Transparent, Transluscent, Wet, Woven Surfaces)
(Appearances of Fluids - smoke, fire, water)
Image Mosaics
Image-Based Rendering
View Synthesis
Motion Estimation and Warping
Single and Multi-View Geometry
3-D Reconstruction
Texture and Reflectance Modeling
Environment Matting and Compositing
Light Field Rendering
Face Analysis and Synthesis
Appearance-Based Face and Object Recognition

Grading

Class Presentations and Discussion 40%
Project 60%

Lecture Presentations and Reading Material

[Acknowledgements]

Parts of this course are similar to courses offered at CMU (Srinivasa Narasimhan), Stanford (Pat Hanrahan, Marc Levoy, Ron Fediw), UC San Diego (Henrik Wann Jensen), David Jacobs (UMD), David Lowe (UBC), UW Madison (Chuck Dyer), UWash (Steve Seitz), Utah (Pete Shirley), Rutgers (Kristin Dana), Cornell (Steve Marschner, Kavita Bala), Technion (Yoav Schechner), Princeton (Szymon Rusinkiewicz), MIT (Ted Adelson), Ko Nishino (Drexel), Rahul Swaminathan (Deutsch Telecom). The instructor thanks the instructors of these courses for the materials (slides, content) used in this course. In addition, several photographs and illustrations are borrowed from internet sources.

Museum Assignment

Final Project

Final Project Description

Syllabus + Assigned Reading Material (to be updated throughout the course)

Class 1: Introduction PPT

Class 2: Basics of Imaging and Radiometry PPT

E. Trucco and A. Verri Chapter 2 in Introductory Techniques for 3-D Computer Vision, pp. 15--26, Prentice Hall.

F. X. Sillion and C. Puech Chapters 1 and 2 in Radiosity and Global Illumination, Morgan Kaufmann Publishers Inc. 1994.,

Class 3: Principles and Surface Reflection + Models for Specular Surfaces + Split Off-Specular Surfaces PPT

F. E. Nicodemus, J.C. Richmond and J.J. Hsia, Geometrical Considerations and Nomenclature for Reflectance, Institute of Basic Standards, National Bureau of Standards, October 1977.

R. L. Cook and K. E. Torrance, A reflection model for computer graphics, SIGGRAPH, Vol. 15, No. 3, pp. 307—316, 1981.

S. C. Pont and J. J. Koenderink, Split off-specular reflection and surface scattering from woven materials, Journal of Applied Optics, Vol. 42, No. 8, 2003. Presented by Kshitiz Garg.

Class 4: Models for Diffuse Surfaces + Models for Dust PPT + Measuring BRDFs PPT

M. Oren and S.K Nayar, Generalization of the Lambertian model and its implications for machine vision,, IJCV, Vol 14, pp 227—251, 1996. Presented by Patrick Brown.

S. Marschner et al., Image-Based BRDF Measurement Including Human Skin.

Class 5: Scattering (Part I) PPT

P. Hanrahan and W. Krueger, Reflection from Layered Surfaces due to Subsurface Scattering, SIGGRPAH 1993. Presented by Jinwei Gu.

H. Wann Jensen et al., A Practical Model for Subsurface Light Transport, SIGGRAPH, 2001. Presented by David Harmon.

Class 6: Scattering (Part II) + Measuring BRDFs PPT

Sun et al., A Practical Analytic Single Scattering Model for Real Time Rendering, SIGGRAPH, 2005. Presented by Bo Sun.

J. Blinn, Light reflection functions for simulation of clouds and dusty surfaces, SIGGRAPH, pp 21—29, 1982. Presented by Kalyan Sunkavali.

Class 7: Textures and Texture Mapping PPT

P. S. Heckbert Survey of Texture Mapping, (figures), IEEE Computer Graphics and Applications, Nov. 1986.

J. F. Blinn, Simulation of Wrinkled Surfaces, IEEE Computer Graphics and Applications, Nov. 1978. Presented by Alex Severin.

Dana et al., Reflectance and Texture of Real World Surfaces, ACM Trans. On Graphics, 1999.

A. Efros and W. Freeman, Image Quilting for Texture Synthesis and Transfer, SIGGRAPH 2001.

J. Gu et al., Time Varying Textures, to be published, 2006. Presented by Jinwei Gu.

· Class 8: Metropolitan Museum Field Trip -- Meet Outside on Top Steps of Front Entrance at 12:00pm (Directions: http://www.metmuseum.org/visitor/index.asp#directions)

Museum Assignment is due on the day of the Field Trip (March 8^th)

Young Woman with a Water Pitcher, ca. 1662
Johannes Vermeer (Dutch, 1632–1675)

http://www.metmuseum.org/Works_of_Art/viewOne.asp?dep=11&viewmode=0&item=89.15.21

· Class 9: Guest Lecturer on Texutre (Jinwei Gu)

· Class 10: Multiview Geometry, Stereo, and Motion PPT

Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2000.

Olivier Faugeras, Quang-Tuan Luong, and T. Papadopoulo, The Geometry of Multiple Images, MIT Press, 2001.

C. Tomasi and T. Kanade, Shape and Motion from Image Streams: A Factorization Method, IJCV, 9(2), 1992, 137-154.

Longuet-Higgins, A computer algorithm for reconstructing a scene from two projections, Nature, Vol. 293, 1981, pp. 133-135.

Class 11 : View Morphing PPT and 3D Image-Based Modeling

E. Chen and L. Williams, View Interpolation for Image Synthesis, SIGGRAPH 1993.

S. Seitz and C. Dyer, View Morphing SIGGRAPH, pp. 21--30, 1996.

C.J. Taylor and D. Kriegman, Structure and motion from line segments in multiple images, IEEE Trans. PAMI, Vol 17, 11, pp 1021-1033, Nov 1995. Presented by David Lariviere.

Paul E. Debevec and C.J.Taylor and Jitendra Malik, Modeling and Rendering Architecture from Photographs:A hybrid geometry-and image-based approach, SIGGRAPH 1996, pp11-20. Presented by David Lariviere.

Class 12 : Plenoptic Function and Light Field Rendering

E. H. Adelson and J. R. Bergen. The plenoptic function and the elements of early vision. In M. Landy and J. A. Movshon, editors, Computational Models of Visual Processing, pages 3-20. MIT Press, Cambridge, MA, 1991. Presented by John Arroyo.

S. Chen, Quicktime VR - an image-based approach to virtual environment navigation, pages 29-38, SIGGRAPH, 1995.

S. J. Gortler, R. Grzeszczuk, R. Szeliski and M. F. Cohen The Lumigraph, SIGGRAPH, pp 43--54, 1996. Presented by Sharmila Gupta.

M. Levoy and P. Hanrahan, Light Field Rendering , SIGGRAPH, 1996. Presented by Sharmila Gupta.

Class 13 : Relighting PPT PPT with movies and Face Synthesis

M. Koudelka, P. Belhumeur, S. Magda and D. Kriegman Image-based Modeling and Rendering of Surfaces with Arbitrary BRDFs, IEEE CVPR, pp. 568--575, 2001.

S. Nayar, P. Belhumeur, and T. Boult, Lighting-Sensitive Displays, ACM TOG, 2003.

D. Mahajan, R. Ramamoorthi, and P. Belhumeur, A First Order Analysis of Lighting, Shading, and Shadows, to appear in TOG, 2006. Presented by Dhruv Mahajan.

P. Debevec et al., Acquiring the Reflectance Field of a Human Face, SIGGRAPH, 2000. Presented by Sean West.

V. Blanz and T. Vetter, A Morphable Model for the Synthesis of 3D Faces, SIGGRAPH, 1999. Presented by Nandan Dixit.

Class 14 : Class Presentations