Peter N. Belhumeur
Columbia University Department of Computer Science, Professor
Research
Peter N. Belhumeur is currently a Professor in the Department of Computer Science at Columbia University and the Director of the Laboratory for the Study of Visual Appearance (VAP LAB). He received a Sc.B. in Information Sciences from Brown University in 1985. He received his Ph.D. in Engineering Sciences from Harvard University under the direction of David Mumford in 1993. He was a postdoctoral fellow at the University of Cambridge's Isaac Newton Institute for Mathematical Sciences in 1994. He was made Assistant, Associate and Professor of Electrical Engineering at Yale University in 1994, 1998, and 2001, respectively. He joined Columbia University as a Professor of Computer Science in 2002. His research focus lies somewhere in the mix of computer vision, computer graphics, and computational photography. He is a recipient of the Presidential Early Career Award for Scientists and Engineers (PECASE) and the National Science Foundation Career Award. He won both the Siemens Best Paper Award at the IEEE Conference on Computer Vision and Pattern Recognition and the Olympus Prize at the European Conference of Computer Vision.
New York Times article on our Digital Field Guide
Courses
- Fall 2013 Course: Biometrics
- Spring 2013 Course: Computer Vision and Machine Learning on Mobile Platforms
Research Scientists
Postdoctoral Students
- Tolga Eren
- Francesc Moreno-Noguer
Students
- Thomas Berg
- Jiongxin Liu
- Jinwei Gu
- Dhruv Kumar Mahajan
- Nandan Dixit
- Athinodoros Georghiades
- Hansen Chen
- Melissa Koudelka
- Todd Zickler
- Tolga Eren
- Neeraj Kumar
Databases
The extended Yale Face Database B contains 16128 images of 28
human subjects under 9 poses and 64 illumination conditions.
The data format of this database is the same as the Yale Face
Database B.
The database contains 5760 single light source images of 10
subjects each seen under 576 viewing conditions (9 poses x 64
illumination conditions). For every subject in a particular
pose, an image with ambient (background) illumination was also
captured. Hence, the total number of images is in fact
5760+90=5850. The total size of the compressed database is
about 1GB.
The Yale Face Database (size 6.4MB) contains 165 grayscale
images in GIF format of 15 individuals. There are 11 images per
subject, one per different facial expression or configuration:
center-light, w/glasses, happy, left-light, w/no glasses,
normal, right-light, sad, sleepy, surprised, and wink.
Our project aims to build a first generation of electronic
field guides: computing devices that allow a taxonomist in the
field access to critical comparative information on plant
species.
A novel relighting algorithm has been developed that uses a
compact representation of a large set of images of the scene
that correspond to different lighting conditions. Unlike
previous relighting algorithms, this one exploits not only
image correlations over the illumination dimensions but also
coherences over the spatial dimensions of the image. This
enables the algorithm to achieve high quality relighting in
real time. It can render 640×480 images of scenes under
complex and varying illuminations at 15 frames per second using
a 2GHz processor. This algorithm was used to develop a Lighting
Sensitive Display that can render a 3D scene such that it
always appears to be lit by the real environment that the
display resides in.
Traditional computer graphics rendering generally assumes that
the appearance of surfaces remains static over time. Yet, there
are a number of natural processes that cause surface appearance
to vary dramatically, such as burning of wood, wetting and
drying of rock and fabric, decay of fruit skins, or corrosion
and rusting of steel and copper. To investigate time-varying
surface appearance, we formulate this problem as TSV-BRDF
(Time-and-Space-Varying BRDF).
The properties of virtually all real-world materials change
with time, causing their BRDFs to be time-varying. However,
none of the existing BRDF models and databases take time
variation into consideration; they represent the appearance of
a material at a single time instance. In this work, we address
the acquisition, analysis, modeling and rendering of a wide
range of time-varying BRDFs.
Type Specimen Register of the US National Herbarium The Type
Specimen Register of the United States National Herbarium was
begun in 1966 and contains images and data for more than 90,000
type specimens of algae, lichens, bryophytes, ferns,
gymnosperms and angiosperms. Yet to be imaged are the lichens,
bryophytes and algae, as well as any type that has been on loan
since before the start of the project. Types that have been
imaged are indicated with a bold letter ‘I’ at the
end of the record.
Projects
2008
We describe a working computer vision system that aids in the
identification of plant species. A user photographs an isolated
leaf on a blank background, and the system extracts the leaf
shape and matches it to the shape of leaves of known species.
In a few seconds, the system displays the top matching species,
along with textual descriptions and additional images. This
system is currently in use by botanists at the Smithsonian
Institution National Museum of Natural History. The primary
contributions of this paper are: a description of a working
computer vision system and its user interface for an important
new application area; the introduction of three new datasets
containing thousands of single leaf images, each labeled by
species and verified by botanists at the US National Herbarium;
recognition results for two of the three leaf datasets; and
descriptions throughout of practical lessons learned in
constructing this system
We have created the first image search engine based entirely on
images with faces. Users can search through our database of
over 1 million images (containing over 2 million faces) using
simple text queries. Faces are automatically detected,
extracted, and aligned from the original images using a
commercial face detector. They are then labeled on the basis of
various attributes using a novel combination of Adaboost and
Support Vector Machines. We compare against prior works on
attribute classification and show state-of-the-art
classification results using our method. Our framework is fully
automatic, easily extensible, and computes all labels off-line,
leading to very fast on-line search performance. We show the
results of various searches on two fully functional systems
– an internet image search engine and a personal photo
organizer. Our image and face datasets (including a large
number of manually labeled faces), as well as our image search
engine, will be made publicly available at the time of
publication.
In this project, we present a complete system for automatic
face replacement in images. Our system uses a large library of
face images created automatically by downloading images from
the internet, extracting faces using face detection software,
and aligning each extracted face to a common coordinate system.
Complex reflectance phenomena such as specular reflections
confound many vision problems since they produce image
‘features’ that do not correspond directly to
intrinsic surface properties such as shape and spectral
reflectance. A common approach to mitigate these effects is to
explore functions of an image that are invariant to these
photometric events. In this paper we describe a class of such
invariants that result from exploiting color information in
images of dichromatic surfaces. These invariants are derived
from illuminant-dependent ‘subspaces’ of RGB color
space, and they enable the application of Lambertian-based
vision techniques to a broad class of specular, non-Lambertian
scenes. Using implementations of recent algorithms taken from
the literature, we demonstrate the practical utility of these
invariants for a wide variety of applications, including
stereo, shape from shading, photometric stereo, material based
segmentation, and motion estimation.
We propose a new method named compressive structured light for
recovering inhomogeneous participating media. Whereas
conventional structured light methods emit coded light patterns
onto the surface of an opaque object to establish
correspondence for triangulation, compressive structured light
projects patterns into a volume of participating medium to
produce images which are integral measurements of the volume
density along the line of sight. For a typical participating
medium encountered in the real world, the integral nature of
the acquired images enables the use of compressive sensing
techniques that can recover the entire volume density from only
a few measurements. This makes the acquisition process more
efficient and enables reconstruction of dynamic volumetric
phenomena. Moreover, our method requires the projection of
multiplexed coded illumination, which has the added advantage
of in- creasing the signal-to-noise ratio of the acquisition.
Finally, we propose an iterative algorithm to correct for the
attenuation of the participating medium during the
reconstruction process. We show the effectiveness of our method
with simulations as well as experiments on the volumetric
recovery of multiple translucent layers, 3D point clouds etched
in glass, and the dynamic process of milk drops dissolving in
water.
2007
In this work, we address the acquisition, analysis, modeling
and rendering of a wide range of time-varying BRDFs. We have
developed an acquisition system that is capable of sampling a
material’s BRDF at multiple time instances, with each
time sample acquired within 36 seconds. We have used this
acquisition system to measure the BRDFs of a wide range of
time-varying phenomena which include the drying of various
types of paints (watercolor, spray, and oil), the drying of wet
rough surfaces (cement, plaster, and fabrics), the accumulation
of dusts (household and joint compound) on surfaces, and the
melting of materials (chocolate). Analytic BRDF functions are
fit to these measurements and the model parameters variations
with time are analyzed.
We derive a complete first order or gradient theory of
lighting, reflection and shadows, taking both spatial and
angular variation of the light field into account. The gradient
is by definition a sum of terms, allowing us to consider the
relative weight of spatial and angular lighting variation,
geometric curvature and bump mapping. Moreover, we derive
analytic formulas for the gradients in soft shadow or penumbra
regions, demonstrating applications to gradient-based
interpolation and sampling.
Rendering of clean transparent objects has been well studied in
computer graphics. However, real-world transparent objects are
seldom clean-their surfaces have a variety of contaminants such
as dust, dirt, and lipids. These contaminants produce a number
of complex volumetric scattering effects that must be taken
into account when creating photorealistic renderings. In this
paper, we take a step toward modeling and rendering these
effects. We make the assumption that the contaminant is an
optically thin layer and construct an analytic model following
results in radiative transport theory and computer graphics.
Moreover, the spatial textures created by the different types
of contamination are also important in achieving visual
realism. To this end, we measure the spatially varying
thicknesses and the scattering parameters of a number of glass
panes with various types of dust, dirt, and lipids. We also
develop a simple interactive synthesis tool to create novel
instances of the measured contamination patterns. We show
several results that demonstrate the use of our scattering
model for rendering 3D scenes, as well as modifying real 2D
photographs.
We present a system for refocusing images and videos of dynamic
scenes using a novel, single-view depth estimation method. Our
method for obtaining depth is based on the defocus of a sparse
set of dots projected onto the scene. In contrast to other
active illumination techniques, the projected pattern of dots
can be removed from each captured image and its brightness
easily controlled in order to avoid under- or over-exposure.
The depths corresponding to the projected dots and a color
segmentation of the image are used to compute an approximate
depth map of the scene with clean region boundaries. The depth
map is used to refocus the acquired image after the dots are
removed, simulating realistic depth of eld effects. Experiments
on a wide variety of scenes, including close-ups and live
action, demonstrate the effectiveness of our method.
The properties of virtually all real-world materials change
with time, causing their bidirectional reflectance distribution
functions (BRDFs) to be time varying. However, none of the
existing BRDF models and databases take time variation into
consideration; they represent the appearance of a material at a
single time instance. In this paper, we address the
acquisition, analysis, modeling, and rendering of a wide range
of time-varying BRDFs (TVBRDFs). We have developed an
acquisition system that is capable of sampling a
material’s BRDF at multiple time instances, with each
time sample acquired within 36 sec. We have used this
acquisition system to measure the BRDFs of a wide range of
time-varying phenomena, which include the drying of various
types of paints (watercolor, spray, and oil), the drying of wet
rough surfaces (cement, plaster, and fabrics), the accumulation
of dusts (household and joint compound) on surfaces, and the
melting of materials (chocolate). Analytic BRDF functions are
fit to these measurements and the model parameters’
variations with time are analyzed. Each category exhibits
interesting and sometimes nonintuitive parameter trends.
These parameter trends are then used to develop analytic TVBRDF models. The analytic TVBRDF models enable us to apply effects such as paint drying and dust accumulation to arbitrary surfaces and novel materials.
These parameter trends are then used to develop analytic TVBRDF models. The analytic TVBRDF models enable us to apply effects such as paint drying and dust accumulation to arbitrary surfaces and novel materials.
Imaging of objects under variable lighting directions is an
important and frequent practice in computer vision, machine
vision, and image-based rendering. Methods for such imaging
have traditionally used only a single light source per acquired
image. They may result in images that are too dark and noisy,
e.g., due to the need to avoid saturation of highlights. We
introduce an approach that can significantly improve the
quality of such images, in which multiple light sources
illuminate the object simultaneously from different directions.
These illumination-multiplexed frames are then computationally
demultiplexed. The approach is useful for imaging dim objects,
as well as objects having a specular reflection component. We
give the optimal scheme by which lighting should be multiplexed
to obtain the highest quality output, for signal-independent
noise. The scheme is based on Hadamard codes. The consequences
of imperfections such as stray light, saturation, and noisy
illumination sources are then studied. In addition, the paper
analyzes the implications of shot noise, which is
signal-dependent, to Hadamard multiplexing. The approach
facilitates practical lighting setups having high directional
resolution. This is shown by a setup we devise, which is
flexible, scalable, and programmable. We used it to demonstrate
the benefit of multiplexing in experiments.
2006
We have captured the first time-varying surface appearance
database (with 26 samples), which includes a variety of natural
processes – burning, drying, decay and corrosion. We have
developed a novel Space-Time Appearance Factorization (STAF)
model, which factors space- and time-varying appearance
effects. The STAF model includes an overall temporal appearance
variation characteristic function, two spatially varying
textures corresponding to the initial and final frames, and two
spatially varying textures corresponding to the rates and
offsets at each point on the material that determine the
evolution of the appearance over time.
Image-based object reconstruction is the process of estimating
the shape and surface reflectance properties on an object from
its images. Applications include graphics (accurate rendering
for virtual and augmented reality) and shape measurement
(reverse engineering, visual inspection, digital object
archival).
The sensor network localization problem is one of determining
the Euclidean positions of all sensors in a network given
knowledge of the Euclidean positions of some, and knowledge of
a number of inter-sensor distances. This paper identifies
graphical properties which can ensure unique localizability,
and further sets of properties which can ensure not only unique
localizability but also provide guarantees on the associated
computational complexity, which can even be linear in the
number of sensors on occasions. Sensor networks with minimal
connectedness properties in which sensor transmit powers can be
increased to increase the sensing radius lend themselves to the
acquiring of the needed graphical properties. Results are
presented for networks in both two and three dimensions.
We present a unifed framework for separating specular and
diffuse reflection components in images and videos of textured
scenes. This can be used for specularity removal and for
independently processing, filtering, and recombining the two
components. Beginning with a partial separation provided by an
illumination-dependent color space, the challenge is to
complete the separation using spatio-temporal information. This
is accomplished by evolving a partial differential equation
(PDE) that iteratively erodes the specular component at each
pixel. A family of PDEs appropriate for differing image sources
(still images vs. videos), differing prior information (e.g.,
highly vs. lightly textured scenes), or differing prior
computations (e.g., optical ow) is introduced. In contrast to
many other methods, explicit segmentation and/or manual
intervention are not required. We present results on
high-quality images and video acquired in the laboratory in
addition to images taken from the Internet. Results on the
latter demonstrate robustness to low dynamic range, JPEG
artifacts, and lack of knowledge of illuminant color. Empirical
comparison to physical removal of specularities using
polarization is provided. Finally, an application termed
dichromatic editing is presented in which the diffuse and the
specular components are processed independently to produce a
variety of visual effects.
It has been observed that the variations between the images of the same face due to lighting
and pose are almost always larger than image variations due to change in facial identity. The same person, with the same facial expression,
can appear strikingly different when light source direction and viewpoint vary. These variations
are made even greater by additional factors such as facial expression, perspiration, hair styles,
cosmetics, and even changes due to aging.
2005
By framing the problem as scattered-data interpolation in a
mixed spatial and angular domain, reflectance information is
shared across the surface, exploiting the high spatial
resolution that images provide to fill the holes between
sparsely observed view and lighting directions. Since the BRDF
typically varies slowly from point to point over much of an
object’s surface, this method enables image-based
rendering from a sparse set of images without assuming a
parametric reflectance model. In fact, the method can even be
applied in the limiting case of a single input image.
We present a photometric stereo method for non-diffuse
materials that does not require an explicit reflectance model
or reference object. By computing a data-dependent rotation of
RGB color space, we show that the specular reflection effects
can be separated from the much simpler, diffuse (approximately
Lambertian) reflection effects for surfaces that can be modeled
with dichromatic reflectance. Images in this transformed color
space are used to obtain photometric reconstructions that are
independent of the specular reflectance. In contrast to other
methods for highlight removal based on dichromatic color
separation (e.g., color histogram analysis and/or
polarization), we do not explicitly recover the specular and
diffuse components of an image. Instead, we simply find a
transformation of color space that yields more direct access to
shape information. The method is purely local and is able to
handle surfaces with arbitrary texture.
This paper is concerned with information structures used in
rigid formations of autonomous agents that have leader-follower
architecture. The focus of this paper is on sensor/network
topologies to secure control of rigidity. We extend our
previous approach for formations with symmetric neighbor
relations to include formations with leader-follower
architecture. Necessary and sufficient conditions for stably
rigid directed formations are given including both cyclic and
acyclic directed formations. Some useful steps for creating
topologies of directed rigid formations are developed. An
algorithm to determine the directions of links to create stably
rigid directed formations from rigid undirected formations is
presented. It is shown that k-cycles (k , 3) do not cause
inconsistencies when measurements are noisy, while 2-cycles do.
Simulation results are presented for (i) a rigid acyclic
formation, (i) a flexible formation, and (iii) a rigid
formation with cycles.
2004
Although display devices have been used for decades, they have
functioned without taking into account the illumination of
their environment. In this project, an initial step has been
taken towards addressing this limitation. We are exploring the
concept of a lighting sensitive display (LSD) – a display
that measures the surrounding illumination and modifies its
content accordingly.
We present a method for controlling the appearance of an
arbitrary 3D object using a projector and a camera. Our goal is
to make one object look like another by projecting a carefully
determined compensation image onto the object. The
determination of the appropriate compensation image requires
accounting for spatial variation in the object’s
reflectance, the effects of environmental lighting, and the
spectral responses, spatially varying fall-offs, and non-linear
responses in the projector-camera system. Addressing each of
these effects, we present a compensation method which calls for
the estimation of only a small number of parameters, as part of
a novel off-line radiometric calibration. This calibration is
accomplished by projecting and acquiring a minimal set of 6
images, irrespective of the object. Results of the calibration
are then used on-line to compensate each input image prior to
projection. Several experimental results are shown that
demonstrate the ability of this method to control the
appearance of everyday objects. Our method has direct
applications in several areas including smart environments,
product design and presentation, adaptive camouflages,
interactive education and entertainment.
Here we work out the mathematical details and assorted applications for coordinated motion using rigid formations. This work appears in a number of different publications:
Operations on Rigid Formations of Autonomous Agents +
Rigidity, Computation, and Randomization
in Network Localization +
Information Structures to Secure Control of Globally Rigid Formations +
Information Structures to Control Formation Splitting and Merging +
Merging Globally Rigid Formations.
2003
Natural materials often exhibit complex reflectance and
intricate geometry posing a real challenge in surface modeling.
We investigate this problem in our volumetric surface
reconstruction and modeling project. In the process, we have
compiled a database of several complex volumetric surface
textures. We have decided to make this valuable resource
available to other researchers interested in the topic.
A major limitation of existing projection display systems is
that they rely on a high quality screen for projecting images.
We believe that relaxing this restriction will make projectors
more useful and widely applicable. The fundamental problem with
using an arbitrary surface for a screen is that the surface is
bound to have its own colors and textures (bricks of a wall,
painting on a wall, tiles of a ceiling, grain of a wooden door,
etc.) or surface markings (paint imperfections, scratches,
nails, etc.). As a result, when an image is projected onto the
surface, the appearance of the image is modulated by the
spatially varying reflectance properties of the surface. Humans
are very sensitive to such modulations. In this paper, we
present a method that enables a projector to display images
onto an arbitrary surface such that the quality of the images
is preserved and the effects of the surface imperfections are
minimized. Our method is based on an efficient off-line
radiometric calibration that uses a camera to obtain
measurements from the surface corresponding to a set of
projected images. The calibration results are then used on-line
to compensate each display image prior to projection. Several
experimental results are shown that demonstrate the advantages
of using our compensation method.
Helmholtz stereopsis has been introduced recently as a surface
reconstruction technique that does not assume a model of
surface reflectance. In the reported formulation,
correspondence was established using a rank constraint,
necessitating at least three viewpoints and three pairs of
images. Here, it is revealed that the fundamental Helmholtz
stereopsis constraint defines a nonlinear partial differential
equation, which can be solved using only two images. It is
shown that, unlike conventional stereo, binocular Helmholtz
stereopsis is able to establish correspondence (and thereby
recover surface depth) for objects having an arbitrary and
unknown BRDF and in textureless regions (i.e., regions of
constant or slowly varying BRDF). An implementation and
experimental results validate the method for specular surfaces
with and without texture.
2001
Surface properties of many real-life objects often cannot be
effectively captured by any existing lighting models (such as
Phong). Not only can the reflectance properties be arbitrary,
but they can also vary over the entire surface. This project
deals with this problem, specifically, how to reconstruct the
surface of an object with arbitrary and spatially varying BRDF,
and how to render synthetic images of that object under novel
illumination.
When an unknown object with Lambertian ressectance is viewed orthographically, there is an implicit
ambiguity in determining its 3-d structure: we show that the visible surface of an object is indistinguishable
from a three parameter family of Generalized Bas-Relief transformations on the shape of the object.
For each image of the object illuminated by an arbitrary
number of distant light sources, there exists an identical image of the transformed object illuminated by similarly
transformed light sources. This result holds both for the illuminated regions of the object as well as those in cast and
attached shadows. Furthermore, neither small motion of the object, nor of the viewer will resolve the ambiguity in
determining the ssattening (or scaling) of the objectOs surface. Implications of this ambiguity on structure recovery
and shape representation are discussed.
2000
We consider the problem of determining functions of an image of
an object that are insensitive to illumination changes. We
first show that for an object with Lambertian reflectance there
are no discriminative functions that are invariant to
illumination. This result leads us to adopt a probabilistic
approach in which we analytically determine a probability
distribution for the image gradient as a function of the
surface’s geometry and reflectance. Our distribution
reveals that the direction of the image gradient is insensitive
to changes in illumination direction. We verify this
empirically by constructing a distribution for the image
gradient from more than 20 million samples of gradients in a
database of 1,280 images of 20 inanimate objects taken under
varying lighting condition. Using this distribution, we develop
an illumination insensitive measure of image comparison and
test it on the problem of face recognition.
1998
The appearance of an object depends on both the viewpoint from
which it is observed and the light sources by which it is
illuminated. If the appearance of two objects is never
identical for any pose or lighting conditions, then – in
theory – the objects can always be distinguished or
recognized. The question arises: What is the set of images of
an object under all lighting conditions and pose? In this
paper, we consider only the set of images of an object under
variable illumination, including multiple, extended light
sources and shadows. We prove that the set of n-pixel images of
a convex object with a Lambertian reflectance function,
illuminated by an arbitrary number of point light sources at
infinity, forms a convex polyhedral cone in IRn and that the
dimension of this illumination cone equals the number of
distinct surface normals. Furthermore, the illumination cone
can be constructed from as few as three images. In addition,
the set of n-pixel images of an object of any shape and with a
more general reflectance function, seen under all possible
illumination conditions, still forms a convex cone in IRn.
Extensions of these results to color images are presented.
These results immediately suggest certain approaches to object
recognition. Throughout, we present results demonstrating the
illumination cone representation.
1997
We develop a face recognition algorithm which is insensitive to
large variation in lighting direction and facial expression.
Taking a pattern classification approach, we consider each
pixel in an image as a coordinate in a high-dimensional space.
We take advantage of the observation that the images of a
particular face, under varying illumination but fixed pose, lie
in a 3D linear subspace of the high dimensional image space-if
the face is a Lambertian surface without shadowing. However,
since faces are not truly Lambertian surfaces and do indeed
produce self-shadowing, images will deviate from this linear
subspace. Rather than explicitly modeling this deviation, we
linearly project the image into a subspace in a manner which
discounts those regions of the face with large deviation. Our
projection method is based on Fisher’s Linear
Discriminant and produces well separated classes in a
low-dimensional subspace, even under severe variation in
lighting and facial expressions. The Eigenface technique,
another method based on linearly projecting the image space to
a low dimensional subspace, has similar computational
requirements. Yet, extensive experimental results demonstrate
that the proposed ’’Fisherface’’ method
has error rates that are lower than those of the Eigenface
technique for tests on the Harvard and Yale Face Databases.
Publications
2009
- “Attribute and Simile Classifiesrs for Face Verification,”
International Conference on Computer Vision, 2009. (N. Kumar,
A. Berg, P. Belhumeur, S. K. Nayar)
[PDF] - “Moving Gradients: A Path-Based Method for Plausible Image Interpolation,”
ACM Trans. on
Graphics (SIGGRAPH), August 2009. (D. Mahajan, F.C. Huang, W. Matusik, R. Ramamoorthi, P. Belhumeur)
[PDF] - “Removing Image Arifacts Due to Dirty Camera Lenses and Thin Occluders,”
ACM Trans. on
Graphics (SIGGRAPH), 2009. (J. Gu, R. Ramamoorthi, P. Belhumeur, S. K. Nayar)
[PDF]
2008
- “Face Swapping: Automatically Replacing
Faces in Photographs,” ACM Trans. on
Graphics (SIGGRAPH), August 2008. (D. Bitouk, N. Kumar, P. N.
Belhumeur, S. K. Nayar)
[PDF] - “Rigid Formations with Leader-Follower
Architecture,” IEEE Trans. on
Robotics, 2008. (T. Eren, W. Whiteley, P. N.
Belhumeur)
[PDF] - “Color Subspaces as Photometric
Invariants,” International Journal of Computer
Vision, 2008. (T. Zickler, S. Mallick, P. N. Belhumeur, D.
Kriegman)
[PDF] - “Face Tracer: A Search Engine for Large
Collections of Images with Faces,”
European Conference on Computer Vision, 2008. (N. Kumar,
P. Belhumeur, S. K. Nayar)
[PDF] - “Compressive Structured Light for
Recovering Inhomogeneous Participating Media,”
European Conference on Computer Vision, 2008. (J. Gu,
S. K. Nayar, E. Grinspun, P. Belhumeur, R. Ramamoorthi)
[PDF] - “Searching the World’s Herbaria: A System for the
Visual Identification of Plant Species,”
European Conference on Computer Vision, 2008. (S.
Shirdhonkar, S. White, S. Feiner, D. Jacobs, J. Kress, P. N.
Belhumeur)
[PDF]
2007
- “Active Refocusing of Images and
Video,” ACM Trans. on Graphics (SIGGRAPH), August
2007 (F. Moreno-Noguer, S. K. Nayar, and P. N. Belhumeur)
[PDF] - “A Theory of Locally Low Dimensional Light
Transport,” ACM Trans. on Graphics (SIGGRAPH),
August 2007. (D. K. Mahajan, R. Ramamoorthi, I. Kemelmacher, P. N.
Belhumeur)
[PDF] - “Photometric Depth Ranging of Non-Lambertian Surfaces,” submitted to International Journal of Computer Vision, 2007. (S. Magda, D. Kriegman, P. N. Belhumeur)
- “Graphical Properties of Easily Localizable Sensor
Networks,” Wireless Networks, 2007. (B. Anderson, R.
Yang, D. Goldberg, A. S. Morse, W. Whiteley, T. Eren, P.
Belhumeur)
[PDF] - “Time Varying BRDFs,” IEEE Trans.
on Visualization and Computer Graphics, pp. 595-609, May/June
2007. (B. Sun, K. Sunkavalli, R. Ramamoorthi, P. N. Belhumeur, S.
Nayar.)
[PDF] - “A First Order Analysis of Lighting,
Shading, and Shadows,” to appear in ACM Trans. of
Graphics, 2007. (R. Ramamoorthi, D. K. Mahajan, and P. N.
Belhumeur)
[PDF] - “Dirty Glass: Modeling and Rendering Contamination on
Transparent Surfaces,” in the Proc. EuroGraphics
Symposium on Rendering, 2007. (J. Gu, P. N. Belhumeur, R.
Ramamoorthi, and Shree Nayar)
[PDF]
2006
- “Time-Varying Surface Appearance:
Acquisition, Modeling and Rendering,” ACM Trans. on
Graphics (SIGGRAPH), August 2006. (J. Gu, R. Ramamoorthi, P.
N. Belhumeur, and S. Nayar)
[PDF] - “Ongoing Challenges in Face
Recognition,” Frontiers of Engineering: Reports on
Leading-Edge Engineering, National Academy of Engineering,
National Academy Press, pp. 5-14, 2006.
[PDF] - “First Steps Toward an Electronic Field
Guide for Plants,” Taxon, 2006. G. Agarwal, H. Ling,
D. Jacobs, S. Shirdhonkar, W. Kress, R. Russell, P. Belhumeur, N.
Dixit, S. Feiner, D. Mahajan, K. Sunkavalli, and S. White)
[PDF] - “Multiplexing for Optimal Lighting,”
IEEE Trans. on Pattern Analysis and Machine Intelligence,
2006. (Y. Schechner, S. Nayar, and P. N. Belhumeur)
[PDF] - “Reflectance Sharing: Predicting Appearance
from a Sparse Set of Images of a Known Shape,” IEEE
Trans. on Pattern Analysis and Machine Intelligence, 2006. (T.
Zickler, R. Ramamoorthi, S. Enrique and P. N. Belhumeur)
[PDF] - “Rigid Formations with Leader Follower Architecture,” submitted to IEEE Transactions on Robotics, January 2006. (T. Eren, W. Whiteley, and P. N. Belhumeur)
- “A Theory of Network Localization, to
appear in IEEE Transactions on Mobile Computing, 2006. (J.
Aspnes, T. Eren, D. K. Goldenberg, A. S. Morse, W. Whiteley, Y. R.
Yang, B. D. O. Anderson, and P. N. Belhumeur)
"[PDF]”:journal/network-localization-TMC06.pdf - “Directed Rigid Formations of Autonomous Agents,” the 45th IEEE Conference on Decision and Control, San Diego, California, 2006. (T. Eren, W. Whiteley, P. N. Belhumeur)
- “Specularity Removal in Images and Videos:
A PDE Approach,” Proc. European Conference on Computer
Vision, 2006. (S. P. Mallick, T. E. Zickler, P. N. Belhumeur,
and D. J. Kriegman)
[PDF] - “Color Spaces as Photometric Invariants,” Proc.
IEEE Conf. Computer Vision and Pattern Recognition, 2006. (T.
Zickler, S. P. Mallick, D. J. Kriegman, and P. N. Belhumeur)
[PDF]
2005
- “A Fourier Theory for Cast Shadows”,
IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol.27, no.2, 2005. (R. Ramamoorthi, M.
Koudelka, and P. Belhumeur). .
[PDF] - “Graphical Properties of Easily Localizable Sensor Networks,” submitted to Wireless Networks, Journal of Mobile Communication, Computation and Information, Springer, December 2005. (B. D. O. Anderson, P. N. Belhumeur, T. Eren, D. K. Goldenberg, A. S. Morse, W. Whiteley, and Y. R. Yang)
- “Optimal Illumination for Image and Video
Relighting”, ACM SIGGRAPH 2005 Technical Sketches.
(F. Moreno-Noguer, S. K. Nayar, P. N. Belhumeur).
[PDF] - “Optimal Illumination for Image and Video
Relighting”, IEE European Conference on Visual Media
Production (CVMP), 2005. (F. Moreno-Noguer, S. K. Nayar, P. N.
Belhumeur)
[PDF] - “Beyond Lambert: Reconstructing Specular
Surfaces Using Color,” Proc. IEEE Conf. Computer Vision
and Pattern Recognition, 2005. (S. P. Mallick, T. E. Zickler,
D. J. Kriegman, P. N. Belhumeur)
[PDF] - “Reflectance Sharing: Image-based Rendering
from a Sparse Set of Images,” Proc. Eurographics
Symposium on Rendering, pp. 253-265, 2005. (T. E. Zickler, S.
Enrique, R. Ramamoorthi, P. N. Belhumeur)[
[PDF] - “Reflectance Sharing: Image-based Rendering
from a Sparse Set of Images,” ACM SIGGRAPH Technical
Sketch, 2005. (T. E. Zickler, S. Enrique, R. Ramamoorthi, P.
N. Belhumeur)
[PDF] - “Using Eye Reflections for Face Recognition
Under Varying Illumination,” IEEE Int’l Conf. on
Computer Vision ICCV, 2005. (K. Nishino, P. N. Belhumeur, and
S. K. Nayar)
[PDF] - “Further Results on Sensor Network Localization Using
Rigidity,” Proceedings of the Second European Workshop on
Sensor Networks (EWSN), January 2005, pp. 405-409. (T. Eren,
W. Whiteley, and P. N. Belhumeur)
[PDF]
2004
- “Lighting Sensitive Displays,”
ACM Transactions on Graphics 23, 4 (2004), pp. 963-979.
(S. Nayar, P. Belhumeur, and T. Boult)
[PDF] - “Operations on Rigid Formations of
Autonomous Agents,” Communications in Information and
Systems, September 2004, pp. 223-258. 2004. (T. Eren, W.
Whiteley, A. S. Morse, B. D. O. Anderson, and P. N.
Belhumeur)
[PDF] - “A Fourier Theory for Cast Shadows,”
European Conference on Computer Vision, May 2004. (R.
Ramamoorthi, M. Koudelka, P. Belhumeur)
[PDF] - “Making One Object Look Like Another:
Controlling Appearance Using a Projector-Camera System,”
Proc. IEEE Computer Vision and Pattern Recognition (CVPR),
Vol. 1, p. 452-459, 2004. (M. D. Grossberg, H. P., S. K. Nayar, and
P. N. Belhumeur)
[PDF] - “Rigidity, Computation, and Randomization
in Network Localization,” Proceedings of the
International Annual Joint Conference of the IEEE Computer and
Communications Societies (INFOCOM), Hong Kong, March 2004, pp.
2673-2684. T. Eren, D. Goldenberg, W. Whiteley, Y. R. Yang, A. S.
Morse, B. D. O. Anderson, and P. N. Belhumeur)
[PDF] - “Information Structures to Secure Control
of Globally Rigid Formations,” Proceedings of the
American Control Conference, Boston, July 2004, pp. 4945-4950.
(T. Eren, W. Whiteley, A. S. Morse, P. N. Belhumeur, and B. D. O.
Anderson)
[PDF] - “Information Structures to Control
Formation Splitting and Merging,” Proceedings of the
American Control Conference, Boston, July 2004, pp. 4951-4956.
(T. Eren, B. D. O. Anderson, W. Whiteley, A. S. Morse, and P. N.
Belhumeur)
[PDF] - “Merging Globally Rigid Formations,”
Proceedings of AAMAS (the Third International Joint Conference
on Autonomous Agents & Multi Agent Systems), New York,
July 2004, pp. 1258-1259. (T. Eren, W. Whiteley, A. S. Morse, P. N.
Belhumeur, and B. D. O. Anderson)
[PDF]
2003
- “Binocular Helmholtz Stereopsis,”
Proc. IEEE International Conference on Computer Vision,
October 2003. pp. 1411-1417. (T. Zickler, J. Ho, D. Kriegman, J.
Ponce, and P. Belhumeur)
[PDF] - “Toward a Stratification of Helmholtz
Stereopsis,” Proc. IEEE Conference on Computer Vision and
Pattern Recognition, June 2003. Vol. I, pp. 548-555. (T.
Zickler, P. Belhumeur, and D. Kriegman)
[PDF] - “A Projection System with Radiometric
Compensation for Screen Imperfections,” Proc. of the IEEE
Inter. Workshop on Projector Camera Systems, Nice, October
2003. (S. Nayar, H. Peri, M. Grossberg, and P. Belhumeur )
[PDF] - “Acquiring, Compressing, and Synthesizing
Bidirectional Texture Functions,” Texture 2003: Third
International Workshop on Texture Analysis and Synthesis,
Nice, France, October 2003. (M. Koudelka, S. Magda, P. Belhumeur,
D. Kriegman)
[PDF] - “Sensor and Network Topologies of
Formations with Distance-Direction-Angle Constraints,”
IEEE Conference on Decision and Control, 2003, submitted.
(T. Eren, W. Whiteley, A. S. Morse, and P. Belhumeur)
[PDF] - “Helmholtz Stereopsis: Exploiting Reciprocity for Surface
Reconstruction,” Proc. 7th European Conference on
Computer Vision, May 2002. Vol. III, pp. 869-884. (T. Zickler,
P. Belhumeur, and D. Kriegman)
[PDF]
2002
- “Helmholtz Stereopsis: Exploiting
Reciprocity for Surface Reconstruction,” Int. Journal of
Computer Vision, Vol. 49 No. 2/3, pp. 215-227.
September/October, 2002. (T. Zickler, P. Belhumeur and D.
Kriegman)
[PDF] - “A Framework for Maintaining Formations
Based on Rigidity,” Proceedings of the 2002 IFAC World
Congress, July, 2002, Barcelona, Spain. (T. Eren, P.
Belhumeur, B. D. O. Anderson, and A. S. Morse)
[PDF] - “Closing Ranks in Vehicle Formations Based on
Rigidity,” Proceedings of the 2002 IEEE Conference on
Decision and Control, December 2002, Las Vegas, NV, USA. (T.
Eren, P. Belhumeur, and A. S. Morse)
[PDF]
2001
- “From Few to Many: Illumination Cone Models
for Face Recognition Under Variable Lighting and Pose,”
IEEE Trans. PAMI, 23(6), pp. 643-60, 2001. (A.
Georghiades, P. Belhumeur and D. Kriegman)
[PDF] - “What Shadows Reveal About Object
Structure,” Journal of the Optical Society of America
– A, pp. 1804-1813, August, 2001, (D. Kriegman and P.
Belhumeur)
[PDF] - “Image-based Modeling and Rendering of
Surfaces with Arbitrary BRDFs,” Proc. IEEE Conf.
CVPR, submitted, 2001. (M. Koudelka, P. Belhumeur, S. Magda
and D. Kriegman)
[PDF] - “Finding Folds: On the Appearance and
Identification of Occlusion,” Proc. IEEE Conf. CVPR,
submitted, 2001. (P. Huggins, H. Chen, P. Belhumeur, and S.
Zucker)
[PDF] - “Beyond Lambert: Reconstructing Surfaces
with Arbitrary BRDFs,” Proc. Int. Conf. of Computer
Vision, to appear, 2001. (S. Magda, T. Zickler, D. Kriegman
and P. Belhumeur)
[PDF] - “Lighting-Sensitive Displays,”
SIGGRAPH Technical Sketch, p. 218, 2001. (S. Nayar, P.
Belhumeur, and T. Boult)
[PDF] - “Judging Whether Multiple Silhouettes Can Come From the
Same Object,” Proc. Fourth Int. Workshop on Visual
Form, pp. 533-41, 2001. (D. Jacobs, P. Belhumeur, and I.
Jermyn)
[PDF]
2000
- “Shedding Light on Image-Based Rendering,” SIGGRAPH Technical Sketch, p. 255, 2000. (S. Magda, J. Lu, D. Kriegman and P. Belhumeur)
- “In Search of Illumination
Invariants,” Proc. IEEE Conf. CVPR, vol. 2, pp.
254-61, 2000. (H. Chen, P. Belhumeur and D. Jacobs)
[PDF] - “From Few to Many: Generative Models of Recognizing Faces
under Variable Pose and Illumination,” Proc. Fourth IEEE
Int. Conf. on Automatic Face and Gesture Recognition, pp.
277-84, 2000. (A. Georghiades and P. Belhumeur)
[PDF]
1999
- “Determining Generative Models of Objects
Under Varying Illumination: Shape and Albedo from Multiple Images
Using SVD and Integrability,” Int. Journal of Computer
Vision, 35(3), pp. 203-22, 1999. (A. Yuille, D. Snow, R.
Epstein and P. Belhumeur)
[PDF] - “The Bas-Relief Ambiguity,” Int.
Journal of Computer Vision, 35(1), pp. 33-44, 1999. (P.
Belhumeur, D. Kriegman and A. Yuille)
[PDF] - “Computational Vision at Yale,”
Int. Journal of Computer Vision, 35(1), pp. 5-12, 1999.
(P. Belhumeur, J. Duncan, G. Hager, D. McDermott, A. S. Morse and
S. Zucker)
[PDF] - “Tracking in 3D: Image Variability
Decomposition for Recovering Object Pose and Illumination,”
Pattern Analysis and Applications, 2(1), pp. 82-91, 1999.
(P. Belhumeur and G. Hager)
[PDF] - “Shadows, Shading, and Projective Ambiguity,” Shape, Contour, and Grouping in Computer Vision, D. Forsyth, J. Mundy, V. Gesu, R. Cipolla, (Eds.), Springer-Verlag, pp. 132-51, 1999. (P. Belhumeur, D. Kriegman and A. Yuille)
- “Representations for Recognition Under Variable Illumination,” Shape, Contour, and Grouping in Computer Vision, D. Forsyth, J. Mundy, V. Gesu, R. Cipolla, (Eds.), Springer-Verlag, pp. 95-131, 1999. (D. Kriegman, P. Belhumeur and A. Georghiades)
- “Shape and Enlightenment: Reconstruction and Recognition under Variable Illumination,” Int. Symposium on Robotics Research, pp. 79-88, October 1999. (D. Kriegman, P. Belhumeur and A. Georghiades)
- “Illumination-Based Image Synthesis: Creating Novel
Images of Human Faces Under Differing Pose and Lighting,”
Proc. IEEE Workshop on Multi-View Modeling and Analysis of
Visual Scenes, pp. 47-54, 1999. (A. Georghiades and P.
Belhumeur)
[PDF]
1998
- “What Is the Set of Images of an Object
Under All Possible Illumination Conditions?” Int. Journal
of Computer Vision, 28(3), pp. 245-60, 1998. (P. Belhumeur and
D. Kriegman)
[PDF] - “Efficient Region Tracking with Parametric
Models of Geometry and Illumination,” IEEE Trans.
PAMI, 20(10), pp. 1025-39, October 1998. (G. Hager and P.
Belhumeur)
[PDF] - “Shadows, Shading, and Projective Ambiguity,” Int. Joint Workshop on Shape, Contour, and Grouping, Palermo, Italy, May 1998. Paper also later appeared in Shape, Contour, and Grouping in Computer Vision, D. Forsyth, J. Mundy, V. Gesu, R. Cipolla, (Eds.), Springer-Verlag, pp. 132-51, 1999. (P. Belhumeur, D. Kriegman and A. Yuille)
- “Representations for Recognition Under Variable Illumination,” Int. Joint Workshop on Shape, Contour, and Grouping, Palermo, Italy, May 1998. Paper also later appeared in Shape, Contour, and Grouping in Computer Vision, D. Forsyth, J. Mundy, V. Gesu, R. Cipolla, (Eds.), Springer-Verlag, pp. 95-131, 1999. (D. Kriegman, P. Belhumeur and A. Georghiades)
- “Tracking in 3D: Image Variability
Decomposition for Recovering Object Pose and Illumination,”
Proc. Int. Conf. on Pattern Analysis and Applications,
1998. (P. Belhumeur and G. Hager)
[PDF] - “Illumination Cones for Recognition Under
Variable Lighting: Faces,” Proc. IEEE Conf. CVPR,
pp. 52-58, 1998. (A. Georghiades, D. Kriegman and P.
Belhumeur)
[PDF] - “Comparing Images Under Variable
Illumination,” Proc. IEEE Conf. CVPR, pp. 610-17,
1998. (D. Jacobs, P. Belhumeur and R. Basri)
[PDF] - “What Do Shadows Reveal About Object Structure?”
Proc. Fifth European Conf. on Computer Vision, vol. 2, pp.
399-414, 1998. (D. Kriegman and P. Belhumeur)
[PS]
1997
- “Eigenfaces vs. Fisherfaces: Recognition
Using Class Specific Linear Projection,” IEEE Trans.
PAMI, Special Issue on Face Recognition, 19(7), pp. 711-20,
July 1997. (P. Belhumeur, J. Hespanha and D. Kriegman)
[PDF] - “The Bas-Relief Ambiguity,” Proc. IEEE Conf.
CVPR, pp. 1060-66, 1997. (P. Belhumeur, D. Kriegman and A.
Yuille)
[PDF]
1996
- “A Bayesian Approach to Binocular
Stereopsis,” Int. Journal of Computer Vision, 19(3),
pp. 237-60, 1996. (P. Belhumeur) [PS]
[PS] - “A Computational Theory for Binocular Stereopsis,” D. Knill and W. Richards (Eds.), Perception as Bayesian Inference, Cambridge University Press, pp. 323-64, 1996. (P. Belhumeur)
- “What Is the Set of Images of an Object
Under All Possible Illumination Conditions?” Proc. IEEE Conf.
CVPR, pp. 270-277, 1996. (P. Belhumeur and D. Kriegman)
[PDF] - “Real-Time Tracking of Image Regions with
Changes in Geometry and Illumination,” Proc. IEEE Conf.
CVPR, pp. 403-10, 1996. (G. Hager and P. Belhumeur)
[PDF] - “Eigenfaces vs. Fisherfaces: Recognition
Using Class Specific Linear Projection,” Proc. Fourth
European Conf. on Computer Vision, vol. 1, pp. 45-58, 1996.
(P. Belhumeur., J. Hespanha and D. Kriegman)
[PDF] - “Learning Object Representations from
Lighting Variations,” Proc. Int. Workshop on Object
Representation in Computer Vision II, pp. 179-99, 1996. (R.
Epstein, A. Yuille and P. Belhumeur)
[PDF] - “Estimation of Motion Boundary Location and Optical Flow
Using Dynamic Programming,” Proc. IEEE Int. Conf. on
Image Processing, vol. 3, pp. 509-12, 1996. (X. Papademetris
and P. Belhumeur)
[PDF]
1995
- “Recovering Object Surfaces from Viewed Changes in
Surface Texture Patterns,” Proc. IEEE Fifth Int. Conf. on
Computer Vision, pp. 876-81, 1995. (P. Belhumeur and A.
Yuille)
[PDF]
1994
- “Global Priors for Binocular Stereopsis,” Proc.
IEEE Int. Conf. on Image Processing, vol. 2, pp. 730-4, 1994.
(P. Belhumeur)
[PDF]
1993
- “Bayesian Models for Reconstructing the
Scene Geometry in a Pair of Stereo Images,” Proc. IEEE
Conf. Info. Sciences and Systems, Johns Hopkins University,
Baltimore, 1993. (P. Belhumeur)
[PS] - “A Binocular Stereo Algorithm for Reconstructing Sloping,
Creased, and Broken Surfaces in the Presence of
Half-occlusion,” Proc. IEEE Fourth Int. Conf. on Computer
Vision, pp. 431-8, 1993. (P. Belhumeur)
[PDF]
1992
- “A Bayesian Treatment of the Stereo Correspondence
Problem Using Half-occluded Regions,” Proc. IEEE Conf.
CVPR, pp. 506-12, 1992. (P. Belhumeur and D. Mumford)
[PDF]
1989
- “Toward a Model-based Bayesian Theory for Estimating and
Recognizing Parameterized 3-D Objects Using Two or More Images
Taken from Different Positions,” IEEE Trans. PAMI,
11(10), pp. 1028-52, October 1989. (B. Cernuschi-Frias, D. Cooper,
Y. Hung and P. Belhumeur)
[PDF]
1986
- “3-D Object Position Estimation and Recognition Based on
Parameterized Surfaces and Multiple Views,” Proc. IEEE
Conf. Robotics and Automation, vol. 1, pp. 639-44, 1986. (B.
Cernuschi-Frias, D. B. Cooper and P. Belhumeur)
[PDF]
1985
- “Estimating and Recognizing Parameterized 3-D Objects Using a Moving Camera,” Proc. IEEE Conf. CVPR, pp. 167-71, 1985.
Pending
- “In Search of Illumination Invariants,” Int.
Journal of Computer Vision, under revision. (H. Chen, P.
Belhumeur and D. Jacobs)
[PDF]
Contact
623 Schapiro (CEPSR)
Dept. of Computer Science
Columbia University
New York, New York 10027
belhumeur@cs.columbia.edu
Phone: 212.939.7087
Fax: 212.939.7008