Time Varying Surface Appearance |
 |
|
In this project, we are interested in acquiring, modeling and rendering
time-varying surface appearance. Traditional computer graphics and
computer vision algorithms generally assume that the appearances of
surfaces remain constant over time. However, there are a number of
natural processes that cause surface appearance to vary dramatically,
such as the burning of wood, the wetting and drying of rocks and
fabrics, the decay of fruit skins, and the corrosion and rusting of
steel and copper. We have captured the first time-varying surface
appearance database (with 26 samples), which includes a variety of
natural processes -- burning, drying, decay and corrosion. We have
developed a novel Space-Time Appearance Factorization (STAF) model,
which factors space- and time-varying appearance effects. The STAF model
includes an overall temporal appearance variation characteristic
function, two spatially varying textures corresponding to the initial
and final frames, and two spatially varying textures corresponding to
the rates and offsets at each point on the material that determine the
evolution of the appearance over time. Our experiments demonstrate that
the STAF model can accurately capture appearance variations due to a
wide variety of natural processes. Moreover, it enables a number of
novel rendering applications, such as the transfer of a time-varying
effect to a new material, the control of the evolution of a time-varying
process, and the extrapolation of a process beyond what is captured by
the acquired data. This project is done in collaboration with Ravi Ramamoorthi and Peter Belhumeur at Columbia University, and Chien-i Tu and Wojciech Matusik at MERL.
|
Publications
print_paperentry_byid: more than 2 or no entry matches.
|
Pictures
 |
|
Database of Time-Varying Surface Appearance and Factorization:
This image shows the 26 samples acquried in the database and the time-varying process that each sample is subjected to.
|
 |
|
Space-Time Appearance and Factorization (STAF):
This image shows the STAF representation of a drying wood sample. The top panel compares the reconstruction of a sample using
STAF to the original sample for a chosen lighting and viewing direction. STAF model can also be used for time normalization,
wherein the overall appearance changes maintained but the spatial drying patterns are removed. The panel below shows the estimated temporal characterisitic curves for both diffuse and specular parameters. We also show visualizations of the spatial diffuse textures A, D, R and O, along with the normalized initial frame A*f(0)+D and final frame A*f(1)+D. Please see the paper for more details.
|
 |
|
STAF Results for More Samples:
This image shows STAF estimation results for 5 samples in the database, each of which has 3 spatial locations marked. The middle row shows the time-varying curves p(x,y,t) (for the red diffuse component) for spatial locations A, B and C. The curves are quite different for the three
different points. In the bottom row, we align these time-varying curves using STAF model. The data closely matches the temporal characteristic
curve f(t') computed from all the points on the sample.
|
 |
|
Examples of Rendering:
This image shows some of our rendering results (please see videos as well). STAF model enables us to manipulate the acquired data.
We can create a time-varying texture with larger extent by applying standard 2D texture synthesis to the initial and final frames.
Within limits, we have extrapolate the time domain of the original acquired data.
We can control the rate and offset terms according to physical laws or user-derfined rules. We also can transfer a time-varying effect to
a new material.
|
|
Videos
 |
|
SIGGRAPH 2006 Video:
This video shows the complete pipeline we have used for our work on
time-varying surface appearance, including acqusition, modeling, and
rendering. (With narration)
.
|
 |
|
Some Samples in the Database:
This video shows the acquired data for 6 samples in our database for a
single lighting and view direction. Our captured data includes a large
number of lighting and viewing directions. (With narration)
.
|
 |
|
Rendering:
This video shows 5 rendering examples including texture synthesis,
time-extrapolation, control, and transfer. (With narration)
.
|
 |
|
More Synthesis Results:
This video shows texture syntheis examples for the drying rock and the
drying wood samples.
.
|
|
Slides
SIGGRAPH 2006 presentation With videos (zip file)
|
Databases
 |
|
STAF: Time Varying Appearance Database & Source Code
The database includes 26 samples, covering a range of phenomena
including burning, corrosion, drying on smooth and rough surfaces, and
decay. We also include the source code used to process the captured data
and estimate the STAF model. For each sample there are three parts of
data to download: 6 of the 1280 measurements of the raw data at each
time step, the fitted BRDF parameters, and the estimated STAF results.
The total size of the database is around 3 GB. Due to its enormous size
(about 1,000 Gb), we are not able to release all of the 1280
measurements of the raw data. If you really need it, please send an email
to staf@cs.columbia.edu |
|
Related Projects
Bidirectional Texture Function
Time-Varying BRDF
Oren-Nayar Reflectance Model |
|
