Time Varying Surface Appearance |
 | In this project, we are interested in acquiring,
modeling and rendering time-varying surface appearance. Traditional computer
graphics and computer vision algorithms generally assume that the appearances
of surfaces remain constant over time. However, there are a number of natural
processes that cause surface appearance to vary dramatically, such as the
burning of wood, the wetting and drying of rocks and fabrics, the decay of
fruit skins, and the corrosion and rusting of steel and copper. We have
captured the first time-varying surface appearance database (with 26 samples),
which includes a variety of natural processes -- burning, drying, decay and
corrosion. We have developed a novel Space-Time Appearance Factorization (STAF)
model, which factors space- and time-varying appearance effects. The STAF model
includes an overall temporal appearance variation characteristic function, two
spatially varying textures corresponding to the initial and final frames, and
two spatially varying textures corresponding to the rates and offsets at each
point on the material that determine the evolution of the appearance over time.
Our experiments demonstrate that the STAF model can accurately capture
appearance variations due to a wide variety of natural processes. Moreover, it
enables a number of novel rendering applications, such as the transfer of a
time-varying effect to a new material, the control of the evolution of a
time-varying process, and the extrapolation of a process beyond what is
captured by the acquired data. This project is done in collaboration with Ravi Ramamoorthi and Peter Belhumeur at Columbia
University, and Chien-i Tu and Wojciech Matusik at
MERL. |
Publications
"Time-varying Surface Appearance: Acquisition, Modeling, and Rendering,"
J. Gu, C. Tu, R. Ramamoorthi, P. Belhumeur, W. Matusik and S. K. Nayar,
ACM Trans. on Graphics (also Proc. of ACM SIGGRAPH),
Jul, 2006.
[PDF] [bib] [©]
|
Images
 |
|
Database of Time-Varying Surface Appearance and Factorization:
This image shows the 26 samples acquried in the database and the time-varying
process that each sample is subjected to.
|
| |
|
|
 |
|
Space-Time Appearance and Factorization (STAF):
This image shows the STAF representation of a drying wood sample. The top
panel compares the reconstruction of a sample using STAF to the original sample
for a chosen lighting and viewing direction. STAF model can also be used for
time normalization, wherein the overall appearance changes maintained but the
spatial drying patterns are removed. The panel below shows the estimated
temporal characterisitic curves for both diffuse and specular parameters. We
also show visualizations of the spatial diffuse textures A, D, R and O, along
with the normalized initial frame A*f(0)+D and final frame A*f(1)+D. Please see
the paper for more details.
|
| |
|
|
 |
|
STAF Results for More Samples:
This image shows STAF estimation results for 5 samples in the database, each
of which has 3 spatial locations marked. The middle row shows the time-varying
curves p(x,y,t) (for the red diffuse component) for spatial locations A, B and
C. The curves are quite different for the three different points. In the bottom
row, we align these time-varying curves using STAF model. The data closely
matches the temporal characteristic curve f(t') computed from all the points on
the sample.
|
| |
|
|
 |
|
Examples of Rendering:
This image shows some of our rendering results (please see videos as well).
STAF model enables us to manipulate the acquired data. We can create a
time-varying texture with larger extent by applying standard 2D texture
synthesis to the initial and final frames. Within limits, we have extrapolate
the time domain of the original acquired data. We can control the rate and
offset terms according to physical laws or user-derfined rules. We also can
transfer a time-varying effect to a new material.
|
| |
|
|
|
Videos
If you are having trouble viewing these .avi videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.
 |
|
SIGGRAPH 2006 Video:
This video shows the complete pipeline we have used for our work on
time-varying surface appearance, including acqusition, modeling, and rendering.
(With narration)
|
| |
|
|
 |
|
Some Samples in the Database:
This video shows the acquired data for 6 samples in our database for a single
lighting and view direction. Our captured data includes a large number of
lighting and viewing directions. (With narration)
|
| |
|
|
 |
|
Rendering:
This video shows 5 rendering examples including texture synthesis,
time-extrapolation, control, and transfer. (With narration)
|
| |
|
|
 |
|
More Synthesis Results:
This video shows texture syntheis examples for the drying rock and the drying
wood samples.
|
| |
|
|
|
Slides
SIGGRAPH 2006 presentation With videos (zip file)
|
Database
 |
|
STAF: Time Varying Appearance Database & Source Code::
The database includes 26 samples, covering a range of phenomena including burning, corrosion, drying on smooth and rough surfaces, and decay. We also include the source code used to process the captured data and estimate the STAF model. For each sample there are three parts of data to download: 6 of the 1280 measurements of the raw data at each time step, the fitted BRDF parameters, and the estimated STAF results. The total size of the database is around 3 GB. Due to its enormous size (about 1,000 Gb), we are not able to release all of the 1280 measurements of the raw data. If you really need it, please send an email to staf@cs.columbia.edu
|
| |
|
|
|
Bidirectional Texture Function
Time-Varying BRDFs
Oren-Nayar Reflectance Model
|
|
|
