Photorealistic Rendering of Rain Streaks

Photorealistic rendering of rain streaks with lighting and viewpoint effects is a challenging problem. Raindrops undergo rapid shape distortions as they fall, a phenomenon referred to as oscillations. Due to these oscillations, the reflection of light by, and the refraction of light through, a falling raindrop produce complex brightness patterns within a single motion-blurred rain streak captured by a camera or observed by a human. The brightness pattern of a rain streak typically includes speckles, multiple smeared highlights and curved brightness contours.

In this project, we develop a model for rain streak appearance that captures the complex interactions between the lighting direction, the viewing direction and the oscillating shape of the drop. Our model builds upon a raindrop oscillation model that has been developed in atmospheric sciences. We have measured rain streak appearances under a wide range of lighting and viewing conditions and empirically determined the oscillation parameters that are dominant in raindrops. The image shown above compares measured rain streaks with streaks rendered using our model, for different lighting and viewing directions. Using our model and estimated parameters, we have rendered thousands of rain streaks to create a database that captures the variations in streak appearance with respect to lighting and viewing directions. We have developed an efficient image-based rendering algorithm that uses our streak database to add rain to a single image or a captured video with moving objects and sources. Our rendering algorithm is very simple to use as it only requires a coarse depth map of the scene and the locations and properties of the light sources. Our rendering results show that the proposed physically-based rain streak model greatly enhances the visual realism of rendered rain.

Publications

"Photorealistic Rendering of Rain Streaks,"
K. Garg and S.K. Nayar,
ACM Trans. on Graphics (also Proc. of ACM SIGGRAPH),
Jul, 2006.
[PDF] [bib] [©]

Image

  Overview of Rain Rendering Pipeline:

In this illustration, we show the main steps involved in the image-based rendering algorithm used for inserting rain in images and videos.
     

Videos

If you are having trouble viewing these .mov videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.

  Complex Appearance of Rain Streaks:

This video shows the rapid shape distortions (i.e oscillations) that a raindrop undergoes as it falls. The reflection of light by, and the refraction of light through, these oscillating raindrops produce complex brightness patterns within a single motion-blurred rain streak. We have empirically determined the parameters of raindrop oscillations and use it to develop a rain streak appearance model that can faithfully produce these complex pattern. (With narration)
     
  Varying Illumination Direction:

This video shows rain rendered against a black background to highlight the variation in rain appearance with illumination direction. One can see the strong dependence of streak appearance on the illumination direction. We also compare our results with rain rendered using the commonly used constant-brightness streak model. When rain that is close to the camera is rendered using constant-brightness model it looks unrealistic as it fails to produce the complex streak patterns and the effects of changing illumination. (With narration)
     
  Night Scene with Isotropic Source:

This video shows rain added to a simple night scene with an isotropic light source that is immersed in rain. Illumination effects such as the halos around sources and camera effects such as defocus are captured using our algorithm. (With narration)
     
  Colored and Anisotropic Sources:

This video shows rain added to a scene with multiple light sources, including anisotropic traffic lamps. Note the shading of the rain due to the traffic lights and how the streak appearance varies with source direction. (With narration)
     
  Changing Sky Illumination:

Here we show rain added to a scene where the illumination changes from an overcast sky to bright sunlight. The streaks have smooth intensity pattern when illuminated by ambient lighting, such as the overcast sky. However, as the illumination changes to highly directional sunlight, we see high frequency intensity patterns within the streaks. (With narration)
     
  Rain with Moving Light Sources:

This video shows rain rendered for a scene with a moving car with headlights. The appearance and brightness of the rain streaks can be seen to vary with the positions and orientations of the headlights. (With narration)
     

Slides

SIGGRAPH 2006 presentation     With videos (zip file)

Database

  Rain Streak Database:

The database captures the effects of lighting direction, camera viewing direction and oscillations parameters on the appearance of rain streaks.
     

Detection and Removal of Rain

Selecting Camera Parameters for Rain Removal

Participating Media: Multiple Scattering Model

Participating Media: Single Scattering Model