Detection and Removal of Rain |
 | The visual effects of rain are complex. Rain consists of spatially
distributed drops falling at high velocities. Each drop refracts and reflects
the environment, producing sharp intensity pattern in an image. A group of such
falling drops creates a complex time-varying signal in videos. In addition, due
to the finite exposure time of the camera, intensities due to rain are motion
blurred and hence depend on the background intensities. Thus, the visual
manifestations of rain are a combination of both the dynamics of rain and the
photometry of the environment. In this project, we have conducted a
comprehensive analysis of the visual effects of rain on an imaging system. We
have developed a correlation model that captures the dynamics of rain and a
physics-based motion blur model that captures the photometry of rain. Based on
these models, we have developed simple and efficient algorithms for detecting
and removing rain from videos. The effectiveness of our algorithms is
demonstrated via experiments on videos of complex scenes with moving objects
and time-varying textures. The techniques presented here can be used in a wide
range of applications including video surveillance, vision based navigation,
video/movie editing and video indexing/retrieval. |
Publications
"Detection and Removal of Rain from Videos,"
K. Garg and S.K. Nayar,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Vol. I, pp. 528-535, Jun. 2004.
[PDF] [bib] [©]
|
Image
 |
|
Overview of Rain Detection and Removal Algorithm:
The algorithm uses photometric and dynamic constraints to detect and remove
rain from pixels affected by rain.
|
| |
|
|
|
Videos
 |
|
Moving Objects and Rain:
This video shows results for a rain scene from the movie Magnolia
(courtesy of New Line Productions, Inc.). The detection and removal is
challenging here due to fast motion of textured objects (shirt creases and
folds on the moving arm). The algorithm only detects pixels with rain. The
detection result is shown in the form of a needle map, where the direction of
the needle represents the direction of rain and its length represents the
strength of the rain. There is slight time lag in detection since 30
consecutive frames were used for the detection in each frame.
|
| |
|
|
 |
|
Rain and Ripples:
The ripples of water in this video may be viewed as a temporal texture with
frequencies similar to those produced by rain. However, explicit modeling of
rain photometry allows us to distinguish other time-varying textures from rain.
In the removal results one sees some severely defocused streaks which produce
very small changes in pixel intensities and hence are difficult to remove in
presence of camera noise.
|
| |
|
|
 |
|
A Person in Rain:
In this video the direction of rain changes with time. The detection algorithm
finds the correct rain direction as indicated by the changing of the directions
of the needles with time.
|
| |
|
|
|
Slides
CVPR 2004 presentation With videos (zip file)
|
Selecting Camera Parameters for Rain Removal
Vision through Fog and Haze
|
|
|