Computer Vision Talks at Columbia University

Shadow removal for robust video surveillance systems

Yasuyuki Matsushita

Institute of Industrial Science, The University of Tokyo

Interschool Lab, 7th Floor CEPSR 

Host: Prof. Shree Nayar 

Abstract 

Cast shadows produce troublesome effects for video surveillance systems, typically for object tracking from a fixed viewpoint, since it yields appearance variations of objects depending on whether they are inside or outside the shadow. To robustly eliminate these shadows from image sequences as a preprocessing stage for robust video surveillance, we propose a framework based on the idea of intrinsic images. Unlike previous methods for deriving intrinsic images, we derive time-varying reflectance images and corresponding illumination images from a sequence of images. Using obtained illumination images, we normalize the input image sequence in terms of incident lighting distribution to eliminate shadow effects. We also propose a illumination normalization scheme which can potentially run in real time, utilizing the illumination eigenspace, which captures the illumination variation due to weather, time of day, etc. and a shadow interpolation method based on shadow hulls. In this talk, we'll present the theory of the framework with simulation results, and show its effectiveness with object tracking ran on real scene data sets for traffic monitoring.