Shadow detection based on colour segmentation and estimated illumination

In this paper we show how to improve the detection of shadows in natural scenes using a novel combination of colour and illumination features. Detecting shadows is useful because they provide information about both light sources and the shapes of objects thereby illuminated. Recent shadow detection methods use supervised machine learning techniques with input from colour and texture features extracted directly from the original images (e.g. Lalonde et al. ECCV 2010, Zhu et al. CVPR 2010). It seems sensible to augment these with estimates of scene illumination, as can be obtained with an intrinsic image extraction algorithm. Intrinsic image extraction separates the illumination and reflectance components in a scene, and the resulting illumination maps contain robust intensity change features at shadow boundaries. In this paper, we make two main contributions. First we improve upon existing methods for extracting illumination maps. Second we show how to use these illumination maps together with colour segmentation to extend the Lalonde's approach to shadow detection. Illumination maps are extracted using a steerable filter framework based on global and local correlations in low and high frequency bands respectively. The illumination and colour features so extracted are then input to a decision tree trained to detect shadow edges using AdaBoost. We tested variations of our proposed approach on two public databases of natural scenes. This study showed that our approach improves on that of Lalonde both in terms of sensitivity to shadow edges and rejection of false positives. Following Lalonde we show that our detection results are further improved by imposing an edge continuity constraint via a conditional random field (CRF) model.