[back]

Shadow Carving
Silvio Savarese, Holly Rushmeier, Fausto Bernardini, Pietro Perona

Abstract. The shape of an object may be estimated by observing the shadows on its surface. Assuming that a conservative estimate of the object shape is available, our method analyzes images of the object illuminated with known point light sources and taken from known camera locations. The surface estimate is adjusted using the shadow regions to produce a refinement that is still a conservative estimate. A proof of correctness is provided. The method has been tested and validated with experimental results.

Introduction and motivation. We introduce a method for using self-shadows to estimate shape of an object that can be implemented using inexpensive lighting and imaging equipment.We assume that we have as a starting point a conservative estimate of object shape – that is, the volume enclosed by the current surface estimate completely contains the physical object.We analyze images of the object illuminated with known point light sources taken from known camera locations. We adjust the current surface estimate using the shadow regions to produce improved shape estimates that remain conservative. A proof of correctness is provided. Shape from shadows has the advantage that it does not rely on surface texture for establishing correspondence, or on a model of the surface reflectance characteristics. However, past methods for shape from shadows can give either poor results or fail to converge when applied to physical data that contains error. Our method is robust with respect to a conservative classification of shadow regions – pixels in shadow may be mislabelled as being lit.No assumptions about the surface topology are made (multi-part objects and occlusions are allowed), although any non-smooth regions over the object’s surface are supposed to be detected.Our motivation for pursuing this work is the construction of an inexpensive and robust scanning system based on shape from silhouettes and shape from shadows.Furthermore, the method can be located within a similar volumetric framework as that employed by traditional shape-from-silhouette and recent multiple view algorithms.

Setup and Geometry. Consider an object in 3-D space and a point light source L illuminating it. A certain number of shadows are cast over the object by parts of the object itself. The scene is observed by a calibrated camera with center Oc. See figure below.

In order to simplify the discussion we consider a slice of the scene. A slice is defined by one of the (infinite possible) planes defined by the light source and the center of the camera. Thus, now we talk about image line, object's area and contour rather than image plane, object's surface and volume. The extension to the 3D case is immediate by observing that the slice may sweep the entire object’s volume.

The method. We start from a conservative estimate of the object countour (upper bound estimate). See figure below. The object is depicted in brown whereas its upper bound is in yellow. The light source L illuminates the object and cast a shadow S on its surface. Let us suppose that we are able to process the image in order to separate the shadows from the rest of the observed object. We call Si the observed shadow. What do the observed object self-shadows tell us about the real surface? The main information comes from the inconsistencies between the shadows which would be produced by the estimated surface and the observed shadows which are actually produced by the real surface. Thus, we could remove (carve out) regions from the current object estimate in order to reduce the inconsistencies, and therefore incrementally compute better estimates of the object’s shape, while making sure that at each step an upper bound estimate is maintained (conservative carving). We call this procedure shadow carving because we carve out areas from the object by using observed shadows. Here is an example of how to find a carvable area:

The observed shadow Si and the center of the camera define the green sector in figure.

Call Se the projection of the observed shadow Si into the current upper bound object estimate

The light source and Se defines the pink sector in figure.

The intersection between the green sector, the pink sector and the current upper bound object area (in yellow) give the carvable area (in red).

Thus, the carvable area can be removed from the current upper bound estimate of the object’s contour, generating an improved conservative approximation.

The process can be iterated by using multiple light sources. As the the light source changes, different shadows are cast over object contour. Thus, for each light source position, different carvable areas can be found and removed, producing incrementally better estimates of the object’s shape. See figure below.

We provide a proof of correctness. Namely, we show that the carvable area is well defined in the general case (i.e. complex surface topology, shadows occluded by other object parts, surfaces with low albedo or high specular regions) and always lying outside the actual object. See the technical paper for details.

Results. The method has been implemented and tested in the simple trial system shown below.

The results of the carving process are shown in the 2 lower panels. Consider for istance the panel on the left. The leftmost image in each row is a picture of the actual object. The second image in each row shows the initial upper bound estimate of the object (achieved by standard shape-from-silhouttes technique). The third and fourth images show the object after carving using shadows with 2 different light source locations. Altough the reconstruction is still coarse, the concavities around eyes and ears have become visible.

Publications
Shadow Carving. S. Savarese, Holly Rushmeier, Fausto Bernardini and P. Perona, in Proc. of the Int. Conf. on Computer Vision, Vancouver, Canada, June 2001.