IE RESTOS EEE 
i 
A A A 
EEA 
| 
| 
i 
[ 
| 
1 
| 
n 
| 
| 
i 
i 
ane ee ae 
SEE 
TT 
ER 
A 
ES 
A 
LEID END UM 
  
186 
MULTIPLE DEPTH AND NORMAL MAPS FOR SHAPE FROM MULTIPLE VIEWS AND VISUAL CUES 
Thomas Fromherz and Martin Bichsel 
University of Zurich 
Department of Computer Science 
MultiMedia Laboratory 
Winterthurerstr. 190 
8057 Zurich-Irchel 
Phone: +41 1 257 43 17 
Fax: +41 1 363 00 35 
Email: fromherz Q ifi.unizh.ch 
Email: bichsel Q ifi.unizh.ch 
KEY WORDS: shape reconstruction, shape representation, multiple visual cues, multiple views, sculptor's principle. 
ABSTRACT 
This paper introduces the sculptor's principle a new framework for shape representation and surface reconstruction from 
multiple shape cues. On one hand, a new shape from local brightness algorithm based on a standard voxel volume 
representation is presented. The sculptor's principle, on the other hand, provides the basis for a new and efficient 
representation of object surfaces by multiple depth maps. The sculptor's principle, in a natural way, combines shape 
information from complementary information sources such as multiple depth maps, multiple views, or multiple 
complementary shape cues. This leads to an elegant solution of the fusion problem, and to an efficient representation of 
object surfaces. : 
Surface reconstruction based on multiple depth maps is illustrated with depth from contours. The surface representation 
by multiple depth maps is reconstructed with O(N?n?) operations where N is the number of views and n? is the number of 
pixels in an image. For moderate N, this leads to very fast reconstruction algorithms. The additional storage requirement 
is O(Nn?) and thus only equals the amount of storage required to store the original images. 
1. INTRODUCTION 
This paper introduces the sculptor's principle as a new framework for shape representation and surface reconstruction 
from multiple shape cues. The sculptor's principle in a natural way combines shape information from complementary 
information sources such as multiple views, or multiple complementary shape cues. Fusion of different information. 
sources is a highly important problem in computer vision that attracts the attention of a large number of vision scientists. 
Related work in shape from multiple views either describes the object as a voxel cluster (Fromherz, 1994a) or uses an 
octree geometry as discussed by (Srivastava, et al, 1990), and (Szeliski, 1993). In the area of shape from multiple visual 
cues, the paper by (Zheng, 1994) presents shape from multiple visual cues using a triangulation of the recovered 3D 
points. 
Different volume representations serve different reconstruction needs. For instance, the voxel cluster which results from 
a standard shape from contours procedure from an image sequence, as described in (Fromherz, 1994a), represents all 
points on the convex hull of the object but none of the points of the object areas unexposed to contours. The resulting 
voxel representation in this case allows for a very elegant new algorithm to handle these unexposed areas by exploiting 
the local brightness information, the luminance, in the image sequence. 
However, when dealing with more sophisticated visual cues which involve an illumination model, we need a volume 
representation that enables an easy extraction of the surface normals of the object. The sculptor's principle provides a 
basis for such an efficient representation of object surfaces by multiple depth maps. 
In comparison to octree or voxel representations, the depth values of a depth map are floating points which allows to 
hold the entire reconstruction in floating point accuracy. Depth maps, where x and y coordinates need no additional 
storage, are a more compact data format than a surfaces triangulation. The fact that all data points are naturally ordered 
leads to highly efficient algorithms, especially when surface normals have to be calculated. Depth maps are easily 
converted into maps of surface normals, triangulations, or voxel based representations if needed and, therefore, are 
especially well suited for the fusion of different shape cues when each cue has a different preferred data representation. 
In the following section, we discuss the idea of the sculptor's principle together with the requirements it imposes on a 
reconstruction procedure. In section 3, the shape from local brightness algorithm based on a standard voxel 
representation is introduced. In section 4, surface reconstruction based on depth maps is illustrated with depth from 
contours from a sequence of video images. Section 5 gives a short summary of the object-camera set-up for the 
recording of the video sequence and an explanation of the segmentation algorithm employed. In section 6, we present 
our results for both, shape from luminance and depth from contours, which is followed by some conclusions in section 7. 
IAPRS, Vol. 30, Part 5W1, ISPRS Intercommission Workshop “From Pixels to Sequences”, Zurich, March 22-24 1995 
A fn OM OO OTTO b 
ny 
M -—— wm A Du us | 
e STRESSES CT RATS