Figure 9: Optimization of an extruded object, (a) Initial position, (b) The edges are assumed to 
form 90 degrees angles. After optimization using only the top view, the object’s projection 
matches the image features in the top view only, (c) After optimization using both views, 
the object’s projections match the image features in both views. 
3 3-D Surface Reconstruction 
Given the task of reconstructing a surface from multiple images whose vantage points may be very different, 
we need a surface representation that can be used to generate images of the surface from arbitrary view 
points, taking into account self-occlusion, self-shadowing, and other viewpoint-dependent effects. Clearly, 
a single image-centered representation is inadequate for this purpose. Instead, an object-centered surface 
representation is required. 
Many object-centered surface representations are possible. However, practical issues are important in 
choosing an appropriate one. First, the representation should be general-purpose in the sense that it should 
be possible to represent any continuous surface, closed or open, and of arbitrary genus. Second, it should be 
relatively straightforward to generate an instance of a surface from standard data sets such as depth maps or 
clouds of points Finally, there should be a computationally simple correspondence between the parameters 
specifying the surface and the actual 3-D shape of the surface, so that images of the surface can be easily 
generated, thereby allowing the integration of information from multiple images. 
A regular 3-D t.riangulation is an example of a surface representation that meets the criteria stated above, 
and is the one we have chosen for our previous work. In our implementation, all vertices except those on 
the edges have six neighbors and are initially regularly spaced. Such a mesh defines a surface composed of 
three-sided planar polygons that we call triangular facets, or simply facets. Triangular facets are particularly 
Fua -11