639 
It is well known that vanishing point based analysis requires images with strong perspective effects due to wide angle lenses, close 
objects, and often oblique viewing angles. Aerial imagery, however, normally present weak perspective effects because of long- 
range shootings. In this paper, we present a model-based method for reconstructing rectilinear buildings from single images. The 
recovery algorithm is formulated in terms of two objective functions which are based on the equivalence between the vector normal 
to the interpretation plane in the image space and the vector normal to the rotated interpretation plane in the object space. These 
objective functions are minimized with respect to the camera pose, the building dimensions, locations and orientations to obtain 
estimates for the structure of the scene. The comparison with the vanishing points based method indicates that our method is 
significantly superior over the vanishing points based method. The effectiveness of this approach is also demonstrated quantitatively 
through simulations and actual images. 
1. INTRODUCTION 
1.1 Background 
3D object reconstruction from images is a common problem in 
computer vision and photogrammetry. In the case of only one 
image available, 3D reconstruction from single images has to be 
performed. The existing single view reconstruction methods 
may be roughly divided into two broad categories: geometry 
constraints based and model based. Here we do not include non 
metric single view reconstruction methods (e.g. Hoiem et al., 
2005) in these two categories. The geometry constraint based 
methods (e.g. Leibowitz et al., 1999) use the geometry inherent 
in the images (i.e. vanishing points) to derive the camera 
calibration information and result in a 3D reconstruction; while 
most model-based methods (e.g. Debevec et al, 1996) recover 
model parameters through a model-to-image fitting algorithm 
which involves a minimization of total disparity errors between 
observed edges and projections of the reconstructed lines. 
Vanishing points are defined as points at which the extensions 
of parallel lines appear to converge in the perspective view of 
the image. The limitations of vanishing points based methods 
are obvious. As vanishing points are points in the image at 
infinity, slight inaccuracy in the measurements of lines will 
result in large errors in the positions of calculated vanishing 
points. Automatic methods improve the accuracy of vanishing 
detection but often require sufficient straight lines which are 
detectable in the images. In general, the images may, however, 
contain very few straight lines. Further, no existing edge 
detection algorithms can provide only useful edges reliably 
from images of a common scene; human intervention is always 
needed in those automatic methods. From a practical point of 
view, manual digitization of straight lines in the images is often 
involved. In this situation, the vanishing points based methods 
do not work well, in particular for those images with weak 
perspective effects (e.g. aerial images). 
In this paper, we present a model-based method to reconstruct 
rectilinear buildings up to a scale factor from single images. 
The difference from previous model-based methods is that our 
method does not require a model-to-image fitting algorithm, 
and therefore avoid a minimization procedure. The method is 
based on manual feature correspondence between pre-defined 
parameterized 3D model edges and corresponding image edges. 
The algorithm then automatically recovers camera pose and 
model dimensions. 
To our knowledge, the comparison between vanishing points 
based methods and model-based methods are rarely reported. In 
this paper, we also compared the performance of the vanishing 
points based method (Zhang et al. 2001) with our model-based 
method using identical synthetic and real data. The quantitative 
analysis results indicate that our method is significantly 
superior over the vanishing points based method in terms of 
feasibility for various images with strong or weak perspective 
effects. 
1.2 Related Work 
1.2.1 Vanishing Points Based Methods: The existing 
vanishing points detection methods may include manual 
detection, using Hough Transform (e.g. Tuytelaars 1997), 
searching over Gaussian sphere (e.g. van den Heuvel, 1998), 
and using projective geometry (e.g. Birchfield, 1998). Most of 
automatic vanishing points detection methods are not only 
computational intensive but also require human interaction, 
which are hard to reach operational level. Manual detection of 
vanishing points satisfies operational level but suffer problem 
that the determined vanishing points may not be accurate. There 
also exist a bunch of papers on vanishing points based 3D 
reconstruction (e.g. Guillou et al., 2000). Since our method is 
not in this route, we do not include a detailed review of the