In: Stilla U, Rottensteiner F, Paparoditls N (Eds) CMRT09. IAPRS, Vol. XXXVIII, Part 3/W4 — Paris, France, 3-4 September, 2009
217
REFINING BUILDING FACADE MODELS WITH IMAGES
Shi Pu and George Vosselman
International Institute for Geo-Information Science and Earth Observation (ITC)
Hengelosestraat 99, P.O. Box 6, 7500 AA Enschede, The Netherlands
spu@itc.nl, vosselman@itc.nl
Commission III/4
KEY WORDS: building reconstruction, data fusion, image interpretation
ABSTRACT:
Laser data and optical data have a complementary nature to 3D features’ extraction. Building reconstruction by fusion of the two data
sources can reduce the complexity of approaches from either side. In this paper we present a model refinement method, which uses
the strong lines extracted from close-range images to improve building models reconstructed from terrestrial laser point clouds. First,
model edges are projected from model space to image space. Then significant line features are extracted from an image with Canny
edge detector and Hough transformation. Each model edge is then compared with its neighboring image lines to determine the best
match. Finally the model edges are updated according to their corresponding image lines. The refinement process not only fixes certain
geometry errors of the original models, but also adapts the models to the image data, so that more accurate texturing is achieved.
1 INTRODUCTION
The technique of automated building facade reconstruction is use
ful to various applications. For urban planning, building facade
models provide important references to the city scenes from the
street level. For historical building documentation, a large num
ber of valuable structures are contained on the facades, which
should be recorded and reconstructed. For all virtual reality ap
plications with users’ view on the street, such as virtual tourism
and computer games, the accuracy and/or realistic level of the
building facade models are vital to successfully simulate an ur
ban environment.
A number of approaches (Dick et al., 2001, Schindler and Bauer,
2003, Frueh et al., 2005, Pollefeys et al., 2008) are available for
reconstructing building facades automatically or semi-automatically.
Close range image and terrestrial laser point cloud are the com
monly used input data. Image based building reconstruction has
been researched for years. From multiple 2D images captured
from different positions, 3D coordinates of the image features
(lines for example) can be calculated. Although acquisition of
images is cheap and easy, the difficulties of image understand
ing make it still difficult to automate the reconstruction using
only images. Laser altimetry has been used more and more in
recent years for automated building reconstruction. This can be
explained by the explicit and accurate 3D information provided
by laser point clouds. Researches (Vosselman. 2002, Frueh et al.,
2004, Brenner, 2005) suggest that the laser data and images are
complementary to each other, and efficient integration of the two
data types will lead to a more accurate and reliable extraction of
three dimensional features.
In the previous work we presented a knowledge based building
facade reconstruction approach, which extracts semantic facade
features from terrestrial laser point clouds and combines the fea
ture polygons to water-tight polyhedron models (Pu and Vossel
man, 2009). Some modeling errors still exist, and some of them
can hardly be corrected by further exploiting the laser data. In
this paper, we present a model refinement method which uses
strong line features extracted from images to improve the build
ing facade models generated from only terrestrial laser points.
The refinement not only fixes the models’ geometry errors, but
also solves inconsistencies between laser and image data, so that
a more accurate texturing can be achieved.
This paper is organized as follows. Section 2 gives an overview
of the presented method. Section 3 provides the context research
of building reconstruction from terrestrial laser scanning. Section
4 explains the preprocessing steps such as perspective conversion
and spatial resection, to make images usable for refining mod
els. Section 5 elaborates the image processing algorithms used
for significant line extraction and the matching and refinement
strategies. Experiments on three test cases are discussed in sec
tion 6. Some conclusions and outlooks are drawn in the final
section.
2 METHOD OVERVIEW
A building facade model may contain various errors. For a model
reconstructed from terrestrial laser points, the model edges may
have certain offset with their actual positions. These errors are
caused by gaps in laser points and the limitations of laser data
based reconstruction algorithms. Edges are delineated accurately
in images. After registering to the model space, image lines can
provide excellent reference from which the model edge errors can
be fixed. Another necessity of this refinement is to solve the in
consistencies between the laser space and the image space, so that
accurate texturing can be achieved.
Before starting the refinement, a 2D image needs to be referenced
to the 3D model space, a problem often referred as spatial resec
tion in photogrammetry. We use the standard resection solution
of collinearity equations, which requires minimum three image
points with their coordinates in model space. To find significant
line features from an image, we first detect edges using the Canny
algorithm (Canny, 1986), then apply the Hough transform to fur
ther extract strong line features from edges. Then model edges
are projected to the image space and matched with the image
lines. The best match is determined by the geometric properties
of candidates and the geometric relations between candidates and
the model edge. Finally each model edge with successful match
ing is projected to the matched image line accordingly, and model
edges without any matching are also adjusted to maintain a well
shape. Figure 1 gives a flowchart of the refinement process.
