International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B3, 2012 
XXII ISPRS Congress, 25 August — 01 September 2012, Melbourne, Australia 
LINE-BASED MULTI-IMAGE MATCHING FOR FACADE RECONSTRUCTION 
Tee-Ann Teo * *, Chung-Hsuan Kao? 
“ Dept. of Civil Engineering, National Chiao Tung University, Hsinchu, Taiwan 30010 - tateo@mail.nctu.edu.tw 
? Dept. of Civil Engineering, National Chiao Tung University, Hsinchu, Taiwan 30010 - rockkao2001 (hotmail.com 
Commission III, WGIII/1 
KEY WORDS: building, facade, linear feature, multiple images matching. 
ABSTRACT: 
This research integrates existing LOD 2 building models and multiple close-range images for facade structural lines extraction. The 
major works are orientation determination and multiple image matching. In the orientation determination, Speeded Up Robust 
Features (SURF) is applied to extract tie points automatically. Then, tie points and control points are combined for block adjustment. 
An object-based multi-images matching is proposed to extract the facade structural lines. The 2D lines in image space are extracted 
by Canny operator followed by Hough transform. The role of LOD 2 building models is to correct the tilt displacement of image 
from different views. The wall of LOD 2 model is also used to generate hypothesis planes for similarity measurement. Finally, 
average normalized cross correlation is calculated to obtain the best location in object space. The test images are acquired by a non- 
metric camera Nikon D2X. The total number of image is 33. The experimental results indicate that the accuracy of orientation 
determination is about 1 pixel from 2515 tie points and 4 control points. It also indicates that line-based matching is more flexible 
than point-based matching. 
1. INTRODUCTION 
Three-dimensional building model is an important geospatial 
data for a cyber city. A building model not only meets the need 
of a cyber city but also provides useful information in the 
domain of Location-Based Service (LBS). OGC (Open 
Geospatial Consortium) has established a standard format called 
CityGML for 3D building models (Grôger et al, 2008). The 
detail of building models in CityGML can be distinguished into 
LOD 1 (only block model), LOD 2 (with roof structure), LOD 3 
(with facade structure) and LOD 4 (with indoor structure). A 
detailed building model is not only similar to its true 
appearance, but also facilitates decision making procedures. 
As the LOD 1 and LOD 2 models focus on the shape of roof top, 
airborne sensors are usually selected to generate them. On the 
contrary, LOD 3 and LOD 4 model which are usually obtained 
by ground-based sensors, concentrate on the detail of facade 
and indoor facilities. Regardless of the types of LOD, the core 
process of model generation is feature extraction for different 
building structures. 
Image matching is a technique to relate the same location in 
different images. The correspondent features can be extended to 
three-dimensional features using space intersection technique. 
The matching algorithms can be classified into three categories, 
ie. area-based matching, feature-based matching and hybrid 
matching. Area-based matching calculates the similarity of gray 
value while feature-based matching compares the geometric 
similarity of extracted features. Hybrid matching utilizes the 
characteristics of both area-based and feature-based matching. 
From another point of view, matching strategy can be 
characterized by the data processing space, i.e., image-based 
matching (Habib et al, 2003) and object-based matching 
  
* Corresponding author. 
62 
(Zhang and Gruen, 2006). Both approaches consider the 
similarity and geometric constraints simultaneously. The 
image-based matching starts from an image point of the master 
image. Then, the corresponding conjugate points are obtained 
from the slave images. The image point of the master image is 
fixed in the image-based matching. On the contrary, the object- 
based matching starts from an object point in the object space. 
Then, the object point is back-projected to the image spaces and 
the similarity of images in a specific window will be calculated. 
The image point for image matching is not fixed in the object- 
based matching. 
The target for image matching can be a point or a line. The 
linear feature is a high level feature which can provide more 
geometric properties than point feature. As most of the facades, 
such as windows and doors, are composed of straight lines, the 
linear features are more suitable for the facade reconstruction 
when comparing to the point features. 
Several researchers have reported on the line matching for 
building reconstruction. Baillard et al, (1999) employs 
geometric constraints for line matching of aerial images based 
on multiview geometry and photometric similarity. McIntosh 
and Krupnik (2002) integrate airborne lidar data and aerial 
images to generate breaklines for digital surface model. The 
edge matching utilizes several constraints like epipolar lines, 
angle between lines, and correlation of gray value surround line. 
Pu and Vosselman (2009) integrate terrestrial lidar and images 
in a semiautomatic building facade reconstruction. Lidar 
provides plane features while the image provides linear features. 
The integration of these two data is able to modelling the facade 
as well as texture mapping. 
Facade reconstruction using close-range images is a challenging 
problem for several reasons. The scale variety of close-range