FUSION OF LIDAR DATA AND OPTICAL IMAGERY FOR BUILDING MODELING 
Liang-Chien Chen *, Tee-Ann Teo, Yi-Chen Shao, Yen-Chung Lai, Jiann-Yeou Rau 
Center for Space and Remote Sensing Research, National Central University, Chung-Li, Taiwan. 
(lcchen, ann, ycshao, stzac, jyrau)@csrsr.ncu.edu.tw 
Commission WG IV/7 
KEY WORDS: LIDAR, Optical, Image, Building, Reconstruction. 
ABSTRACT: 
This paper presents a scheme for building detection and building reconstruction from LIDAR data and optical imagery. The 
proposed scheme comprises two major parts: (1) detection of building regions, and (2) reconstruction of building models. Spatial 
registration of LIDAR data and optical images is performed as data preprocessing. Then, at the first stage, a region-based 
segmentation and knowledge-based classification are integrated to detect building regions. Once the building regions are detected, 
we analyze the coplanarity of the LIDAR raw data to shape the roof. The accurate position of walls of the building is determined by 
the integration of the edges extracted from optical imagery. Thus the three dimensional building edges can be used for the 
reconstruction. A patented method SMS (Split-Merge-Shape) is employed to generated building models in the last step. Having the 
advantages of high reliability and flexibility, the SMS method provides stable reconstruction even when those 3D building lines are 
  
broken. LIDAR data acquired by Leica ALS 40, QuickBird multispectral satellite images and aerial images were used in the 
validation. 
1. INTRODUCTION 
Building modeling is an essential task in the establishment of 
cyber city for city planning, management, and various 
applications. Building reconstruction may be performed by a 
photogrammetric procedure using aerial stereopairs. A number 
of researches have shown the approaches of combine data for 
building modeling, e.g., LIDAR (LIght Detecting And Ranging) 
and aerial image (Rottensteiner and Jansa, 2002), LIDAR and 
three-line-scanner image (Nakagawa, ef. al., 2002), LIDAR and 
high satellite image (Guo, 2003), LIDAR, aerial image and 2D 
map (Vosselman, 2002). 
To improve the degree of automation, we propose here a 
scheme that integrates LIDAR data and optical images for 
building modeling. LIDAR data provide high accurate 3D 
points but lack breaklines information. On the contrary, optical 
imagery with high spatial resolution provides more accurate 
breaklines information than LIDAR data. Moreover, 
multispectral imagery is beneficial to identification and 
classification of objects, such as building and vegetation. Thus, 
we propose to combine LIDAR data and optical imagery, such 
as QuickBird multispectral satellite images and high spatial 
resolution aerial images, for the building modeling. The 
multispectral satellite images provide spectral information for 
detecting the building region, and the aerial images provide 
texture information for building reconstruction. 
The proposed scheme comprises two major parts: (1) building 
detection, and (2) building reconstruction. Spatial registration 
of LIDAR data and optical imagery is performed as data 
preprocessing. The transformation between LIDAR space and 
image space is determined before the data fusion. It is done in 
such a way that two data sets are unified in the object 
coordinate system. Meanwhile, the exterior orientation 
  
* Corresponding author. 
parameters of the optical imagery are recovered by employing 
ground control points. In the stage of building detection, a 
region-based segmentation and knowledge-based classification 
are integrated. In the segmentation for surface elevation, the 
LIDAR points are resampled to raster form. A QuickBird 
multispectral image with is applied in this stage to improve the 
spectral information. Then, a 'knowledge-based classification 
procedure considering spectral, shape, texture, and elevation 
information is performed to detect the building regions. In the 
stage of building reconstruction, building blocks are divided 
and conquered. Once the building regions are detected, we 
analyze the coplanarity of the LIDAR raw data to shape the 
roof. Then, we perform TIN-based region growing to generate 
3D planes for a building region. The accurate position of walls 
of the building is determined by the edges extracted from aerial 
images. Thus the three dimensional building edges can be used 
for the reconstruction. 
A patented method SMS (Rau and Chen, 2003) is then 
employed to generate building models in the last step. Having 
the advantages of high reliability and flexibility, the SMS 
method provides stable reconstruction even when those 3D 
building lines are broken. The feature of SMS method allows 
incomplete 3D building lines due to occlusions or weak image 
features. 
2. PREPROCESSING 
The data preprocessing consists of two steps, which are 
interpolation of LIDAR data and space registration. 
Interna 
2.1.11 
The LH 
data. 
discrete 
DSM 
interpol 
(Behan, 
23 Sp 
Space r 
objectiv 
betweer 
control 
registrat 
coregist 
The obj 
regions. 
segment 
flow cha 
  
  
  
  
3.1 Reg 
There are 
contour-t 
using ed; 
scgmenta 
pixels wi