In this study, a new approach integrating very high resolution 
imagery and Lidar data is proposed to automatically obtain 
detailed building boundaries with precise geometric position. 
The proposed approach can be proved to preserve the boundary 
details, including some tiny segments at a comer, a short 
segment. Since the boundaries with precise geometric position 
can be directly extracted from very high resolution images (e.g., 
5cm spatial resolution); our approach is focused on improving 
the correctness and completeness of the extracted boundaries by 
integrating Lidar data. This process consists of four steps: Lidar 
data processing, building image generation, line segment 
extraction, and boundary segment selection. Firstly, the 
segmented building points need to be determined from raw 
Lidar data. Secondly, a building image is generated by a 
building bounding rectangle and a building buffer. Thirdly, a 
new algorithm is proposed for determining the principal 
orientations of building boundaries based on rough principal 
orientations constraint, which ensures the accuracy and 
robustness of the subsequent line segments extraction. Finally, 
an algorithm based on Lidar point density analysis and Kmeans 
clustering is proposed to provide a dynamic way to accurately 
identify boundary segments from non-boundary segments. 
The proposed approach is focused on building boundary 
extraction and can be used in 3D building model reconstmction 
and 2D building digital line graph generation. Stereo aerial 
images should be selected for 3D building reconstruction and 
the extracted boundaries will be the basic elements of the 
subsequent processes such as line segments matching and 3D 
line segments generation. While an orthoimage would be more 
appropriate than an aerial stereo pair for getting a 2D digital 
line graph. Aerial stereo, orthoimage, or some other images can 
be processed by using a little different strategy of data 
registration as declared in the next section. 
2.1 Data pre-processing (Lidar data processing) 
In order to obtain the segmented building points from raw Lidar 
data, the first process is usually to separate the ground points 
from non-ground points, and then identify the building points 
from non-ground points. Numerous algorithms have been 
developed to separate ground points from non-ground points. 
Sithole and Vosselman (2004) made a comparative study of 
eight filters and identified that all filters perform well in smooth 
rural landscapes, but all of them produce errors in complex 
urban areas and rough terrain covered by vegetation. They also 
pointed out that the filters estimating local surfaces were found 
to perform best. So the linear prediction algorithm proposed by 
Kraus and Pfeifer (1998) is used for deriving bare DEM from 
raw Lidar data. Comparing the bare DEM and the raw Lidar 
data, non-ground points can be identified. 
In a dataset that contains only non-ground points, building 
points need to be separated from non-building data (mainly 
vegetation). The region-growing algorithm based on a plane 
fitting technique proposed by Zhang, et.al.(2006) is used. In this 
process, areas of non-ground points are firstly found and 
labelled by connecting the eight neighbours of a cell. For each 
non-ground area, inside and boundary points are identified. 
Then non-ground points for each area are segmented by region 
growing based on a plane-fitting technique. Finally, the 
segmented patches for non-building objects are removed. The 
remaining patches are identified as building patches. It is 
reported that the omission and commission errors of determined 
building are 10% and 2% respectively using this approach 
Building image generation 
In order to retrieve the interested building features from a very 
high resolution image, a building image is firstly generated to 
reduce the complexity of processes. In a building image, only 
one building is covered and non-building features are removed. 
A building image is generated by 3 steps as follows. 
(a)aerial image 
(b) project Lidar data onto (a) 
X 
(c) BR(white) from Lidar data (d) cut (a) using the BR 
(e) a buffer from Lidar data 
(f) a building image 
Figure 1. Steps of a building image generation 
Step 1, Data overlay 
In this step, images from different sensors can be processed 
using different strategies of data registration for different 
specific applications. An aerial stereo with orientation 
parameters is used in this study, Lidar points are directly 
projected onto the aerial stereo by collinearity equation. If 
necessary, the orientation parameters can be refined by block 
bundle adjustment. If an orthoimage is used, it can be directly 
overlain by Lidar data, as both spatial references are at the same 
coordinate system. For an image with unknown orientation 
parameters, the overlay between the image and Lidar data can 
be done by a manual co-registration operation. Figure 1 (a) is an 
oriented aerial image, Figure 1 (b) are the results by projecting 
the pre-processed Lidar data onto the aerial image using 
collinearity equation. 
Step 2, Image cutting by a bounding rectangle (BR) 
After a convex hull is constructed based on the projected Lidar 
points in 1 (b), a bounding rectangle (BR) of a building can be 
created based on the convex hull which is shown as a white 
rectangle on the image in 1 (c). The BR should be enlarged with 
a threshold to ensure all the boundaries of a building in the 
aerial image can be fully covered. The result cut from the aerial 
image in 1 (a) using the BR is shown in 1 (d). 
Step 3, Image filtered by a buffering zone 
A raster image is generated by interpolating the projected Lidar 
data in Figure 1 (b), and then a buffering image can be created 
shown in Figure 1(e). Figure 1(f) is the result by filtering the 
Figure 1(d) using the buffering zone, in which non-building 
features are removed from the image to get a building image. 
694