The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B3b. Beijing 2008 
710 
W represents the discrepancy vector; 
e stands for error vector; 
is the unknown parameter vector; 
Z is the covariance matrix of the observations while 
2 
0 is the variance component; 
c,n,u are the number of conditions, observations, 
and unknown parameters, respectively. 
To compensate for hidden parts, if any, our approach 
hypothesizes the hidden parts by tracking the higher edges of 
the neighboring building using Floyd-Warshall, the shortest 
path algorithm (Cormen et al., 2003). As demonstrated in 
Figure 9, the hidden parts (black dot lines) of the lowest 
buildings (colored black) are eventually compensated. 
3.3 Refine 
Two modes were considered for the refinement procedures. One 
is automatic mode and the other is semi-automatic mode 
designed for whenever meeting extraction and/or reconstruction 
deficiency. The details of two modes are illustrated as follows. 
3.3.1 Automatic mode: Having 2D line features available, 
automatic refinement considers the following steps: (1). 
Validation of building boundaries by imposing 2-D and 3-D 
geometric inferences in image space and in object space, 
respectively; (2). Merging and adjusting 3D line features of 
building boundaries from aerial images through adjustment 
computation. (3) Fusing initial building models from different 
sources (LIDAR system with image system). Figure 10 shows 
the proposed scheme for automatic refinement mode (Jaw and 
Cheng, 2007a). 
3.3.2 Semi-Automatic Mode: In order to implement the 
process of building roof reconstruction in a more efficient as 
well as reliable fashion, point-based, line-based and hybrid 
user-interface modules (Jaw and Cheng, 2007b) are designed as 
follows: 
Point-based: At point-based mode, the traditional approach of 
photogrammetric intersection by sequentially measuring the 
conjugate comer points in, at least, a stereo-pair is adopted. 
Then the comers determined by photogrammetric method are 
fused with those derived from 3D line features, ending up with 
refined roofs. 
Figure 10. The proposed scheme of automatic refinement mode 
Line-based: The approach of line-based module is similar to 
point-based one except that line features, instead of point 
features, are measured. 3D line features collected in this mode 
are the result of the intersections of conjugate 2D line features. 
Then the initial roof models of image system are formed by 
using Construct-Shape process. After that, refined roofs are 
determined by fusing initial roof models resulting from image 
system and LIDAR system. 
Hybrid module: The hybrid module provides the most flexible 
method for measuring the features. Demanded by the users 
when taking the measurements, point features, line features, or 
the combination of both can be measured. Again, once 3D line 
features are formed upon measurements, the Construct-Shape 
processes are followed for acquiring the initial roof models of 
image system. Then fusion of different roof models is 
undertaken for improving the roof quality. 
4. EXPERIMENTAL RESULT 
Reconstructing building roof models were tested by fusing 
LIDAR data sets with aerial imagery. The test area is in 
Hsinchu, Taiwan. The LIDAR data sets were collected in 2002 
by airborne laser scanning system while the images with a scale 
of 1:5800, B/H=0.225, were taken almost the same time as 
acquiring the LIDAR data sets. Figure 11 and Figure 12 depict 
the LIDAR DSM and aerial images, respectively, of the test 
area. The black lines in Figure 11 are 3D line features extracted 
from LIDAR DSM by 3D line feature extraction engine. Figure 
13 gives the results of initial roof models from LIDAR system, 
Figure 14 displays the results of refined roof models through 
CSR processes, where the red, blue and green roofs were 
obtained by point-based, line based and hybrid module, 
respectively, while the yellow roof was reconstructed by the 
automatic refinement mode. In addition, the white roofs show 
the results reconstructed purely based on images (that were 
missing from LIDAR data sets). Due to the lack of external 
reference data, we calculated the fitting error of LIDAR point to 
the roof obtained through CSR approach as a kind of quality 
indicator and 20cm of fitting error was found. To report the 
reconstruction accuracy, we actually looked at the theoretical 
accuracy of comer points through the adjustment output.