Bofang Zhao 
  
these segments. Based on the matched lines, buildings are reconstructed by piecewise plane formation and plane 
intersection. This paper will describe the first two components of the system in Section 3 and 4 respectively. 
3. BUILDING DETECTION 
3.1 DSM Segmentation 
Building detection is primarily based on DSM segmentation, which is motivated by the common knowledge that 
buildings are higher than the surrounding topographic surfaces. DSMs can be either derived by stereo matching with 
pairs of images, or obtained using airborne laser-scanner, as in (Hug, 1994; Hug and Wehr, 1997). Since the DSM is 
used only for building detection, and texture and shadow information is incorporated into the detection process, high 
accuracy of DSMs is not required in this step. As demonstrated in Section 5, DSMs generated from using general 
commercial digital photogrammetric systems, such as Socet Set and VirtuoZo, are adequate. 
A DSM not only models all 3D objects in the scene, but also models the terrain surface, on which the objects are 
located. Hence, the first step is to separate “high objects” from the terrain surface. Several techniques have been 
presented for this step, including edge detection in DSMs, grey-scale mathematical morphology method (Weidner and 
Forstner, 1995) and “multiple height bins (MHB)" method (Baltsavias et al., 1995). In our approach, grey-scale 
mathematical morphology is applied. Specifically, it involves four steps: (i) “erosion” operation followed by “dilation” 
operation on DSM with a structural element S to derive an approximate DEM; (ii) subtraction of DEM from DSM to 
yield “height peaks”; (iii) thresholding the “height peaks” to extract regions of interest (ROIs); (iv) filtering ROIs. 
During the above process, domain knowledge of minimum size, maximum size and minimum heights of buildings in 
the area, together with the context knowledge of the image scale are employed. 
3.2 ROI Refinement 
It is clear that besides buildings, the initial ROIs derived from DSM may also include other “high objects” like trees. If 
DSM is accurate enough, information of surface normals of the DSM can be used to distinguish buildings from other 
objects (Brunn and Weidner, 1998). In our approach, texture information has been utilised for this purpose. While 
existing texture measures are usually unable to distinguish all classes of objects present in aerial images with desirable 
accuracy, texture characteristics within the constrained areas, i.e. the detected ROIs, are discernible. Texture filters 
proposed by Laws (1980) are used for texture calculation on ROIs in our approach. Classification of the texture images 
can be accomplished by Bayesian methods. For practical purposes it seems necessary to establish a texture database in 
the system and to develop different approaches for the various types of buildings. Since the proposed system is 
modular, modifying this component will not affect other modules of the system. After classification, shape attributes are 
calculated for each refined ROI as auxiliary information for subsequent processing. Domain knowledge of minimum 
size of buildings is again applied to reject small regions. Since the building region on texture image is usually smaller 
than its true size, due to the effect of widow size of the texture filters, the refined ROIs are expanded up to half of the 
window size through morphological operation “dilation”. 
An Sun's orientation 
     
    
  
building shadow line 
A gradient direction 
Figure 2. Building Shadow Lines 
3.3 ROI Verification 
To improve reliability in the reconstruction process, the hypothesised ROIs require further verification. In our system 
this evidence is provided by shadows cast by the buildings. In addition, building shadow lines will also be used in 
  
1028 International Archives of Photogrammetry and Remote Sensing. Vol, XXXIII, Part B3. Amsterdam 2000. 
  
be wo po v^ € ju 
— 227