Edward M. Mikhail 
  
Thresholding operation: The thresholds, thus computed, were used to get the result shown in Figure 16. Note that the 
rooftops have been color-coded by the 
identifying zone. 
  
Discussion: In the above analysis, we 
     
N © identified the key attributes of the 
A Aa lcm. respective datasets available to us, 
Figure 16. Rooftops Identified via Data-Fusion Spectral data is best used in the 
  
  
  
identification of elemental 
composition, while the DEM identifies the data element in the functional sense. Data fusion is thus justifiable, with the 
analysis utilizing the respective attributes of the HYDICE data and the DEM towards the target application. 
It is critical to point out that the quality of the fusion of the DEM data and the hyperspectral data depends on the rigor 
of modeling the hyperspectral sensor. As was shown in section 2, the Gauss-Markov and use of linear features yielded 
excellent rectification of that imagery. It will also be shown in subsequent section that using computer vision 
techniques on frame imagery fused with other data depends upon rigorous photogrammetric modeling. 
3.3 Building Extraction 
Computer vision based three-dimensional urban feature extraction from frame imagery encounters many sources of 
difficulty including those of segmentation, 3-D inference, and shape description. 
Segmentation is difficult due to the presence of large numbers of objects that are not intended to be modeled such as 
sidewalks, landscaping, trees and shadows near the objects to be modeled. The objects to be modeled may be partially 
occluded and contain significant surface texture. 3-D information is not explicit in an intensity image; its inference from 
multiple images requires finding correct corresponding points or features in two or more images. Direct ranging 
techniques such as those using LIDAR or IFSAR can provide highly useful 3-D data though the data typically have 
areas of missing elements and may contain some points with grossly erroneous values. 
Once the objects have been segmented and 3-D shape recovered, the task of shape description still remains. This 
consists of forming complex shapes from simpler shapes that may be detected at earlier stages. For example, a building 
may have several wings, possibly of different heights, that may be detected as separate parts rather than one structure 
initially. The approach used in this effort is to use a combination of tools: reconstruction and reasoning in 3-D, use of 
multiple sources of data and perceptual grouping. Context and domain knowledge guide the applications of these tools. 
Context comes from knowledge of camera parameters, geometry of objects to be detected and illumination conditions 
(primarily the sun position). Some knowledge of the approximate terrain is also utilized. The information from sensors 
of different modalities is fused not at pixel level but at higher feature levels. 
Our building detection system is based on a "hypothesize and verify" paradigm. This system can function with just à 
pair of panchromatic (PAN) images, but can also utilize more images and information from other modalities. This 
system also incorporates abilities for Bayesian reasoning and machine learning. 
3.3.1 Multi-View System, or MVS 
A block diagram of the extraction system is shown in Figure 17. The approach is basically one of hypothesize and 
verify. Hypotheses for potential roofs are made from fragmented lower level image features. The system is hierarchical 
and uses evidence from all the views in a non-preferential, order-independent way. Promising hypotheses are selected 
among these by using relatively inexpensive evidence from the rooftops only. The selected hypotheses are then verified 
by using more reliable global evidence such as from walls and shadows. The verified hypotheses are then examined for 
overlap which may result in either elimination or in merging of them. This system is designed for rectilinear buildings. 
complex buildings are decomposed into rectangular parts. Rooftops thus project to parallelograms in the images (the 
projection is nearly orthographic over the scale of a building). Lines, junctions and parallel lines are the basic features 
used to form roof hypotheses. Consider the images shown in Figure 18. The images are from the Ft. Hood, Texas, site 
that has been in common use by many researchers. The low level features composed of lines, junctions between lines 
and sets of parallel lines are matched among the available views. Two views were used in this example. The set of lines 
extracted from the image (using a Canny edge detector) to start the process is shown in Figure 19. Roof hypotheses art 
formed by a pair of matched parallel lines and U structures (Us represent three sides of a parallelogram). A pair of 
parallel lines may be matched to parallels in more than one view (when more than two views are used) and ead 
matching pair is considered. Closed hypotheses are formed from these features by using the best available image liné 
if any, else closures are synthesized from the ends of the parallel lines. 
  
600 International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B3. Amsterdam 2000. 
  
visit 
imag 
evide 
ques 
conf 
Aftei 
hypo 
also 
builc 
build 
As d 
acqu 
The | 
imag 
multi 
avail 
DEM 
accoi