TT 
the object 
ted DTM 
CS Which 
)Ving the 
ti-sensor. 
POT data 
anning in 
0togram- 
g Extrac- 
irkhäuser 
3D. Com- 
d Spiller, 
of Photo- 
al. (eds.): 
7 
ional Ar- 
Norkshop 
Network 
| Images. 
N orkshop 
om à case 
de (NL), 
ammetric 
ver, CD- 
base revi- 
Gruen, A 
tion mod- 
  
Stephan Scholze 
EXPLOITING COLOR FOR EDGE EXTRACTION AND LINE SEGMENT STEREO MATCHING 
IN HIGH-RESOLUTION AERIAL IMAGERY 
Stephan SCHOLZE, Theo MOONS, Frank ADE, Luc VAN GOOL 
Swiss Federal Institute of Technology 
Communication Technology Lab 
Computer Vision Group 
{scholze, ade, vangool } @vision.ee.ethz.ch 
Theo.Moons @esat.kuleuven.ac.be 
Working Group III/3 
KEY WORDS: Image matching, Multi-spectral data, Automation, Edge extraction, Buildings. 
ABSTRACT 
This paper investigates into the added value of color information for edge extraction and straight edge segment matching 
between stereo views. For edge extraction in color images different methods proposed in the literature are evaluated 
and compared, paying special attention to significance and completeness of the obtained edge-map. To find related edge 
segment pairs in different views, we apply an odd-man-out scheme: starting with all geometrically possible pairs we first 
rule out pairs, for which the chromatic information provided by the regions flanking the edge segments is incompatible. 
To further restrict the number of pairs we compute a chromatic similarity measure based on cross-correlation in the color 
bands. Both steps result in a significant reduction of candidate pairs, yet no correct pairs get lost. A main application of 
our technique is for automatic 3D building reconstruction from high resolution aerial images. 
1 INTRODUCTION 
The automatic reconstruction of 3D city models from high resolution aerial images is still an area of active research. The 
currently available color images of dense urban scenes have nowadays a resolution of 10 x 10 cm? on the ground or 
better. This huge amount of data with its immanent information is on one hand the key to better solve the correspondence 
problem, on the other hand it holds enormous complexity. This complexity might be the reason, why color information 
was not used to its full extent until now. 
Our strategy to sytematically reduce the complexity of the problem is based on an iterative scheme. After each reduction 
step a more sophisticated elimination procedure can be applied, since the number of remaining combinations gets smaller 
after each step. Since the vast majority of man-made objects, especially building roofs, are of polygonal shape with 
straight lines delimiting the roof edges, we find it natural to base our method on straight line segments. Thus our method 
could be labeled feature-based. Moreover, it is fully based on color information; at no time we refer to a grey-level image 
às data source. The main contributions of our work are: edge detection in the color images and the use of chromatic 
information for the stereo matching algorithm. 
For edge extraction in color images we have evaluated and compared two different methods proposed in the literature. 
The key problem is the fusion of the gradient information from the individual RGB-bands. The maximum of the norms of 
the gradients in the three bands turns out to be a good compromise between computational simplicity and completeness 
of the edge map (Section 2). 
For 3D building reconstruction straight line segments at roof edges need to be matched between overlapping views. This 
task is not straightforward. First, the edge detection step will always produce incorrect results to some degree. True 
Image features might be detected only partially, continuous contours in the image might get fragmented, even cases where 
Important image features are not detected at all can happen in real world applications. Second, due to the weak epipolar 
constraint, we will find a huge number of possible pairings between extracted edges in different views. A calculation of 
all geometrically possible 3D line reconstructions would therefore yield mostly futile results. To overcome the geometric 
ambiguities, we take into account the color of regions flanking the extracted line segments. By comparing the integrated 
Color distribution in the flanking regions of putative pairs, we determine a statistical chromatic similarity measure. This 
Step reduces the possible combinations to typically 30 percent, before calculating the 3D reconstruction. For the remaining 
Pairs we calculate the 3D information. A further significant reduction to about 20 percent is now achieved by computing 
Across-correlation based chromatic similarity measure, exploiting the pixel-wise one-to-one correspondence induced by 
'Pipolar geometry (Section 3). 
  
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B3. Amsterdam 2000. 815