In: Paparoditis N., Pierrot-Deseilligny M.. Mallet C., Tournaire O. (Eds). IAPRS. Vol. XXXVIII. Part 3A - Saint-Mandé, France, Septeniber 1-3, 2010 
A NEW STRAIGHT LINE RECONSTRUCTION METHODOLOGY 
FROM MULTI-SPECTRAL STEREO AERIAL IMAGES 
A. O. Ok a ' *, J. D. Wegner b , C. Heipke b , F. Rottensteiner b , U. Soergel h . V. Toprak 3 
a Dept, of Geodetic and Geographic Information Tech.. Middle East Technical University, 06531 Ankara. Turkey 
- (oozgun, toprak)@metu.edu.tr 
h Institute of Photogrammetry and Geoinformation, University of Hannover, 30167 Hannover, Germany 
- (wegner, heipke, rottensteiner, soergel)@ipi.uni-hannover.de 
Commission III - VVG 111/4 
KEY WORDS: pair-wise line matching, line reconstruction, straight line extraction, stereo aerial images 
ABSTRACT: 
In this study, a new methodology for the reconstruction of line features from multispectral stereo aerial images is presented. We take full 
advantage of the existing multispectral information in aerial images all over the steps of pre-processing and edge detection. To accurately 
describe the straight line segments, a principal component analysis technique is adapted. The line to line correspondences between the stereo 
images are established using a new pair-wise stereo matching approach. The approach involves new constraints, and the redundancy inherent 
in pair relations gives us a possibility to reduce the number of false matches in a probabilistic manner. The methodology is tested over three 
different urban test sites and provided good results for line matching and reconstruction. 
1. INTRODUCTION 
Reliable extraction of corresponding lines in overlapping images 
can be used for different purposes such as 3D object extraction, 
improving the automated triangulation, image registration, motion 
analysis etc. Therefore, line matching is a challenging task, and still 
a vivid field of research. Up to now, a significant number of 
research papers have been published in this field; here, we only 
review the methods developed to extract 3D line features from 
aerial images. A useful classification of existing line matching 
approaches was proposed by Schmid and Zisserman (1997). They 
divided the line matching algorithms into two types, (i) those that 
match individual line segments, and (ii) those that match groups of 
line segments. In any case, the search space for matches has to be 
pruned in some way in order to limit complexity. For most of the 
studies, basic geometric parameters of line segments such as 
orientation, length, mid-point, etc. are involved to filter the set of 
correspondence hypotheses; however, probably the most preferred 
constraint is the quadrilateral constraint generated using the 
epipolar geometry (Collins et. al., 1998; Noronha and Nevada, 
2001; Kim and Nevatia, 2004; Suveg and Vosselman, 2004). Some 
studies also integrated the radiometric information around the line 
segments along with the geometrical primitives (Henricsson, 1998; 
Scholze et. al., 2000; Zhang and Baltsavias, 2000). Additional 
constraints such as uniqueness and ordering (Suveg and Vosselman, 
2004), figural continuity (Zhang, 2005) can also be included; 
however, even for a simple stereo line matching problem, these 
constraints are not sufficient to solve the image to image multi 
correspondence problem. Thus, additional effort has been spent on 
different algorithms to select the best line correspondences. For 
example, dynamic programming (Yip and Ho; 1996), weighted 
criterion functions (Henricsson, 1998), modal analyses (Park et. al., 
2000), and probability relaxation (Zhang and Baltsavias, 2000; 
Zhang, 2005) are among those approaches. On the other hand, 
additional view(s) can also be incoiporated in the line matching 
stage; some examples can be found in Schmid and Zisserman 
(1997), Collins et. al. (1998), Noronha and Nevatia (2001), and 
Kim and Nevatia (2004). 
Pair-wise stereo line matching is also introduced in several studies; 
for example, Park et. al. (2000) proposed a matching approach 
which takes into account only geometric relations between the 
lines. However, they assumed that the lengths and angles between 
the lines in a pair are almost exact copies of each other. Zhang and 
Baltsavias (2000) proposed a pairwise edge matching approach 
using relaxation labelling for 3D road reconstruction. Due to the 
nature of the relaxation algorithms, the smoothing effects and the 
issue of convergence with a large execution time could be the main 
concerns. Noronha and Nevatia, (2001), and Kim and Nevatia 
(2004) proposed a pair-wise approach based on the epipolar 
geometiy. However, they utilize multiple images to reduce the pair 
wise ambiguities and due to constraint of orthogonality, their 
approach is restricted for the line pairs detected over rectilinear flat 
objects. 
So far, in a stereo environment, the ambiguity problem of line 
matching is an issue that remain unsolved. The major problem 
arises from the lack of measure(s) and/or constraint(s) for line 
features that are invariant under different viewing conditions. Up to 
now, the general attempt to reduce the ambiguity problem is to 
strengthen the geometrical constraint by integrating one or more 
additional views (Schmid and Zisserman, 1997; Collins et. al., 
1998; Noronha and Nevatia, 2001; Kim and Nevatia. 2004; Zhang. 
2005). Several others utilized external DSMs (Jung and Paparoditis, 
2003; Taillandier and Deriche, 2004) or supplementary matched 
point features (Zhang, 2005) for both reducing the search space and 
filtering out the matching ambiguities. Nevertheless, the final 
matching performance of those algorithms is highly dependent and 
determined by the efficiency and the quality of those auxiliary 
datasets. On the other hand, the probabilistic relaxation based 
methods (Zhang and Baltsavias, 2000; Zhang, 2005) utilize the 
predefined local neighbourhood information which mostly suffer 
from the piecewise smoothness constraints involved. Inevitably, 
smoothing based on the local neighbourhood violates the standpoint 
of height discontinuity (except artificial edges such as shadows etc.) 
of the edges and the subsequent line matching. 
One important different aspect from the review of the literature is 
that while aerial images have been rich of multispectral 
information, this fact was completely discarded or not efficiently 
used during the low level processing such as filtering, edge 
detection etc. However multispectral aerial images provide 
opportunities to extract line features that cannot be detected in the 
grayscale images (Scholze et. al., 2000, Koschan and Abidi, 2005) 
due to several reasons, such as low contrast, accidental object 
alignments etc. 
2. METHODOLOGY 
In this study, we introduce a new methodology for the 3D 
reconstruction of line features from multispectral stereo aerial 
images. In order to maximize the line detection completeness, we 
take full advantage of the existing multispectral information in 
aerial images throughout the steps of pre-processing and edge 
Corresponding author.