-rance. September 1-3. 2010 
In: Paparoditis N., Pierrot-Deseilligny M.. Mallet C... Tournaire O. (Eds). 1APRS. Vol. XXXVIII. Part ЗА - Saint-Mandé, France, September 1-3, 2010 
ms are assumed to comply 
litional constraints to the 
a correlation constraint 
plane and a spatiogram 
nilarity dominated by the 
nodels. 
3D plane fitted to the line 
t the assumptions that (i) 
elong to a single plane. A 
;a marked by the 3D lines 
'iate, since there may be 
teys. dormers etc). Thus, 
mmediate vicinity of the 
; plane, which can also be 
nd 5b). We fixed the side 
on the same direction of 
d = 2 nt. Fig. 5a and 5b 
d plane that is estimated 
However, there may be 
ane formation and the 
ection point of the lines 
ur on a different plane(s) 
(ii), the lines that really 
orm a plane (Fig. 5c - 
line pairs may be hidden 
). It is straightforward to 
lgle of the plane with its 
ily apply the correlation 
larrower than a specific 
er two violations cannot 
hypothesized 3D planes 
»orous experiments, we 
: of con-elation to a very 
ie constraint to eliminate 
egative correlation. 
d by the reference line 
luated. We select the 2D 
ie line pairs. However, it 
ions directly, since the 
long to many different 
ositions of the pixels to 
logical to compare the 
since many parts of the 
ormation; very different 
is. Therefore, we utilize 
ional similarity between 
patiogram measure is 
that it has a unique capability to combine the distribution of the 
radiometric information along with the spatial information. Thus, 
the positional differences occur between the line pairs are handled 
(somewhat alleviated) while providing the histogram information. 
The details of the algorithm are explained in (O Conaire et. al., 
2007), for our case, we fixed the number of bins to 8 for each band 
and applied a spatiogram similarity threshold of T S p a „ ogram > 0.75 
(between 0 and 1) for regional similarity comparison. 
The final pair matches are assigned after a weighted pair-wise 
matching similarity score which is computed over a total of eight 
measures; an epipolar, three geometric, two photometric, a 
correlation and a spatiogram measure. All the measures are 
normalized from 0 to 1 prior to the calculation, and the total 
similarity result is computed as the average of all similarities. 
However, the pair-wise matching does not always guarantee one to 
one matches for each line. Once all the pair-wise matches are 
collected, a probabilistic reasoning is applied to solve the matching 
ambiguity problem. Since a single line is allowed to have a part in 
different pair models, after the pair-wise matching stage, we have 
generally sufficient number of matching redundancy for lines. 
Thus, we have possibility to eliminate particular false line matches 
using this redundancy. In this study, we selected the best line 
correspondences with a single probability threshold (p > 0.6). In 
general, the threshold is exceeded for most of the ambiguities and 
provided very good results. If the probability value computed is 
found to between 0.5 and 0.6, in this study, we keep all the line 
matches. For an immediate future work, our aim is to design a 
better decision scheme by applying a higher level probabilistic post 
processing. 
3. RESULTS AND DISCUSSION 
We processed three urban test sites from Germany to assess the 
performance of our methodology. The first image pair was acquired 
over a densely built up area of the city of Vaihingen (Fig. 6 - 1 st 
column) by the DMC digital camera with 70% forward overlap 
(Cramer and Haala, 2009). The focal length of the camera was 120 
mm and the flying height was approximately 800 m above the 
ground level which corresponds to a final ground sampling distance 
(GSD) of approximately 8 cm. The second image pair (© 
Geoinformation Hannover) belongs to the Schneiderberg region of 
Hannover (Fig. 6 - 2 nd column). The images were taken with RMK 
TOP30 analog camera with standard 60% forward overlap. The 
calibrated focal length of the camera was 305.560 mm with a flying 
height of approximately 1600 m. The images were scanned at 14- 
pm resolution and this corresponds to a final GSD of 7-cm. The 
third image pair (© Geobasisdaten: Land NRW. Bonn, 2111/2009) 
was acquired from the city of Dorsten (Fig. 6 - 3 ld column) where 
the stereo pairs are acquired with DMC camera with 55% percent 
forward overlap. The flying height was around 2000 m above the 
ground level; thus, the GSD of the images was around 20 cm. 
For all test sites, the number of correct and false line matches was 
assessed manually and the number of correct matches is computed 
to be higher than 92% (Table 1). Similarly, the number of false 
matches for all sites is comparable; however, the highest number of 
false matches (8%) is computed for the Hannover image pair. In 
fact, this is an expected result, since the images of the Hannover 
dataset were taken by an analog camera and scanned afterwards. 
Thus, the quality of the images and the noise level involved has 
probably affected the quality of the extracted lines. For each test 
Lines having 
Number of 
RMS Distance 
Lines 
Planes 
cm 
pixels 
1-neighboring plane 
21 
21 
12.7 
1.59 
2-neighboring planes 
38 
76 
13.9 
1.74 
no plane 
1 
- 
- 
- 
Total 
60 
97 
13.6 
1.70 
ited from the line pairs, 
: constraint. 
Test Site 
# of lines 
Matches 
Left 
Right 
Total 
Correct 
False 
Vaihingen 
1726 
1875 
963 
909 (94%) 
54 (6%) 
Hannover 
2339 
2369 
1038 
954 (92%) 
84 (8%) 
Dorsten 
2724 
2764 
1598 
1516(95%) 
82 (5%) 
Table 1. The matching results of the proposed methodology. 
Table 2. RMS distances between the reconstructed lines and 
neighbouring planes 
site, if we compare the total number of matches with respect to the 
number of lines found in each image, the highest percentage of 
matching (58%) was achieved for the Dorsten image pair. This can 
be explained with the characteristics of the buildings in the test site, 
since most the buildings have less detail and complexity with 
respect to the other test sites. In addition, the spatial resolution of 
the image pair used is relatively coarse (20 cm), thus, most of the 
lines extracted belong to the main body of the buildings which is 
suitable to be matched using a pair-wise approach. 
For the Vaihingen data set, the accuracy of the reconstructed lines 
could be evaluated by comparing them to LIDAR data. The LIDAR 
data of the test site were captured with a Leica ALS50 system with 
an accuracy of 3.4 cm (Haala, 2009). In order to compare the 
reconstructed lines, we randomly selected 60 reconstructed 3D lines 
and automatically extracted 3D planes from the point cloud in the 
vicinity of each line. Depending on the type of the line, this plane 
reconstruction process resulted in one plane if the line corresponded 
to a step edge and in two planes if the line corresponded to the 
intersection of two planes. In one case no such planes could be 
found. For each of the 59 lines, we determined the line's average 
orthogonal distance from its neighbouring planes and used these 
distances to compute the RMS average distance between the 
reconstructed lines and the LIDAR planes. The RMS distance was 
determined separately for lines corresponding to one plane and 
those corresponding to two; furthermore, a total RMS distance was 
also determined (Table 2). The total RMS distance, based on 59 
lines and a total of 97 planes, was 13.6 cm. This corresponds to 1.7 
pixels in the original images (GSD 8 cm). Several different error 
sources (inaccuracies during line extraction, epipolar alignment 
problem, the accuracy of the reference data itself etc.) may be 
involved in this result. As we used a simple perspective model 
without any additional parameters for the image orientations, we 
believe that a considerable portion of the error budget may come 
from the un-modelled systematic errors. 
4. CONCLUSIONS AND FUTURE WORK 
A new approach for the reconstruction of 3-D line segments from 
multispectral stereo aerial images was proposed. The following 
aspects outline the contributions of our approach: (i) the 
methodology take full advantage of the existing multispectral 
information in aerial images all over the steps especially during the 
pre-processing and edge detection. Thus, even building boundaries 
that show only a very slight color difference could be detected, (ii) 
With the improvements performed to the straight edge detector, the 
straight line extraction algorithm works quite robust, even for the 
areas where an enormous number of edges were found. This offers 
an opportunity to detect and reconstruct lines that belong to 
buildings and their certain details, (iii) To establish the line 
correspondences between the stereo image pairs, a new pair-wise 
stereo matching approach is presented. The approach involves new 
constraints, and the redundancy inherent in pair relations gives us 
possibility to reduce the number of false matches with a 
probabilistic manner. 
The final results of the methodology are encouraging. Our approach 
shows good results for the stereo line matching and 3D line 
reconstruction for the buildings located in dense and complex 
environments. For an immediate future work, our aim is to design a 
better decision scheme by applying a higher level probabilistic 
29