International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol XXXV, Part B3. Istanbul 2004 
  
resolution. In spite of errors in geo-coding caused by errors in 
the DSM at the building outlines, the segment boundaries in 
figure 4 match the actual roof boundaries quite well. 
  
Figure 4. Left: planar segments detected in the DSM (fig. 3). 
Centre/right: Homogeneous segments from two aerial 
images projected to the DSM. Resolution: 0.1 m. 
The DSM label image is resampled to the same resolution as the 
projections of the image segmentation results. We obtain 
altogether N + / label images, where N is the number of aerial 
images. For each pixel i, we obtain a tuple /; = {Ip; 1}; ..., Int 
of corresponding labels, indicating a matching candidate 
between a planar segment /j5 and N image segments //;; ..., /yjj. 
We determine the number 7; of occurrences for each candidate 
h;. We also compute the percentage pj; = n; / n; of each segment 
1; that contributes to /;; where n; is the number of pixels of /; in 
the label image /, with 7 € /D, 1, ..., N}. 
Due to segmentation and geo-coding errors, the set of matching 
candidates h; will contain errors. Hence, we firstly classify each 
hypothesis h; according to the percentages pj. A hypothesis is 
classified to have strong support if it has at least one component 
J with p;; > 50%. Otherwise, it is said to have partial support if 
there is at least one component with 33% < pj; € 5096, or weak 
support if there is there is at least one component with 525 « pj; 
< 33%. If pj; € 5% for all components of the hypothesis 7; or if 
the number 7; of pixels giving support to it is below a certain 
threshold, 4; is eliminated. Further, if for a hypothesis 7; there 
exists another hypothesis A; = {pe li ..., Iw that has a higher 
support than A; (thus, if there is a planar segment D; # 0 
corresponding to a larger portion of the co-occurrence of image 
segments /j; ... , /y; than Dj), then /; is eliminated as well. In 
this way, contradicting hypotheses between planar segments 
and tuples of image segments are eliminated. 
A set of hypotheses //; € {h;} for each planar segment D; # 0 is 
thus obtained, consisting of all hypotheses 4; for which the first 
component is D;. We improve the initial segmentation by region 
growing, taking into account the matching results. Each pixel 
not yet assigned to a planar segment is tested according to 
whether it belongs to segment D; by computing its height from 
the planar equation of that segment (to avoid errors at the 
building outlines). The resulting 3D point is back-projected to 
all images, and the image labels at the projected positions are 
evaluated. If the set of labels corresponds to one of the 
hypotheses /;, the pixel is assigned to segment D; (figure 5). 
In order to improve the initial segmentation by extracting new 
planar segments, it would be necessary to evaluate the 
hypotheses Ajo i.e. the hypotheses of multiple image labels 
corresponding to areas not yet classified. It would be necessary 
to first compute the plane parameters of these new segments 
using the DSM. This has not yet been implemented. 
A major advantage of our technique is that using multiple 
images can mitigate segmentation errors. For instance, if two 
planes are merged in one image due to poor contrast, but 
correctly separated in another, the algorithm overcomes this 
problem because any coincidence of two or more image labels 
is considered to be a new *combined" image label. 
516 
The left part of figure 5 shows the results of region growing. 
The planar segments resemble the roof planes much better than 
the initial segments from the DSM. The segment boundaries are 
smoothed by morphologic filtering, and co-planar neighbouring 
segments are merged, which results in the label image in the 
centre of figure 5. The right part of figure 5 shows the Voronoi 
diagram of that label image (Fritsch and Ameri, 2000). It is 
used to derive the neighbourhood relations of the planar 
segments, and the boundaries of the segments in the Voronoi 
diagram provide the first estimates of the roof boundaries. 
  
Figure 5. Left: planar segments after region growing. Centre: 
the segments after morphologic filtering and merging 
of co-planar segments. Right: Voronoi diagram. 
3.2 Delineation of the 3D Roof Planes 
Rottensteiner and Briese (2003) describe how each portion of 
the original boundary polygon which separates two planes is 
classified according to whether it is an intersection line, a step 
edge, or both, in which case that polygon portion is split into 
smaller parts. The positions of the intersection lines can be 
determined precisely by the intersection of the neighbouring 
planar segments. The location of the step edges is critical, 
especially if it has to be carried out using LIDAR data alone. 
Here we want to show how the initial boundary polygons can 
be improved using edges extracted from the aerial images by 
polymorphic feature extraction. 
We start by computing 3D straight line segments from image 
edges. As the boundary polygon of a roof plane has to be 
situated in that roof plane, it is possible to match edges 
extracted from different images by using the assumption that 
these edges are situated in or at the border of the plane. Thus, in 
order to delineate the boundary of a roof plane, we project all 
the image edges in a certain neighbourhood of the approximate 
polygon to that plane. If the projections of two edge segments /, 
and /; from two different images are found to be almost parallel 
(indicated by a small angle a between their normal vectors, e.g. 
a < 15°) and if there is at least one point on one of the segments 
that has a distance from the other segment smaller than another 
threshold (e.g. 0.25 m), the image edges are assumed to be the 
images of the same straight line in object space. This straight 
line is computed by adjustment through the end points of the 
projected image edge segments. The end points of the combined 
3D segment are determined so that the combined segment 
merges both projected image segments (Figure 6). If two such 
adjusted straight line segments are found to overlap in object 
space, they will be merged. Thus, we favour long object edges. 
lio 
l» 
  
Figure 6. Matching and merging edge segments. /, and /»: edges 
from images 1 and 2. /;>: combined edge, covering the 
projections of both /, and /;. 
Next, these 3D straight line segments have to be matched with 
the approximate roof polygons. Again, this matching is based 
on geometric proximity and parallelism, with relatively loose 
thresholds because of the poor quality of the approximate