Kerry McIntosh 
extraction and matching of line segments, reconstruction of the matched line segments to 3D, classification of the laser 
data, incorporation of breaklines, and the creation of a triangulated irregular network (TIN) using constrained 
triangulation. 
3.2.1 Edge Extraction. The edge pixels are detected in each image of the stereo pair of digital images. At present, the 
optimal zero-crossing operator (Sarkar and Boyer, 1991) is used for the detection of the edge pixels. The edge pixels 
are analyzed to find connected edge chains, and using these chains, straight line-segments are determined. Line 
segments are utilized as they adequately describe man-made objects and can be more accurately located than point 
features when using feature matching techniques (Fradkin and Ethrog, 1997). Also, each line segment is easily 
described using the start and end points of the segment and the gradient of the line. 
The automatic extraction of edges has certain limitations that must be addressed. Using automatic extraction methods, 
not all surface discontinuities are detected due to factors such as lack of contrast in the imagery. There will be a number 
of incomplete or incorrect breaklines, and other breaklines, that do not represent surface discontinuities, will be 
detected. These particular breaklines will refer to visual edges in the images that are purely changes in gray value, such 
as line markings on roads. The effect of including these breaklines in the data fusion process is to be investigated, 
however it is not expected to be detrimental to the process. 
3.2.2 Line Segment Matching. The segment matching approach used for this research was based on that of Schmid 
and Zisserman (1997), which employs the fundamental matrix (F matrix) to guide the matching process. Line segments 
are compared using the correlation coefficient for corresponding points between line segments. The two segments for 
which the highest average correlation coefficient is determined are taken to be the correctly matched line segments. 
The search space is constrained using epipolar geometry and by determining the maximum disparity between the stereo 
pair of images. Therefore a parallelogram is defined using epipolar lines through the start and end points of the line 
segment, and using the width determined by the maximum disparity (Medioni and Nevatia, 1985). The line segments 
that occur in this search space are used in the process of determining the average correlation coefficients. The 
correlation window size was set to 15 pixels square, as suggested by Schmid and Zisserman (1997). The segment with 
the highest correlation coefficient is taken to be the matching segment. The length of the segments being matched was 
kept to over ten pixels, as shorter lines have greater uncertainty. Other constraints are a slope constraint, which was 
used only lightly as it had to take into account changes in viewing angle of the edges, and a uniqueness constraint, such 
that a segment in the second image could only be matched to one segment in the first image. 
3.2.3 Line Segment Reconstruction. The 3D coordinates of the line segments are calculated using the interior and 
exterior orientation parameters of the aerial imagery. The end points of the overlapping section of the line segment are 
required in image coordinates from each of the stereo pair. As the line segment information is in pixel coordinates, an 
affine transformation is required to convert the information appropriately. The interior orientation of the imagery 
defines this affine transformation, and the rotation matrix and camera position are defined in the exterior orientation. 
The 3D coordinates of each endpoint of the line segment are calculated using this information. (See Kraus (1993) for 
equations.) The 3D coordinates are checked for gross errors using the range of elevations of the laser data and 
comparing these with the elevations of the reconstructed points to see that they are within a suitable range of these 
bounds. 
3.2.4 Filtering Laser Data. For an accurate description of the surface discontinuities in an urban area, it is important 
to detect the roofline of buildings and also the ground surface near that roofline. The laser data is used as an indication 
as to whether the automatically detected breaklines are rooflines or ground surface breaklines. The area surrounding 
each roofline is investigated to determine if there is a corresponding ground breakline. If only the roofline is detected, 
the surface discontinuity will not be accurately delineated. In this case, a breakline is added to define the ground 
surface. 
The laser data is classified as either ground points, vegetation or building points. The laser points nearest the breakline 
are used as an indication as to whether it is associated with a building. As this research is for urban areas, certain 
assumptions are made which allow a simplified approach to the classification of the laser data. It is assumed that the 
terrain is generally flat and that the vegetation will cover smaller areas than the buildings. This approach is adequate for 
the present research, however a more sophisticated classification approach will be implemented in the future, such as 
those described by Axelsson (1999) and Schenk et al. (1999). 
The initial filtering of the laser data determines the ground points from other higher points. In this research, the area 
being investigated will either be over flat terrain or the area will be small enough to be able to discount affects of a 
  
566 International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B3. Amsterdam 2000. 
gra 
ort 
The 
whi 
buil 
freq 
the 
use 
witl 
to 1 
has 
buil 
sho 
“USE 
reso 
veg 
isol: 
larg 
3.2. 
the 
char 
brea 
The 
desc 
outv 
hori 
adde 
sear 
3.2. 
proc 
utili: 
infoi 
phot 
The 
imag 
dune 
the a 
The 
detei 
repre 
breal 
4.1 
The 
analy 
prov 
betw