CA, 9-11 Nov. 1999 
ig imagery to provide edge 
ited by several researchers 
98; Csathó et al., 1999; Haala 
nain concept of the approach 
paper utilizes the beneficial 
c data and laser data to produce 
imagery can provide accurate 
ig the location of surface 
provides accurate elevations, 
te location information of the 
ted in Figure 1. Thus the two 
n the advantages of each, and 
1 of an accurate surface model. 
has been undertaken using an 
faryland. Laser data and aerial 
> day by NASA and NGS 
inary experiments have been 
gorithm. This paper presents 
ia fusion components, describes 
e results from the initial 
ATION APPROACH 
aper utilizes information from 
ammetric data to produce an 
face. The conceptual approach 
st phase, surfaces created from 
stered to the same coordinate 
s of extracting edge information 
hich is used to delineate surface 
ene. By merging the edge 
surface generation, a DSM that 
scene can be produced. 
ndertaken to determine the 
en the laser surface and the 
ice. Theoretically, the two data 
ordinate system, however the 
e laser data may introduce à 
o surfaces, which must be 
may be performed accurately 
nisalignment has been observed 
tial testing of the algorithm, 
surface registration component 
to find the transformation 
These parameters are used to 
he coordinate system of the 
data have higher planimetric 
data (Baltsavias, 1999). The 
neters must be as accurate as 
ary degradation occurs in the 
from merging the data sets. 
    
International Archives of Photogrammetry and Remote Sensing, Vol. 32, Part 3W14, La Jolla, CA, 9-11 Nov. 1999 
The algorithm was developed specifically for matching visible 
surface models of urban areas produced using different 
acquisition methods, and in particular, surfaces from airborne 
laser scanner data and surfaces automatically generated using 
photogrammetric data. The registration algorithm solves for three 
translation parameters, three rotation parameters and one scale 
factor. The surface points are not interpolated to a regular grid 
and there are no identifiable conjugate points in the surfaces. For 
more detailed information on the algorithm and the results of 
testing, the reader is referred to Postolov et al. (1999). 
2.2 Data Fusion 
The main emphasis of the research project is to develop an 
algorithm to produce accurate visible surface models of urban 
areas, utilizing both laser data and photogrammetric data. 
The data fusion component utilizes edges extracted from the 
stereo imagery to obtain accurate horizontal locations of surface 
discontinuities. The edges are defined in three dimensions and 
are used as breaklines when merged with the transformed laser 
data. A new surface is generated using the merged data, which is 
expected to have a higher accuracy than the surface derived from 
either of the separate data sets. 
The conceptual approach formulated in this research proposes 
that the edges representing surface discontinuities are 
automatically derived as 3D line segments from the stereo 
imagery using edge extraction and feature matching techniques. 
Initial research has been undertaken into the automatic extraction 
of 3D line segments. 
The edge pixels are detected in each image of the stereo pair of 
digital images. At present, the optimal zero-crossing operator 
(Sarkar and Boyer, 1991) is used for the detection of the edge 
pixels. The edge pixels are analyzed to find connected lines and 
using these lines, straight line segments are determined. Line 
segments are used as they adequately describe man-made objects 
and can be more accurately located than point features when 
using feature matching techniques (Fradkin and Ethrog, 1997). 
Also, each line segment can be easily described using the start 
and end points of the segment and the gradient of the line. 
For each straight line segment in the first image, the second 
image is searched for a matching segment. To make the 
searching of the images more computationally efficient, a coarse 
to fine approach is utilized, and constraints are applied to the 
search process, including epipolar and ordering constraints 
(Grewe and Kak, 1994). The stereo imagery has exterior 
orientation parameters which allow the 3D coordinates of the line 
segments to be determined when the matching of the line 
segments is completed. 
The automatic extraction of edges has certain limitations which 
must be addressed. Using automatic extraction methods, not all 
surface discontinuities are detected due to factors such as 
foreshortening and lack of contrast in the imagery. There might 
be a number of incomplete or incorrect breaklines, and other 
breaklines may be detected that do not represent surface 
discontinuities may be detected. Instead, these breaklines may 
refer to visual edges in the images, which are purely changes in 
grey value, such as line markings on roads. The effect of 
including these breaklines in the data fusion process is to be 
investigated. 
For this research, the most important edges to be detected are the 
discontinuities between the roof and the ground. These areas are 
important to the accurate representation of the visible surface, as 
they contain dramatic changes in elevation. 
The detected breaklines are expected to be mainly rooflines, 
however to properly define the surface, the ground level near the 
rooflines must also be determined. Therefore, the roofline will be 
projected vertically onto the ground surface to produce a new 
breakline, thus describing the surface at ground level as well as 
roof height. The location of the new breakline must be slightly 
offset outward from the roofline, as the triangulation process 
being used does not allow points to exist with the same horizontal 
location, as would be the case for a vertical plane. 
The elevation of the ground surface at the location of the foot of 
the building must be determined. It is proposed that the laser 
data in the areas surrounding the breakline be searched. In the 
general case of the breakline delineating a surface discontinuity 
which is a roofline, it is expected that one side of the breakline 
will have points of higher elevation than the other side. The 
lower elevations will be assumed to be the elevation at ground 
level. Therefore, the new breakline will be generated and 
assigned the elevation of the ground points in that area. 
In the current testing of the data fusion algorithm, manually 
measured breaklines are used. These breaklines are adequate to 
test the validity of the algorithm and to assess the implementation 
of the approach. 
The laser points which occur on the breaklines are eliminated, 
again due to the triangulation process not accepting points with 
identical horizontal locations. The triangulation process is used 
to generate a TIN which utilizes the laser data points and 
constrains the triangles to follow the breaklines. The inclusion of 
the elevation information to the triangulation provides the digital 
surface model produced using the laser data and the 
photogrammetric data. 
4 PRELIMINARY TESTING AND RESULTS 
The initial testing was carried out using a data set over Ocean 
City, MD, which includes digital stereo imagery and laser data. 
The data set covers different types of areas, including residential 
areas, flat terrain, beach front, dunes, canals and high-rise 
buildings. Only a small portion of a residential area has been 
used for the initial testing of the algorithm. 
The laser data have been filtered so as to only retain the first laser 
measurement and not any subsequent measurements from the 
laser sensor. The laser data has also been transformed to the 
coordinate system of the photogrammetric data using the 
parameters determined by the process surface registration. The 
two data sets are therefore on the same coordinate system and are 
representing the same surface. Laser points in the new data set 
are eliminated if they occur within close proximity to the 
breaklines. A filtering process is undertaken to eliminate points 
within a horizontal buffer zone of the line segments representing