In summary, the dissimilarities between the LIDAR and 
stereo image data reconstructed surfaces slightly resemble 
the contrast between feature-based and area-based 
matching techniques (Toth and Schenk 1992). LIDAR, 
similarly to feature-based matching, delivers very robust 
data, but the localization of the points is somewhat modest 
(as is the case with the conjugate primitives). On the other 
hand, area-based matchers deliver excellent localization, 
provided that good approximations are available. This 
naturally leads to changing the hierarchical warped image- 
based surface reconstruction technique by replacing the 
feature-based global matching with the LIDAR 
observations. In essence, the whole problem is reduced to 
a local matching task whose main objectives are to refine 
and densify the elevation spots’ coordinates. 
We believe that the following related issues should be part 
of any future research on extracting surfaces from 
combined LIDAR and stereo image data: 
Comparing DEMs. Relating DEMs is rather difficult; 
although methods are available, this area still needs 
additional work. Simple methods which analyze 
vertical differences by resampling one DEM into the 
other’s frame are not optimal since they normally do 
not consider the actual slope at the elevation spots or 
the positioning accuracy of the elevation points. For 
ongoing research in this area, see (Schenk 1999). 
Surface representation and modeling are themselves an 
evolving topic; for example, one hard task is 
converting surface discontinuities (Al-Tahir 1998). 
LIDAR boresighting. The LIDAR data provide range 
measurements relative to the data acquisition platform 
position and attitude. However, there is no image 
information on the footprint of the range observations, 
so it is rather difficult to relate the measurements to 
the object space. Nonetheless, such a connection is 
necessary for determining the boresight misalignment 
of the LIDAR with respect to the GPS/INS positioning 
system. The most typical way to accomplish this task 
is to collect LIDAR data over a test site with good 
ground truth data and then to iteratively adjust the 
attitude parameters to find the best fit between the 
LIDAR measurements and the reference surface data. 
For efficient production, the automation of this process 
is necessary. 
Overlapping LIDAR. Using overlapping LIDAR data 
can be very beneficial for many reasons. For one, it 
can help to cope with the strong signal dependence of 
the surface slant. It is likely that for parallel flight 
lines, the surface slopes parallel to the flight direction 
will have a better chance to receive the laser signal 
beam corresponding to the surface normal direction. 
The availability of two almost independent sets of 
LIDAR ranges provides additional support for quality 
control and calibration, such as boresighting. Figure 
10 depicts LIDAR elevation spots from overlapping 
flight lines. 
Figure 10. The distribution of the LIDAR elevation spots 
from different flight lines. 
4. CONCLUSION 
We have reported about our early experiences on 
combining LIDAR data with direct digital frame imagery 
for surface extraction. A test flight was organized to 
simultaneously collect LIDAR data .with high-resolution 
direct digital frame imagery. For both sensors, an 
integrated GPS/INS system provided the georeferencing. 
Special arrangements were made to get multiple coverage 
for both imaging sensors over a well-surveyed test-site. 
Basic trends and characteristic features were discussed 
and illustrated. 
Despite the notable success of the past two decades in the 
area of reconstructing three-dimensional surfaces from 
two-dimensional images, a significant performance 
increase can be expected from combining such data with 
LIDAR range observations. Although the most complete 
surface reconstruction would involve the modeling of 
natural and man-made objects, the introduction of LIDAR 
adds a robustness to supplement existing stereo image- 
based extraction techniques by providing strong geometric 
constraints to guide the image matching process. 
Additionally, LIDAR can separate vegetation canopy from 
topographic surfaces, a feature not available from stereo 
image-based techniques. We believe that the fusion of 
LIDAR with frame imagery is the intermediate step to the 
modeling of objects, which will ultimately lead to 
combined surface reconstruction and object extraction 
techniques. 
Presently we continue with algorithmic research and 
testing. Special areas besides surface reconstruction are 
the efficient integration of multiple LIDAR data sets and 
the system calibration aspects of LIDAR boresighting.