Information Fusion in
Cartographic Feature Extraction
from Aerial Imagery
David M. McKeown
Frederic P. Perlant
Jefferey Shufelt
Digital Mapping Laboratory
School of Computer Science
Carnegie Mellon University
Pittsburgh, PA 15213
Abstract
The extraction of buildings from aerial imagery is a
complex problem for automated computer vision. It requires
locating regions in a scene that possess properties
distinguishing them as man-made objects and opposed to
naturally occurring terrain features. The building extraction
process requires techniques that exploit knowledge about the
structure of man-made objects. Techniques do exist that take
advantage of this knowledge; various methods use edge-line
analysis, shadow analysis, and stereo imagery analysis to
produce building hypotheses. It is reasonable, however, to
assume that no single detection method will correctly
delineate or verify buildings in every scene. As an example,
imagine a feature extraction system that relied on analysis of
cast shadows to predict building locations in cases where the
sun was directly above the scene.
It seems clear that a cooperative-methods paradigm is
useful in approaching the building extraction problem. Using
this paradigm, each extraction technique provides
information which can then be added or assimilated into an
overall interpretation of the scene. Thus, our research focus
is to explore the development of a computer vision system
that integrates the results of various scene analysis
techniques into an accurate and robust interpretation of the
underlying three-dimensional scene.
This paper briefly describes research results in two areas.
First, we describe the problem of building hypothesis fusion
using only monocular cues in aerial imagery. Several
building extraction techniques are briefly surveyed, including
four building extraction, verification, and clustering systems
that form the basis for the work described here. A method
for fusing the symbolic data generated by these systems is
described, and applied to monocular image and stereo image
data sets. Evaluation methods for the fusion results are
described, and the fusion results are analyzed using these
methods.
The second research area examines how estimates of three-
dimensional scene structure, as encoded in a scene disparity
map, can be improved by the analysis of the original
monocular imagery. In some sense this procedure is counter
intuitive. Since we have already used the imagery to perform
stereo matching, what information could be available in
either of the single images that would improve on the stereo
analysis? We describe the utilization of surface illumination
information provided by the segmentation of the monocular
image into fine surface patches of nearly homogeneous
intensity to remove mismatches generated during stereo
matching. Such patches are used to guide a statistical
analysis of the disparity map based on the assumption that
such patches correspond closely with physical surfaces in the
scene. This technique is quite independent of whether the
initial disparity map was generated by automated area-based
or feature-based stereo matching.
1 Introduction
The extraction of significant man-made structures such as
buildings and roads from aerial imagery is a complex
problem that must be addressed in order to produce a fully
automated cartographic feature extraction system. We focus
on the building extraction process since buildings are present
in almost all sites of cartographic interest and their robust
detection and delineation requires techniques that exploit
knowledge about man-made structures. There exist a
multitude of techniques that take advantage of such
knowledge; various methods use edge-line analysis, shadow
analysis, stereo disparity analysis, and structural analysis to
generate building hypotheses 1 ’ 2> 3 > 4,5< 6> 7 > 8 .
It is reasonable, however, to assume that no single building
extraction technique will perfectly delineate man-made
structures in every scene. Consider the use of an edge-
analysis method on an image where the ground intensity is
similar to the intensity of the roofs of the buildings in the
scene. As another example, consider the use of a shadow
analysis method on an image in which the sun was directly
above the scene.
Clearly, a cooperative-methods paradigm is useful in
approaching the building extraction problem. In this
paradigm, it is assumed that no single method can provide a
completely accurate or complete set of building hypotheses
for a scene; each method can, however, provide a subset of
the information necessary to produce an improved
interpretation of building structure in the scene. For instance,
a shadow-based method can provide useful information in
situations where ground and roof intensity are similar; an
edge-line analysis method can provide disambiguating
information in cases where shadows were weak or
nonexistent, or in situations where structures were
sufficiently short that disparity analysis would not provide
useful information. The implicit assumption of this paradigm
is that the information produced by each detection technique
can be integrated into a more meaningful collection of
building hypotheses.
Stereo matching provides a direct measurement of building
location and height. In complex urban scenes, stereo
matching based upon feature matching, i.e., edges, lines, and
contours, appears to provide more accurate and robust
matching than area-based techniques. This is primarily due
to the ability of feature-based approaches to detect large
depth discontinuities found in urban scenes. However,
feature-based techniques generally provide only a sparse set
of match points from which a three-dimensional surface is
usually interpolated. In Section 3 we describe a method to
integrate monocular surface intensity information with the
stereo disparity map to refine the height estimates and reduce
the effect of stereo matching errors.
140
