International Archives of Photogrammetry and Remote Sensing, Vol. 32, Part 7-4-3 W6, Valladolid, Spain, 3-4 June, 1999
53
INCLUSION OF MULTISPECTRAL DATA INTO OBJECT RECOGNITION
1 2
Bea Csathó , Toni Schenk, Dong-Cheon Lee and Sagi Filin
1 Byrd Polar Research Center, OSU, 1090 Carmack Rd., Columbus, OH 43210, email: csatho.l@osu.edu, phone: 1-614-292-6641
2 Department of Civil Engineering, OSU, 2070 Neil Ave., Columbus, OH 43210 email: schenk.2@osu.edu, phone: 1-614-292-7126
KEYWORDS: Data fusion, multisensor, classification, urban mapping, surface reconstruction.
ABSTRACT
In this paper, we describe how object recognition benefits from exploiting multispectral and multisensor datasets. After a brief
introduction we summarize the most important principles of object recognition and multisensor fusion. This serves as the basis for
the proposed architecture of a multisensor object recognition system. It is characterized by multistage fusion, where the different
sensory input data are processed individually and only merged at appropriate levels. The remaining sections describe the major
fusion processes. Rather than providing detailed descriptions, a few examples, obtained from the Ocean City test-data site, have been
chosen to illustrate the processing of the major data streams. The test site comprises of multispectral and aerial imagery, and laser
scanning data.
1. INTRODUCTION
The ultimate goal of digital photogrammetry is the automation
of map making. This entails understanding aerial imagery and
recognizing objects - both hard problems. Despite of the
increased research activities and the remarkable progress that
has been achieved, systems are still far from being operational
and the far-reaching goal of an automatic map making system
remains a dream.
Before an object, e.g. a building, can be measured, it must
first be identified as such. Fully automated systems have been
developed for recognizing certain objects, such as buildings
and roads on monocular aerial imageries, but their
performance largely depends on the complexity of the scene
and other factors (Shufelt, 1999). However, the utilization of
multiple sensory input data, or other ancillary data, such as
DEMs or GIS layers, opens new avenues to approach the
problem. By combining sensors that use different physical
principles and record different properties of the object space,
complementary and redundant information becomes available.
If merged properly, multisensor data may lead to a more
stable and consistent scene description. Active research topics
in object recognition include multi-image techniques using
3D feature extraction, DEM analysis or range images from
laser scanning, map- or GIS-based extraction, color or
multispectral analysis, and/or a combination of these
techniques.
Now the cardinal question is how to exploit the potential
these different data sources offer to tackle object recognition
more effectively. Ideally, proven concepts and methods in
remote sensing, digital photogrammetry and computer vision
should be combined in a synergistic fashion. The combination
may be possible through the use of multisensor data fusion, or
distributed sensing. Data fusion is concerned with the
problem of how to combine data from multiple sensors to
perform inferences that may not be possible from a single
sensor alone (Hall, 1992). In this paper, we propose a unified
framework for object recognition and multisensor data fusion.
We start out with a brief description of the object recognition
paradigm, followed by the discussion of different
architectures for data fusion. We then propose a multisensor
object recognition system. The remaining sections describe
the major fusion processes. Rather than providing detailed
descriptions, a few examples, obtained from the Ocean City
test-data site, have been chosen to illustrate the processing of
the major data streams. Csatho and Schenk (1998) reported
on earlier tests using the same dataset. The paper ends with
conclusions and an outline of future research.
2. BACKGROUND
2.1. Object recognition paradigm
At the heart of the paradigm is the recognition that it is
impossible to bridge the gap between sensory input data and
the desired output. Consider a gray level image as input and a
GIS as the result of object recognition. The computer does not
see an object, e.g., a building. All it has available at the outset
is an array of numbers. On the output side, however, we have
an abstract description of the object, for example, the
coordinates of its boundary. There is no direct mapping
between the two sets of numbers.
A commonly used paradigm begins with preprocessing the
raw sensory input data, followed by feature extraction and
segmentation. Features and regions are perceptually organized
until an object, or parts of an object, emerge from the data.
This data model is then compared with a model of the
physical object. If there is sufficient agreement, the data
model is labeled accordingly. In a first step, the sensor data
usually require some pre-processing. For example, images
may be radiometrically adjusted, oriented and perhaps
normalized. Similarly, raw laser altimeter data are processed
to 3-D points in object space.
The motivation for feature extraction is to capture information
from the processed sensory data that is somehow related to
the objects to be recognized. Edges are a typical example.
