that is not
s. However,
. possible to
) the unified
output links)
)ses the set
re available:
cussion, we
ff procedure
A
lusive inter-
1sform data
t transforms
evel of data
information
ately during
|verse case
ss. We shall
Procedures
ile updating
dures are:
> feature for
; there must
obtain the
Issigns the
ement using
d type; may
analysis and
bolic (class)
1e matching
vel; may be
1e Bayesian
nodel-based
a set of such
is given;
assigns the
ach element
zy) or hard
Procedures
types while
The most
ler data into
ructure; may
sed, texture-
Jes;
e scene restoration - forms a scene description,
based on segmented structure; may be model-
based or use some data base; extracts the
spatial objects;
e object tracking - in the most broad sense,
presumes the estimation of the element
movement; requires to store the results of
previous observations; may use the sequential
image differences, correlation tracking, Kalman
filtering, Bayesian estimation and so on;
extracts the spatio-temporal events.
?. Pairwise fusi
Procedures of this class fuse two input
structures of some type into the one output structure
of the same type. There are two types of fusion:
element fusion and structure fusion.
A. Element fusion procedures. The most
important procedures are:
e element theoretic-set union - fuses two
structures — with corresponding element
dimension of D1 and D2 into the resultant
structure with element dimension (D1+D2);
preserves the structure dimension; typically of
use in fusion at the measurement and feature
levels; results in creation of common space of
measurements or features;
e Soft data fusion - for each soft element fuses
the soft characteristic vectors; preserves the
both structure dimension and element
dimension; may use Bayesian or Dempster-
Schaffer reasoning;
e evidence fusion - fuses evidences to obtain the
hard symbolic data; preserves the both
structure dimension and element dimension;
may use any evidence reasoning approach;
B. Structure fusion procedures. The most
important procedures are:
e Structure union - theoretic-set union of two
structures; preserves the element dimension; of
use in fusion of any structured data;
oe structure intersection - theoretic-set intersection
of two structures; preserves the element
dimension; of use in fusion of any structured
data;
e rang fusion - theoretic-set union of two
structures with accumulation of number of
evidences for each element and exclusion of
elements with this number less than some rang
(threshold); preserves the element dimension:
of use in fusion of any structured data:
These procedures must be implemented for
each of data structures supported by the proper
System of multi-sensor data processing using the
Structure union, intersection or rang fusion.
387
2.2.3. Filteri |
These are the on-level procedures with one
input and one output. These procedures always
preserve the element dimension of data. So, we
need to consider the structure dimension as a
characteristic feature of these procedures.
A. Dimension preserving procedures. The most
important procedures of this type are:
e element transforms - transform the each
element in the uniform way; typically of use at
the measurement and feature levels, e.g.
histogram transforms of intensity images or the
mappings of feature spaces;
e linear and non-linear transforms - processes
raster data in the spatial or frequent domain; the
large set of such procedures can be
implemented.
B. Dimension non-preserving procedures. The
most important procedures of this type are:
e geometric transforms - transform any spatial
data using some geometric model; the large
set of such procedures can be implemented;
may be of use to provide the co-registration
conditions;
e structure evolving - processes the structure to
obtain to better features or behavior, for
example, 2D-segmentation by repetitive
splitting and merging.
Thus, we have formed the complete
theoretical framework for multi-sensor data
processing. The necessity of implementation of
some proper kind of filtering procedure is
determined by the properties and required features
of any certain multi-sensor application to be
developed on the basis of this framework.
This theoretical basis allows to form generic
frame-based software architecture for multi-sensor
data fusion and processing of multisensor image
sequences.
3. THE CAPTURING OF MULTISENSOR IMAGE
SEQUENCES
In order to form the satisfactory and
representative data base of image sequences the
special experimental complex have been designed
(fig.3). It includes:
coherent LL-sensor;
IR-sensor AGA-780 with the control system and
monitor;
e high resolution video camera FTM-800;
DISCON - the transmission device to transform
the IR-signal to video standard;
e PC to store the IR and video images;
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B5. Vienna 1996
