FRAMES BASED ANALYSIS OF MULTISENSOR IMAGE SEQUENCES 
Yury V.Morzeev, Yury V.Visilter, Sergei Y. Zheltov, Alexander A. Stepanov 
(State Research Institute of Aviation Systems, Vicktorenko 7, Moscow, Russia) 
e-mail: yury@fenix.niias.msk.su 
KEYWORDS: multisensory data processing, object-oriented programming, frame net, data fusion, image 
sequences. 
ABSTRACT 
At present day we can see the increasing of interest to the problem of combined analysis of the multisensor 
data. One of the prospect way to solve the fusion problem for this heterogeneous information for the 
improvement of the informational-perceiving characteristics of the designed system is the developing of the 
high-accuracy multisensor data fusion algorithms. This paper dedicated to generalize the results, which we 
achieved in the field of multisensor data fusion analysis. 
1. INTRODUCTION 
The appearance of the new 2D-sensors 
generation (e.g., video, IR, LL, etc. with a lot of 
modifications) increase the interest to the problem 
of combined analysis of the multisensor data [1]. 
The developing of this new real-time measuring 
systems and registration of the 2D-images methods 
makes it possible to process the time-invariant 
images, which have different physical nature. 
The enhancement of informational 
perceiving characteristics of the designed sensor 
systems can be achieved by solving of multisensor 
data fusion problem. Hereupon, the final system 
could be designed to solve one of two principle 
problems: 
e the complex support to help the operator in 
decision making problem in the case of the half- 
automatic targets selection systems; 
e the automatic decision making in the automatic 
targets selection systems. 
Due to these problems, we would like to 
generalize the results, which we achieved in the 
field of multisensor data fusion analysis. In this 
paper the new object-oriented approach to 
multisensor data processing will be discussed. This 
approach presumes any processing scheme to be 
represented as a network of soffware frames [2]. 
The interaction between objects in such network is 
provided by means of the message transmission 
between frames in accordance with some logic 
rules. The designed technology provides the 
automatic analysis of multisensor image sequences. 
384 
2. BASIC PRINCIPLES FOR MULTISENSOR 
IMAGE SEQUENCES PROCESSING 
In this section we want to discuss the 
generic Multi-Sensor Data Processing Framework 
(MSDPF) that, being implemented using our 
software frames, can support the most wide 
spectrum of possible processing schemes [3]. The 
most strong limitation of our framework is a 
modularity assumption. This term, in our 
comprehension, means that the mapping process of 
input data to output data is structured into the set of 
sequential or parallel procedures with input/output 
dependence. Some buffer data structures that are 
the output of some procedures and simultaneously 
the input of some other procedures express this 
dependence. It is easy to see that only the 
structured processing schemes can have the 
adequate frame-based representation. It is obvious 
that the set of control frames is not problem- 
dependent. Thus, we have to specify the set of data 
frames and the set of processing frames. Due to 
data-driven character of our approach, the set of 
data frames must be firstly defined and the set of 
processing frames will be defined later, on the basis 
of known data frame types. For simplicity, we shall 
describe our MSDPF in terms of data types and 
procedure types. 
2.1. The levels of data abstraction 
It is well known nowadays, that there is the 
customary conception of data processing levels in 
the field of data fusion problem. It usually 
corresponds to the sequential modular scheme (see 
fig.1) [1]. 
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B5. Vienna 1996 
  
Fig. 1. 
TI 
direction 
measure 
descriptio 
Last year: 
a union O 
fuzzy) de 
level cor 
about the 
supposes 
separate 
sensor al 
highest le 
being dor 
is execute 
1 
and on-le 
transform 
data of | 
realize tt 
attractive 
feature | 
level is u 
provide tl 
image da 
sensing cC 
enough. € 
considera 
V 
introduce 
Specify 1 
procedure 
types we 
The first 
any data 
structure 
of view, c 
of data 
structures 
may have