n, special
icting the
yes classi-
ability is
Its in the
(6)
"ule. This
> learning
ximating
xpansions
but fixed
jents of c
(7)
(8)
> is equiv-
natrix A.
arbitrary
accuracy
nial x(c).
izing the
d decision
on
(9)
sulting in
5)" ). (10)
to be in-
ous prob-
therefore
their cor-
ning sam-
le results
in off-line
ess, there
both ap-
orvised or
se, a suf-
incoming
1e present
he param-
the result
ndomized
| values of
inted out,
respect to
domain is
reflected by the training sample and the perfect de-
cision rule. An implicit distributed representation
of the objects is used.
2.2 Artificial Neural Networks
While most classical statistical pattern recognition
systems follow the sequential architecture outlined
in Fig.1, the development of architectures based on
simple units is one of the main goals of neural net-
works research activities. As in biological systems
information about objects or classes is represented
in a distributed fashion. The basic processing of a
system is performed by units which adopt models
of neurons. Although such artificial neurons do not
provide calculations with high precision their mu-
tual interaction results in systems of globally high
performance. À network of neurons establishes an
ensemble of nonlinear joint processes. Architecture
deals with the arrangement of neural units and their
synchronization in the network. As a consequence,
à system is viewed as a strong interrelationship be-
tween structure and functionality. Subnets form
specialized modules. Because of the simple basic
units, an artificial neural network has a high connec-
tivity and it provides a massive parallel computing
environment.
Models for neurons are based on the principle of
synaptic summation. The following processing steps
are carried out: The input vector x is manipulated
with respect to a weight vector w giving a scalar
value s(x, w). Then a bias term is subtracted. The
third step provides a nonlinear mapping which may
be enriched by stochastic processes. Hence, the neu-
ral model can be expressed as a function y(x) de-
fined by
y(x) = f(s(x,w) — 6) (11)
Frequently used combinations for s are the Euk-
lidian distance or the scalar product of x and w,
whereas the so called activation function f is real-
ized by the sign-function, tanh, Fermi, or Gaussian.
Its argument characterizes the state of the neuron.
Three major values are distinguished for this state z.
If z > 0 the neuron is called active, for z = 0 quiet,
and for z « 0 obstructing. Based on such kinds of
units an artificial neural network is constructed as a
directed graph defined by a set of states Z for each
node and a set of states W providing the weights
associated with the links between nodes, i.e. neu-
rons, and a set of input and output variables. The
state set of a network covering k nodes and k? links
is represented by
X-2Z^5xw", (12)
where one state c — (z, W) € X. The activities are
denoted by a vector z € Z^ and the weights in a
2 : 2 Ma
matri W € W* . According to this connectivity
713
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B3. Vienna 1996
matrix W = [w;;], the basic types of architectures
can be described
e complete connected network: wj; # OVi, j
e isolated neurons: W diagonal matrix
e weak connected network: w;; # 0 for only a
few pairs 7, j
forward connectivity: w;; = 0ifi«j
small range connectivity: W band matrix
The processing behavior of neural networks is char-
acterized by state transitions, i.e. from a state ot
at time t a new state o**! at time t 4- 1 is achieved.
Like the states, also the transitions are divided into
two components. Changes of activities express the
short term dynamics of the network. For a certain
neuron ¢ it can be described by
a1 — fis(z, w) - 6i) (13)
Long term dynamics refer to changes of weights ac-
cording to
t+1 t t .«
wil ) = wt? + gi (wEP, x La) (14)
Both types of dynamics are of local character and
distribute domain knowledge several units. The pa-
rameters, i.e. the weights and thresholds shall be
learned automatically based on a training sample.
Therefore, two a priori decisions are necessary at
the present state of the art. First of all a unique
type of model neurons has to be selected. Although
a few examples of learning the topology for a net-
work exists, in most cases the number of units and
their connectivity is also fixed a priori. The train-
ing phase therefore adjusts the parameters in a sense
comparable to statistical classifiers.
A large number of neural network architectures is
covered by the description above. But it should be
mentioned, that further types have been developed
and are in successful use. As a few examples there
are Hebb networks, Kohonen maps, and associative
memories. Another type, the so called local linear
maps will be described in the context of hybrid sys-
tems.
2.3 Knowledge Based Interpretation
Knowledge based techniques are influencing com-
puter vision and object recognition approaches for
nearly two decades. The goal is to generate indi-
vidual symbolic descriptions of domain entities, i.e.
objects. Contrarily to classical AI approaches which
deal with symbol to symbol transformations seman-
tic models for object recognition are concerned with
the transformation of numerical data into symbolic
descriptions. It is not possible to achieve the overall
goal by optimizing one decision function or by ad-
justing weights to a given problem, although both
techniques presented so far are of great importance
