3.1 A Semantic Network Language
In contrast to other approaches like KL-ONE or
PSN, in the ERNEST semantic network language
only three different types of nodes and three dif-
ferent types of links exist. They have well defined
semantics and we believe that these structures are
adequate to represent the knowledge for different
pattern understanding tasks. The node type Con-
cept represents classes of objects, events, or abstract
conceptions having some common properties. In the
context of image understanding an important step
is the interpretation of the sensor signal in terms
modeled in the knowledge base. The second node
type, called instance, represents these extensions of
a concept. It associates certain areas of the image
with concepts of the knowledge base. It is a copy
of the related concept where common property de-
scriptions of a class are substituted by values de-
rived from the signal. In an intermediate state of
processing instances of some concepts may not be
computable because certain prerequisites are miss-
ing. Nevertheless, the available information can be
used to constrain an uninstantiated concept. This
is done via the node type modified concept which
represents modifications of a concept due to inter-
mediate results of the analysis.
As in all approaches to semantic networks the part
link decomposes a concept into its natural compo-
nents (i.e. CAR par TYRE): However, in image un-
derstanding it often occurs that a certain concept
is only defined in the context of another one. For
example, if you want to find a spare tyre in an im-
age it only can be identified as a spare tyre in the
context of a related vehicle. Contrarily, an ordinary
tyre can be recognized without any context as the
definition of that term is independent of relation-
ships to other ones. However, the term front tyre is
context-dependent as this property can be only de-
termined by an appropriate context. To model this,
fact à part can be marked as context-dependent and
vice versa a context can be explicitly inserted in a
concept. That means, SPARE.TYRE is for instance
d
a context-dependent part C24 of JEEP and in
SPARE_TYRE the concept JEEP is inserted as a pos-
sible context. Another well-known link type is the
specialization which connects a concept with a more
general concept (i.e. CAR TU SJEEP). Closely re-
lated to that type of link is an inheritance mecha-
nism by which a special concept inherits all prop-
erties of the general one, unless they are explicitly
modified. In order to motivate the third link type,
the description of aggregation in [14] is reported:
*for example, the parts of John Smith, viewed as
a physical object, are his head, arm, etc. When
viewing as a social object, they are its address, so-
cial insurance number, etc.” Two conceptual sys-
tems are distinguished in this example. A concept
716
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B3. Vienna 1996
modeling a person has different parts within each of
these systems. Parts in the social system are social
conceptions, parts in the physical system are physi-
cal conceptions. In complex applications, more than
one such conceptual system will occur, i.e., in image
understanding, lines, geometry, named objects, or
motions. Relationships between concepts belonging
to different conceptual systems are only established
by the link type concrete. Therefore, part and spe-
cialization are restricted in the way that they are
only allowed inside the same conceptual system. For
example, the concepts TYRE and CIRCLE represent
terms of different conceptual systems because bar
belongs to “named object”, while rectangle belongs
to “geometry”. According to the fact that circle is
more concrete to the signal than tyre, the following
link TYRE 3 CIRCLE is established.
In addition to its links, a concept is described by
attributes representing qualitative or numerical fea-
tures and restrictions on these values according to
the modeled term. Furthermore, relations defining
constraints for the attributes can be specified and
must be satisfied for valid instances.
The creation of modified concepts and instances con-
stitutes the knowledge utilization in the semantic
network. For the creation of instances, this process
is based on the fact that the recognition of a com-
plex object needs the detection of all its parts as a
prerequisite. For concepts which model terms only
defined within a certain context the instantiation
process must proceed in the opposite direction. In
this case the context must exist before an instance
of the context-dependent concept can be created. In
the network language, these ideas are expressed by
six problem-independent inference rules. Context-
independent parts, contexts, and concretes are the
prerequisites for the creation of instances and modi-
fied concepts in a data-driven strategy. The opposite
link directions are used for model-driven inferences.
Since the results of an initial segmentation are not
perfect, the definition of a concept is completed by
a judgment function estimating the degree of corre-
spondence of an image area to the term defined by
the related concept. On the basis of these estimates
and the inference rules an A*-like control algorithm
is applied. For a detailed description of the network
language and the control algorithm see [17, 9].
3.2 A Hybrid Approach
To overcome the respective disadvantages of knowl-
edge based and neural techniques we propose a hy-
brid system combining neural and semantic net-
works. The main idea is to associate or attach
ANNS as holistic models to concepts of the semantic
network, with both components modeling the same
object?. That is, the interface between the different
?'The same applies to other concepts modeled in the semantic
network, like events or abstract conceptions. For sake of simplic-
ne
seg
ap
the
ab)
str
ass
rec
ces
by
tac
ust
In
cus
WO
of 1
att
tic:
abc
for
siti
bec
dat
in
niz
ins
In
niz
gos
ma
pet
ane
con
the
