on Process 
h such sys- 
ictional or- 
' decompo- 
functional 
erpretation 
‚ed control 
ating suit- 
respect to 
description | 
  
edge Based 
knowledge 
acquisition, 
e the archi- 
sses the hi- 
this model. 
vledge and 
ich can be 
, a central- 
mentioned 
ies but not 
flow. Data 
can be im- 
ot reflected 
that a cer- 
yus uniform 
arated into 
ly, inquiries 
at different 
ed. Except 
edge, types 
raints must 
pe provides 
sults. Each 
  
       
   
  
  
declarative inferences (intermediate-)results 
knowledge 
8 (procedural knowledge) | 
concepts data-driven: instantiation | instances E 
  
(terms, objects, 
events) 
model-driven: propagation of 
modified concepts 
  
  
  
  
constraints | 
concepts | 
; ; ; nt | 
attributes data-driven: extraction, combination values | 
(numerical, symbolic é s ; 1 
features) model-driven: to constrain range of values for attributes | 
scoring calculus algorithms values | 
conditions data-driven: tests valid / not valid | 
(structural ; | 
1 udgment 
relations) — juce 
  
  
  
model-driven: to constrain 
range of values for attributes 
  
| 
| 
| 
| 
| 
modified concepts 
  
Figure 6: Knowledge Types for Image Understanding 
KNOWLEDGE PROCESSES RESULTS 
  
  
level n task generation output: 
high level 
description B 
eee C 
level i | objects matching symbolic 9 
names T 
oo. oe ... ... R 
level2 | segmentation segmentation segmented © 
i pattern 
| level 1 | distortion normalization enhanced pattern 
| | 
| levelO | pattern recording pattern f(x) 
| formation 
  
  
  
  
  
  
Figure 5: Hierarchical Processing Model 
one can be used in a top-down and bottom-up fash- 
ion. Fig. 6 illustrates such knowledge types for im- 
age understanding. The basic entities are concepts 
modeling objects and events. The models are used 
for verification and propagation purposes. Features 
are described by attributes, again both directions 
of data flow are performed. In a similar way, con- 
straints as relationships between concepts and at- 
tributes are established. One of the most serious 
problems when dealing with noisy uncertain data is 
scoring. The efficiency of a system strongly depends 
on an adequate scoring calculus. It provides the 
hints which transformation should be applied next, 
which knowledge should be evaluated, and what in- 
termediate results are to be processed. 
This depiction of knowledge types forms the basis 
for a hybrid knowledge representation system as de- 
scribed in the next section. It allows us to handle 
the complete interpretation process as state search 
and as an optimization problem regarding a certain 
scoring calculus. Assuming a knowledge base is con- 
structed according to Fig. 6, the optimal interpreta- 
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B3. Vienna 1996 
tion can be characterized by 
B'(f(x) — argmazp(g(B,W,f(x)) (15) 
where B denotes the implicitly given set of potential 
descriptions, Y the knowledge base, and g a scoring 
function of the chosen calculus. 
3 A Hybrid Representation System 
The development of systems for object recognition 
and scene interpretation requires the representation 
of logical and numerical models. Therefore, cooper- 
ation of both knowledge based and statistical /neural 
techniques is necessary. Representations based on 
the statistical evaluation of a training sample are the 
backbone for holistic object recognition based on nu- 
merical features. Explicitly represented knowledge 
provides the decomposition of objects into parts and 
of scenes into objects. Furthermore, it enables the 
use of constraints and relationships which describe 
the structural properties of a problem domain and 
a special task. 
A hybrid representation system is described ac- 
cording to the basic discussion above. Its archi- 
tecture and overall organization of explicit knowl- 
edge has been developed regarding the distinction of 
knowledge types as outlined in Fig. 6. Within this 
paradigm the integration of analogous models like 
statistical classifiers and artificial neural networks is 
achieved in a very natural way. Whereas the knowl- 
edge based components deal with structural prop- 
erties, neural networks are concerned with holistic 
classification and scoring tasks. 
In the next subsection the semantic network lan- 
guage ERNEST is described which builds the frame- 
work for the hybrid representation system. After- 
wards, the hybrid approach is outlined combining 
ERNEST and artificial neural networks! (ANNs). 
In an analogous way statistical classifiers can be incorpo- 
rated. For simplicity, however, we only refer to ANNs in the 
following.