International Archives of Photogrammetry and Remote Sensing, Vol. 32, Part 7-4-3 W6, Valladolid, Spain, 3-4 June, 1999 
72 
Syntactic 
Semantic 
models 
models 
User conjecture modelling 
Information fission 
Information aggregation 
Fig. 4. Simplified diagram of an information fusion system. Heterogenous sources of information are splitted into 
elementary sources: the information fission. The information aggregation completes the fusion process. 
of information commensurability, e.g. estimated texture 
features using co-occurence matrix are not comparable with 
parameters of Markov random fields. 
• Cluster model: image features or estimated physical 
parameters have n-dimensional representations. Due to 
observation noise or model approximations, the feature 
space is not occupied homogeneously. Thus, another level of 
information abstraction, and fusion at the same time, is the 
type of feature grouping, i.e. the cluster models, and the 
associated parameters. Clustering can represent information 
only for one category of the features. 
• Syntactic modelling: stochastic grammars are defined to 
represent the order of joint spaces, image features and 
models. The induced syntax can be interpreted as a 
clustering of incommensurable information representations. 
Thus, information in incommensurable representations can 
be aggregated. In the case of fusion of physical parameters 
extracted from different data types, the result of information 
fusion is equivalent to a supervised classification, otherwise, 
for the case of abstract features, the classification is 
unsupervised. 
• Semantic representation: due to the impossibility to 
estimate the physical parameters, an unsupervised 
classification of the data is produced. Augmentation with 
meaning requires a higher level of abstraction. Prior 
information in form of training datasets or expert knowledge 
is used to create semantic networks. Thus, the observations 
are labelled and the contextual meaning is defined. 
The performance of information extraction depends critically on 
the descriptive or predictive accuracy of the probabilistic model 
employed. Accurate modelling typically requires high 
dimensional and multi-scale modelling. For heterogeneous 
sources, accuracy also depends on adaptation to local 
characteristics. 
6. DATA AND INFORMATION MINING 
One of the difficulties in many data fusion applications for 
remote sensing image classification is the identification of the 
datasets relevant for the user. Here, the information retrieval task 
becomes an information mining task. The Bayesian inference can 
be applied again, but the goal is to find the set of images, 
generally acquired from different sensors, which might contain 
the information desired by the user. Thus, the inference is used to 
find the posterior distribution over the set of available images 
according to the user hypothesis in a given application. 
In the present context, data mining will be understood as 
searching for datasets with certain properties, and information 
mining refers to the search of hidden relationships among image 
features, either in one or in multiple image datasets. Data mining 
will produce an index, information mining a label. 
The concept and hierarchy of information representation allows 
the integration of different tools for information mining. The goal 
is to explore the information content of the images and decide 
which ones are relevant for the user’s application. The tool we 
propose is mainly based on interactive decision using data 
visualization, e.g. image browsing integrated with image 
segmentation, or information visualization, e.g. exploration of 
the clusters’ structure in the feature space and the interactive 
design of the decision surfaces. The block diagram of a simple 
data and information mining tool is presented in Fig. 5. 
Recently, solutions of the MAP estimator have been proposed 
combined with an expectation-maximization and deterministic 
annealing, in which entropy minimization maximizes the amount 
of evidence supporting each parameter, while minimizing 
uncertainty in the sufficient statistics and cross-entropy between 
the model and the data. A similar approach was developed using 
two levels of Bayesian inference, model fitting and model 
selection, for the estimation of the hidden parameters of a 3-level 
hierarchical model. These methods of inference can be integrated 
in tools for information mining in the assumption of datasets 
explained by multiple models that coexist together.