1 
DECISION FUSION OF MULTITEMPORAL SAR AND 
MULTISPECTRAL IMAGERY FOR IMPROVED LAND COVER CLASSIFICATION 
B. Waske a , J. A. Benediktsson b ’* 
a Center for Remote Sensing of Land Surfaces (ZFL), University of Bonn, 
53113 Bonn, Germany - bwaske@uni-bonn.de 
b ’* Department of Electrical and Computer Engineering, University of Iceland, 
107 Reykjavik, Iceland - benedikt@hi.is 
KEY WORDS: multisensor imagery, data fusion, classification, SAR, multispectral 
ABSTRACT 
A strategy for classifying multisensor imagery, consisting of multispectral and SAR data is presented. Each image source is 
individually classified by a support vector machine (SVM). In decision fusion the outputs of the pre-classification are combined to 
derive the final class memberships. This fusion is performed by another SVM. The results are compared with well-known parametric 
and nonparametric classifier methods. The proposed SVM-based fusion approach outperforms all other concepts and improves the 
results of a single SVM that is trained on the whole multisensor data set. Moreover the results clearly show that the individual image 
sources provide different information and a multisensor approach generally outperforms single-source classifications. 
1. INTRODUCTION 
Regions with a high degree of agricultural land-use are 
investigated in numerous remote sensing based land cover 
studies. These areas are characterised by a great temporal 
variability and typical spatial patterns of high-frequent land 
cover changes between individual agricultural field plots. Thus 
single-date approaches are often inefficient, due to great 
temporal differences in crop phenology. Multitemporal 
applications are more appropriate in this context and improve 
the accuracies of existing classifications (e.g., Blaes et al., 
2005). 
However, the efficiency of optical imagery is often limited by 
weather conditions such as solar illumination and cloud cover. 
Hence a reliable generation of image time series within one 
growing season is difficult, especially in regions like Central 
Europe. This is a drawback, particularly for operational 
monitoring systems. SAR data are normally independent from 
these factors and in several studies the accuracies were 
increased by using multitemporal SAR imagery (e.g., Chust et 
al., 2004, Blaes et al., 2005). 
Beside the multitemporal data, in other studies, the positive 
impact of multisensor data on classification accuracy was 
illustrated, e.g., data sets consisting of SAR and multispectral 
imagery (Benediktsson and Kanellopoulos, 1999; Chust et al., 
2004, Blaes et al., 2005, Huang, et al., 2007). 
The main reason for the success of these studies is the use of the 
two different sensor systems that operate in different 
wavelengths, ranging from visible to microwave. Consequently 
dissimilar land cover information is acquired and the 
classification accuracy can be increased by multisensor imagery. 
Even if some applications are based on conventional statistical 
methods such as the well known maximum likelihood classifier 
(Chust et al., 2004, Blaes et al., 2005, Huang, et al., 2007), 
those methods are often not ideal for multisensor imagery, 
because in the very most cases the class distributions cannot be 
modelled by adequate multivariate statistical models 
(Benediktsson et al., 1990). Consequently, more sophisticated 
strategies are more applicable in this context and Richards 
(2005) pointed out that the development of adequate strategies 
to combine inherent information content in complementary data 
sets is perhaps the greatest ongoing challenge in the field of 
remote sensing. 
Multiple classifier systems give an interesting approach, which 
is applied in several studies to multitemporal and mulitsource 
imagery. The general concept of classifier ensemble is based on 
training a classifier on resampled input data. Afterwards the 
outputs of the independent classifiers are combined to create the 
final result. 
Brown de Colstoun (2003) applied a decision tree on 
multitemporal images from the Enhanced Thematic Mapper- 
Plus (ETM+) to differentiate between 11 land cover types. 
Ensemble techniques such as boosting were successfully used 
to improve the final classification results. In Waske et al. (2006) 
a multiple classifier system that is based on a random selection 
of input features, was used successfully for classifying 
multitemporal Envisat ASAR and ERS-2 imagery. The overall 
accuracy was significantly increased compared to the results 
achieved by a single decision tree. In Briem et al. (2002) 
different single and multiple classifiers were considered for 
classifying multisource data. The classifier systems always 
outperformed the single algorithms in terms of accuracy. 
Gislason et al. (2006) applied the concept of Random Forests to 
Landsat MSS image and topographical information and 
achieved relatively high accuracies. 
Other multiple classifier systems are based on a combination of 
different algorithms: In Benediktsson and Kanellopoulos (1999) 
a multisensor data set was classified by combining the outputs 
of a neural network and a statistical classifier. The two 
classifiers were trained individually on the SAR and 
multispectral imagery. Afterwards these pre-classification 
outputs were combined by decision fusion. 
* Corresponding author.