, September 1-3, 2010 
In: Paparoditis N., Pierrot-Deseilligny M.. Mallet C.. Tournaire O. (Eds), 1APRS. Vol. XXXVIII. Part 3A - Saint-Mandé, France. September 1-3, 2010 
11 
0.9 1 
with the parameter 
lividual classifiers, 
irst and the overall 
erence. Two of the 
trategies were also 
d. These strategies 
ted Sum (WS). A 
thods can be found 
y is just a global 
ation process, two 
he performance of 
: commission and 
ication accuracy, 
show how the 
v'es or deteriorates 
bined classifiers, 
orrectly identified 
on errors are the 
have identified as 
is proposed in this 
b) environment. 
SIS 
irithms 
vidual classifiers. 
Table 4. SVMs 
rail classification 
t average overall 
respectively. The 
as a reference in 
uracy 
SVMs 
96.9 
96.5 
97 
96.6 
96.75 
0.24 
classifiers for the 
curacies achieved 
te best individual 
Figure 3. For the 
four test areas, it is clear that the overall performances of FMV 
are better than those of the other combination methods. FMV 
performs better than WS, and both outperform MR. It is worth 
mentioning that even though the MR resulted in the worst 
performance, it still performed better than the best single 
classifier. Taking into account, the limited room for 
improvement beyond 96.9% accuracy due to other errors in 
image acquisition and image to lidar geographic registration, 
the best average improvement in classification accuracy of 
1.1% is obtained from FMV algorithm, followed by 0.82% 
average improvement from WS algorithm. MR resulted in the 
worst performance and only improved the results by 0.66%. 
The question still remains as to whether these improvements are 
statistically significant. In order to answer this question, first, 
the standard deviation (SD) of the classification accuracies 
produced by each classifier for the four test areas is determined 
to express the variability in classification accuracies from the 
mean as shown in table 4. With only four test areas the 
estimate of the SD is limited. However, the low standard 
deviation of 0.24% for the SVM results indicates that the spread 
of the accuracies for the four tests areas is small and hence 
accuracies tend to be very close to the mean. In the case of 
SOM and CTs, the higher SD values. 1.04% and 2.40% 
respectively, indicate that the accuracies are spread over a 
larger range of values for the four test areas. The SD was then 
used as a confidence measure in the conclusions on the quality 
of the accuracies derived by the three classifiers and the 
combined classifiers. We can assume that the reported margin 
of error (MOE) is typically about plus/minus twice the standard 
deviation (a range for an approximately 95% confidence 
interval). For this work we used a margin of accuracy of 0.72%, 
which is three times the standard deviation of the SVM results, 
to define the improvements in accuracy that are considered 
statistically significant, as shown by the dashed line in figure 3. 
Any improvements in classification accuracy more than the 
dashed horizontal line are deemed to be significant. It can be 
concluded that the application of FMV results in the most 
significant improvement in classification accuracy. The 
improvements achieved by other techniques are either 
extremely close to the significance value, and therefore 
considered to be marginally significant, or below the value of 
significance. 
Algorithm 
Figure 3. Performance comparison of the FMV based 
combination with existing algorithms, compared 
with the performance of the best individual 
classifier, SVMs. Improvements exceeding the 
dashed horizontal line are considered to be 
significant. 
5.2 Class-Specific Accuracies 
An assessment of the produced commission and omission errors 
confirms that the FMV fusion performed the best in most cases 
as shown in table 5. Most of the class-commission and omission 
errors are reduced by the FMV fusion. Whereas the application 
of SVMs resulted in average of 4.45 % and 5.13 % for 
commission and omission errors respectively, the application of 
FMV fusion resulted in average of 3.39 % and 2.15 % for 
commission and omission errors respectively. Contrary, there 
was an increase in commission and/or omission errors for a few 
classes as shown in the shaded cells of table 5. However, those 
classes are still classified with relatively low commission and 
omission errors. Another advantage of the FMV fusion over 
SVMs is that the achieved errors are less variable as shown in 
table 5. Whereas the application of SVMs resulted in standard 
deviation of 3.22 % and 5.25 % for commission and omission 
errors respectively, the application of FMV fusion resulted in a 
comparable SD for commission errors, 4.43%. and significantly 
reduced the SD for omission errors to 1.88 %. The visual 
assessment interpretation (Figure 4) clearly shows a relatively 
high degree of noise in the SVMs-based classification results. 
In contrast to this, the classification that is based on the FMV 
appears more homogenous. 
Best Classifier 
FMV Fusion 
Com. (%) 
Om. (%) 
Com. (%) 
Om. (%) 
B 
4.65 
2.77 
1.31 
0.82 
GO 
T 
3.18 
1.97 
1.36 
2.87 
z 
R 
4.81 
0.06 
0.02 
3.99 
G 
0.06 
5.10 
6.17 
0.03 
B 
9.79 
7.80 
16.72 
0.36 
T 
0.35 
6.12 
0.02 
3.82 
C3 
R 
4.36 
0.98 
1.15 
1.69 
03 
G 
10.30 
4.06 
9.34 
1.01 
B 
8.23 
11.11 
3.35 
1.37 
o 
c5 
T 
0.89 
3.36 
2.04 
4.96 
R 
4.08 
0.76 
3.41 
0.01 
Lu 
G 
3.69 
7.04 
0.01 
3.67 
B 
4.04 
21.28 
2.56 
0.56 
E 
T 
0.63 
3.94 
0.85 
5.36 
2 
R 
4.10 
0.42 
0.05 
3.78 
G 
7.96 
5.30 
5.87 
0.06 
Mean 
4.45 
5.13 
3.39 
2.15 
SD 
3.22 
5.25 
4.43 
1.88 
Table 5. Comparison of errors using the best classifier, SVMs, 
with the classification resulting from FMV, for the 
four test areas. B, T, R and G refer to buildings, trees, 
roads and grass respectively. Com. and Om. Refer to 
commission and omission errors respectively. 
Figure 4. (a) Classification results of the best classifier (SVMs); 
(b) Error correction after applying the FMV fusion 
algorithm. Black: buildings; dark grey: trees; light 
grey: roads; white: ground.