inbul 2004 
——À 
thors have 
IS. not 3 
accuracy 
alle score 
tween the 
.) (as the 
D between 
acy of the 
mging the 
; was done 
When two 
lc overlap 
oser mean 
the large 
adiometric 
nce values 
1 sets were 
ssification 
ding mean 
ed. Figure 
en overall 
ing mean 
c 
e 
).35 
| and 
x linear 
meters. 
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol XXXV, Part B4. Istanbul 2004 
This relationship can be seen between various data sets and 
have been investigated by the authors. However the obtained 
regression formula changes for various cases but the strong 
linear relationship between MQ and accuracy measures is 
preserved. This is the predicted relationship at previous section 
and confirms the mentioned relation betwcen uncertainty and 
accuracy. 
4. DISCUSSION 
The obtained linear relationship between mean quadratic score 
and overall accuracy is a good representation of the famous 
inverse relation between accuracy and uncertainty. In the other 
hand this relation can help us to guess the probable value of the 
accuracy. parameters (which need to collect some reference 
data) without any field observations. Although we can't define 
the exact value of these parameters but the estimated 
approximate values will be near the real values to some extent. 
This is caused from this fact that some environmental and 
procedural parameters influence the estimated linear 
relationship and then in some cases slop and intercept of the 
regression line can be the other values. 
Generally all of the factors involved in the accuracy assessment 
process can affect the final estimated accuracy values and 
therefore it may compute some different values for a given 
classification. Some of these factors have been listed by 
Congalton (1991) as: ground data collection, classification 
schema, spatial autocorrelation, sample size and sampling 
scheme. 
Two major aspects of the ground data collection are sampling 
schema, and sample size. These two issues affect the overall 
accuracy estimation and therefore can lead to bias estimation of 
the accuracy. Thus if we change any parameter that have a 
major influence on the estimated accuracy (e.g. sampling 
schema); we have a different value for accuracy and therefore 
MQS can not have a fixed relationship with the all of these 
different accuracies that arc for a particular classification. 
Considering this problem we investigated the relation between 
the accuracy and the sample size and concluded that a sample 
size between 70-100 pixels per class can lead to a reliable 
accuracy assessment. However, generally this depends on some 
environmental aspects [Congalton, 1991]. 
Sampling scheme also can have a notable effect on the accuracy 
assessment. Congalton (1988) notes that it is the spatial 
complexity of a given environment which dictates the 
appropriate sampling scheme(s) to be used for creating error 
matrices necessary to assess the accuracy of maps generated 
from remotely sensed data. Thus each strategy for sampling and 
ground truth gathering can affects the overall accuracy and 
consequently the relationship between MQ and OA. 
Some of the objects properties have influences on the 
uncertainty and accuracy derived from the classification results. 
Geometric properties (e.g. objects size), spectral properties (e.g 
Spectral similarity) of the objects are two major aspects that 
influence both of accuracy and uncertainty measures. Although 
these object properties present in the uncertainty and accuracy 
relation but have not the same effect on the accuracy and 
uncertainty. Therefore they prevent establishing a robust 
relationship between uncertainty and accuracy measures. As a 
consequence of this problem, we can not propose a valid fixed 
formula that gets uncertainty measure and gives the accuracy 
value for all cases. 
977 
As a consequence we can use the mean quadratic score as a cost 
free parameter that can tell us how much the classification is 
reliable without any need to collect the ground truth data. In 
comparing the individual classification results that have the 
same classification algorithm but have been done by different 
persons this parameter can be used. The smaller MQS the more 
accurate result. In the other way if we classify some data and 
after that perform some modifications on the entered data (or 
the other parameters) and then perform a new classification thus 
we can see the results of these modifications by estimating the 
MQS for both of the classifications and comparing them. Again 
that classification which gives the smaller value for the MQ can 
be selected as the better classification. 
5. CONCLUSION 
In this paper a linear relation between an uncertainty measure 
and an accuracy parameter has been investigated. The 
uncertainty measure that used the mean quadratic score with the 
overall accuracy and kappa coefficient was chosen as the 
commonly used accuracy measures. The famous inverse 
relationship between uncertainty and accuracy has been 
confirmed by this experiment and a strong relation between an 
averaged uncertainty value (MQS) and an averaged accuracy 
value (OA) have been found. 
Although we have mentioned that these parameters are 
influenced by the various factors but we can use the MQS in 
comparing different classifications (not classifiers!). In fact this 
is the MQS that can be used to compare the reliability and 
performance of the classifications and the obtained relation 
(between OA and MQS) can not be used to predict the exact 
accuracy of the classification result. This is caused from this 
fact that both of the accuracy and uncertainty are influenced by 
some various factors that can alter the parameters of the linear 
relation. Among these effective factors the sampling scheme, 
sample size, classification procedure, and objects properties are 
some the most important effective parameters on the accuracy 
assessment and uncertainty analysis process. 
In this study maximum likelihood classifier was used as a 
common procedure in the classification literatures. This 
procedure is able to produce probability vectors that are used to 
calculate the quadratic score. Therefore if any classifier that can 
not produce such information is used then we can not compute 
an uncertainty measure. Using another classifier such as 
minimum distance or artificial neural networks we should 
define an appropriate uncertainty measure and then test it 
whether it has any straight relation with the accuracy measures. 
This is a topic for the future investigations but as a general 
consequence it is anticipated that the linear relationship 
between MQS an OA will be remain. 
We have found that among the uncertainty measures the mean 
quadratic score has a strong and reliable relationship with the 
commonly used accuracy measures. This relationship can be a 
good basis for the future investigations that can lead to the 
classification based accuracy measures and avoiding some 
problematic data related issues of ground truth data collection. 
The other uncertainty measures can be tested to define whether 
they have any stronger and more stable relation than the one we 
have found?