INTRODUCING AN ACCURACY 
INDICATOR BASED ON UNCERTAINTY RELATED MEASURES 
* 
S. B Fatemi* B. Mojaradi ”, M. Varshosaz b 
* K.N. Toosi University, Geodesy and Geomatics Faculty, Valiasr Tehran, Iran - (sbfatemi, mojaradi)@yahoo.com 
b > M . . : ER - . - sq» "T - : 
. K.N.Toosi University, Geodesy and Geomatics Faculty, Valiasr Tehran, Iran — varshosazm@kntu.ac.ir 
KEY WORDS: Analysis, Accuracy, Classification, Error, Indicators, Remote Sensing, Simulation 
ABSTRACT: 
Traditionally accuracy assessment of the classification results uses some collected reference data (ground truth). Ground truth 
collection is a time-consuming and money-swallowing activity and usually can not be done completely. Uncertainty is an important 
subject in remote sensing that can appear and be increased sequentially in a chain of remote sensing from data acquisition, geometric 
and radiometric processing to the information extraction. Conceptually the relation between uncertainty and accuracy is an inverse 
relation. This relation can aid us to construct a relation between accuracy measures and uncertainty related measures. In this paper 
we investigate this relation using the generated synthetic images (for the sake of the reliability of the obtained results) and try to find 
an uncertainty related measure that has a strong relationship with the accuracy parameters like overall accuracy. We have found that 
among the uncertainty measures the mean quadratic score has the strong and reliable relationship with the commonly used accuracy 
measures. This relationship can be a good basis for the future investigations that lead to the classification based accuracy measures 
and avoiding some problematic data related issued of ground truth data collection. 
1. INTRODUCTION e Can we define a shift and a scale factor for the uncertainty 
of the overall accuracy? 
Uncertainty is an important subject in remote sensing which has 
recently attracted the attention of many researchers. It can 
appear in a chain of remote sensing from data acquisition, 3. 
geometric and radiometric processing to information extraction 
UNCERTAINTY MEASURES 
  
with its value increasing sequentially during image processing 
and image analysis. A thematic map produced by a different 
approach and various satellite images must be reliable to be 
used in GIS. Therefore the source causing uncertainty to 
increase must be defined and modeled or removed. Having 
extracted any information from satellite imagery, the 
presentation of uncertainty as an indicator is essential for users 
and it is important to define a measure to quantify uncertainty. 
Goodchild (1995) argues that uncertainty is "generic and 
reasonably value-free, and implies nothing about sources or 
whether they can be corrected ". Stephanou and Sage (1987) 
said "uncertainty indicates lack of knowledge and is a concept 
to express the inability to be confident of and knowledgeable 
about the truth value of a particular data". The generic meaning 
of uncertainty implies that is if two individuals give the same 
answer to a question, one might be more certain than the other. 
A simple definition of uncertainty can then be “the probability 
of error". 
In this research it we will try to answer the following questions: 
* [s there any clean and formular relation between the 
uncertainty and the overall accuracy? 
e [f yes, is this relation independent from the source of 
uncertainty? 
* Which uncertainty measure is more stable for showing 
accuracy? 
* Which uncertainty measure isn't sensitive to the source of 
accuracy? 
Regarding the main concept of the research, we need to 
investigate possible uncertainty measures. Based on the 
information theory an information source from set of symbols 
(a,,a,,..a,} generates a random of symbols . The probability of 
the event a; that the source will produce is P(a;) and 
> Pla,)=1 (1) 
= 
[(a;)=Log(1/P(a;))=-Log(P(a;)) (2) 
The amount of self information attributed to event aj is 
inversely related to the probability of a; .The base of the 
logarithm in equation (2) determines the unit used to measure 
information. 
If k source symbols are generated, the low of large numbers 
stipulates that, for a sufficiently large value of k, symbol a; will 
(on average) be output k*p(aj). Thus the average sell 
information obtained from k outputs is: 
-k P(a,) Log P(aj)-k P(a;)L og P(a»)-.....-k P(a,) Log P(a;) 
— KS. P(a,)LogP(a;) (3) 
i=} 
Interna 
Aera 
In ima 
variabl 
for i 
uncerte 
pixel i 
inform 
The en 
inform 
its true 
uncerte 
them b 
As ant 
score ( 
quadra 
uncerte 
probal 
accura 
where 
score ¢ 
This n 
maxim 
measur 
Figure 
slope « 
for ex 
probab 
Uncertair 
A