Roland Geibel 
  
3.2 Performance measurement 
3.2.1 Measures for comparing ground truth and machine segments 
In the following segments of the sensed truth data are denoted as T-segments and segments which are produced by a 
segmentation procedure, are denoted as machine segments (M-segments). In the ideal case the segmentation procedure 
produces one M-segment for each T-segment and both segments are exactly equal. Since the borderlines can not be 
found exactly because of distortions in the real case the T- and M-segments will only intersect partly even if the relation 
is one-by-one. To meter the correspondence the intersection area of T-segment and M-segment is taken as measure. In 
order to denote the intersection ratio independently of the size of the intersection area it is normed by the size of the 
segment. This produces the measures 
Sr =As/Ar and SM =As/AM (4) 1111 
   
  
| which are related to the area of the T-segment A1 and of the M-segment Ay. 
  
  
  
  
  
  
  
Correct classification. If the intersection ratio of M- and T-segment is very a 
low, then it should not be assumed that they are corresponding. As a criterion 
for a correct classification a minimal relative intersection 
ST 2 Sh and SM > Si (5) 
  
  
is required. To ensure that a T-segment can correspond to only on M-segment as 
correct, Sy, > 0.5 is required. The example in Fig. 4a shows with b 
$1215/2420.625 and Sm=15/20=0.75 and thus a correct classification, while in 
Fig. 4b S,=8/24=0.333 and Sm=8/20=0.4 and thus the segments are not 
classified as correct. 
  
  
  
  
  
  
  
  
  
Fig. 4: One M-Segment. 
a) correct, b) not correct 
Over-segmentation. It is possible that the expected T-segment is represented by several parts of M-segments. The 
intersection area Ag then results from the total area of the M-segments ( S - T ^ (MI U M2 ... ). The relative 
intersection is defined analogously to equation (4) as 
Sto = As/AT and Smo = As/ Aviom m (6) 
If more than one M-segment intersects with the T-segment, then 
only one M-segment can be marked as classified correctly. If no 
M-segment was assigned as corresponding correctly, then a T- 
segment is classified as over-segmented, if 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
a b 
Sto > Sun and Smo > Sin (7) 
hold. If an M-Segment was assigned a correct classification, 
then the T-segment is classified as over-segmented, if in 
addition the average of the measures of the new correspondence 51 — 
is bigger than that of the measures of the one M-segment 32 81 82 
classified as correct. M1 M2 M2 
S10 *Syo , S1 *Sy (8) fet 
> 
2 2 ; 
Fig. 5: Two M-segments. a) not correct, not over- 
If the condition in equation (8) is not satisfied, then the segmented, b) not correct, over-segmented c) 
classification of the one correct segment will be kept and the correct, over-segmented, d) correct not over- 
other M-segments are not associated to the T-segment. segmented 
Fig. 5 shows cases, in which the T-segment is intersected by two M-segments. In Fig. 5a-b the M-segments are so 
small, that in both cases the M-segments could not be classified as correct. In Fig. 5a the T-segment is neither over- 
segmented, since both of the M-segments together do not fulfil the condition of equation (7), while in Fig. 5b it holds. 
In Fig. 5c-d M2 was assigned to the T-segment as a correct classification. Since by the position of MI in Fig .5c 
  
330 International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B3. Amsterdam 2000. 
  
  
CY MN == AN m0 M 
rN 65 = 
Kn rf a EN ef m oY OW Pf 
823 
rN "mm 1 rm