ISPRS Commission III, Vol.34, Part 3A ,,Photogrammetric Computer Vision", Graz, 2002 
  
Recognition Rate Depending on the Minimum Score Receiver Operating Characteristic 
  
  
   
   
2 
e 
False Positives [96] 
Recognition Rate [%] 
eo e 
Qu cum m m. 
| 
I 
/ 
l 
MN 
9 
  
  
  
  
o 
1 
o 
o 
  
  
  
— - Normalized Cross Correlation 
20 40 60 80 30 40 50 60 70 80 90 100 
Minimum Score [%] Recognition Rate [%] 
—— Shape Based Matching —e- PatMax 
Modified Hough Transform — - Hausdorff Distance 
  
  
Figure 4: The recognition rate of different approaches indicates 
the robustness against occlusions. The left figure shows the 
recognition rate of the five candidates depending on the minimum 
score. In the right figure the receiver operating characteristic is 
shown. 
Normalized Cross Correlation Hausdorff Distance 
  
x 
m 
80 E 
t 
60 s C 
  
rare 
Score [%] 
| 
40} 
  
  
20 
0 
2 
  
80 100 20 40 60 80 100 
Visibility 1%] 
Modified Hough Transform PatMax 
= 
0 40 60 
Visibilitv [961 
Shape Based Matching 
  
  
  
  
  
  
20 80 100 20 80 1 80 100 
60 0 60 0 60 
Visibility [26] Visibility [26] Visibility [26] 
Figure 5: Extracted scores plotted against the visibility of the 
object. 
To avoid that this peculiarity results in an unfavorable compari- 
son for PatMax we decided to take the angle tolerance interval 
into account when using PatMax®. The recognition rate of the 
normalized cross correlation does not reach 50% at all, even if 
the minimum score is chosen small. In the right plot of Figure 
4 the receiver operating characteristic curve is shown, i.e., the 
false positive rate is plotted depending on the recognition rate. 
Even for a small recognition rate the number of false positives 
dramatically increases up to 32% (not visible in the plot due to 
axis scaling) when using the Hausdorff distance. The normalized 
cross correlation also tends to return false positives if the recog- 
nition rate approaches the maximum value of about 50%. For 
high recognition values even PatMax®returns wrong matches. 
Also here, the best results are obtained using the modified Hough 
transform and the shape-based matching. 
Figure 5 displays a plot of the returned score value against the 
estimated visibility of the object, i.e., the correlation between the 
visibility of the object and the returned score value is visualized. 
The instances in which the model was not found are denoted by 
a score of 0, i.e., they lie on the x axis of the plot. For all ap- 
proaches except for the Hausdorff distance the minimum score 
was set to 30%, i.e., in those images in which the object has a 
visibility of more than 30%, it should be found by the recognition 
method. For the Hausdorff distance a minimum forward fraction 
of 50% was used. In the plot of the Hausdorff distance the wrong 
matches either have a forward fraction of 0% or close to 50%, 
because of some false positives. Here, a noticeable positive cor- 
relation can be observed, but several objects with a visibility of 
far greater than 50% could not be recognized. This explains the 
A- 372 
  
Recognition Rate Depending on the Minimum Score 
  
  
  
  
  
  
  
= 
2 
© 
X 
£c 
2 
= 
c 
o 
o 
o 
a 
—— Shape Based Matching N 
20r Modified Hough Transform N 
— - Normalized Cross Correlation \ 
10r| -e- PatMax \ 
— - Hausdorff Distance me 
  
  
9 0 20 30 40 50 60 70 80 90 
Minimum Score [?6] 
Figure 6: The recognition rate of the different approaches indi- 
cates the robustness against arbitrary illumination changes. This 
figure shows the recognition rate of the five candidates depending 
on the minimum score. 
lower recognition rate in comparison to our approaches, which 
was mentioned above. The normalized cross correlation also 
shows positive correlation but the points in the plot are widely 
spread and many objects with high visibility were not recognized. 
In contrast, the plots of our new approaches show a point distri- 
bution that is much closer to the ideal: The positive correlation 
is evident and the points lie close to a fitted line, the gradient of 
which is close to 1. In addition, objects with high visibility are 
recognized with a high probability. Also PatMax® results in a 
nearly ideal point distribution. Nevertheless, in some occlusion 
cases the object was not found even though the visibility was sig- 
nificantly higher than 3096. 
Illumination. Figure 6 shows a comparison of the robustness 
of all approaches. The recognition rate of the normalized cross 
correlation is now substantially better than in the case of occlu- 
sions. This can be attributed to its normalization, which compen- 
sates at least global illumination changes. The Hausdorff distance 
shows also good results especially in the case of large values for 
the minimum score, but could not reach the performance of the 
shape-based matching approach by far. If the minimum score 
is set low enough, the recognition rate of the modified Hough 
transform surpasses that of the shape-based matching, however, 
for higher values its recognition rate rapidly falls. Here, also 
PatMax® shows very good results: the recognition rate is nearly 
constant when increasing the minimum score from 10% to 60% 
but also drops down during further increase. 
3.3.2 Accuracy Since the Hausdorff distance does not return 
the object position in subpixel accuracy, only the accuracy of the 
five remaining candidates are evaluated in this section. To assess 
the accuracy of the extracted model position and orientation a 
straight line was fitted to the mean extracted coordinates of posi- 
tion and orientation. This is legitimated by the linear variation of 
the position and orientation of the IC in the world as described in 
Section 3.2. The residual errors of the line fit, shown in the Fig- 
ures 7 and 8, are an extremely good indication of the achievable 
accuracy. 
As can be seen from the Figure 7 the position accuracy of the nor- 
malized cross correlation, PatMax®, the modified Hough trans- 
form, the shape-based matching and the least-squares adjustment 
and are very similar. The corresponding errors are in most cases 
smaller than 1/20 pixel. The two conspicuous peaks in the error 
plot of Figure 7 occur for all three approaches with similar mag- 
nitude. Therefore, and because of the nearly identical lines, it is 
probable that the chip was not shifted exactly and thus, the error 
must be attributed to a deficient acquisition. Since the errors in y 
during a vertical translation approximately have the same magni- 
tude as the errors in x we refrain from presenting these plots.