ISPRS Commission III, Vol.34, Part 3A ,,Photogrammetric Computer Vision", Graz, 2002
Recognition Rate Depending on the Minimum Score Receiver Operating Characteristic
2
e
False Positives [96]
Recognition Rate [%]
eo e
Qu cum m m.
|
I
/
l
MN
9
o
1
o
o
— - Normalized Cross Correlation
20 40 60 80 30 40 50 60 70 80 90 100
Minimum Score [%] Recognition Rate [%]
—— Shape Based Matching —e- PatMax
Modified Hough Transform — - Hausdorff Distance
Figure 4: The recognition rate of different approaches indicates
the robustness against occlusions. The left figure shows the
recognition rate of the five candidates depending on the minimum
score. In the right figure the receiver operating characteristic is
shown.
Normalized Cross Correlation Hausdorff Distance
x
m
80 E
t
60 s C
rare
Score [%]
|
40}
20
0
2
80 100 20 40 60 80 100
Visibility 1%]
Modified Hough Transform PatMax
=
0 40 60
Visibilitv [961
Shape Based Matching
20 80 100 20 80 1 80 100
60 0 60 0 60
Visibility [26] Visibility [26] Visibility [26]
Figure 5: Extracted scores plotted against the visibility of the
object.
To avoid that this peculiarity results in an unfavorable compari-
son for PatMax we decided to take the angle tolerance interval
into account when using PatMax®. The recognition rate of the
normalized cross correlation does not reach 50% at all, even if
the minimum score is chosen small. In the right plot of Figure
4 the receiver operating characteristic curve is shown, i.e., the
false positive rate is plotted depending on the recognition rate.
Even for a small recognition rate the number of false positives
dramatically increases up to 32% (not visible in the plot due to
axis scaling) when using the Hausdorff distance. The normalized
cross correlation also tends to return false positives if the recog-
nition rate approaches the maximum value of about 50%. For
high recognition values even PatMax®returns wrong matches.
Also here, the best results are obtained using the modified Hough
transform and the shape-based matching.
Figure 5 displays a plot of the returned score value against the
estimated visibility of the object, i.e., the correlation between the
visibility of the object and the returned score value is visualized.
The instances in which the model was not found are denoted by
a score of 0, i.e., they lie on the x axis of the plot. For all ap-
proaches except for the Hausdorff distance the minimum score
was set to 30%, i.e., in those images in which the object has a
visibility of more than 30%, it should be found by the recognition
method. For the Hausdorff distance a minimum forward fraction
of 50% was used. In the plot of the Hausdorff distance the wrong
matches either have a forward fraction of 0% or close to 50%,
because of some false positives. Here, a noticeable positive cor-
relation can be observed, but several objects with a visibility of
far greater than 50% could not be recognized. This explains the
A- 372
Recognition Rate Depending on the Minimum Score
=
2
©
X
£c
2
=
c
o
o
o
a
—— Shape Based Matching N
20r Modified Hough Transform N
— - Normalized Cross Correlation \
10r| -e- PatMax \
— - Hausdorff Distance me
9 0 20 30 40 50 60 70 80 90
Minimum Score [?6]
Figure 6: The recognition rate of the different approaches indi-
cates the robustness against arbitrary illumination changes. This
figure shows the recognition rate of the five candidates depending
on the minimum score.
lower recognition rate in comparison to our approaches, which
was mentioned above. The normalized cross correlation also
shows positive correlation but the points in the plot are widely
spread and many objects with high visibility were not recognized.
In contrast, the plots of our new approaches show a point distri-
bution that is much closer to the ideal: The positive correlation
is evident and the points lie close to a fitted line, the gradient of
which is close to 1. In addition, objects with high visibility are
recognized with a high probability. Also PatMax® results in a
nearly ideal point distribution. Nevertheless, in some occlusion
cases the object was not found even though the visibility was sig-
nificantly higher than 3096.
Illumination. Figure 6 shows a comparison of the robustness
of all approaches. The recognition rate of the normalized cross
correlation is now substantially better than in the case of occlu-
sions. This can be attributed to its normalization, which compen-
sates at least global illumination changes. The Hausdorff distance
shows also good results especially in the case of large values for
the minimum score, but could not reach the performance of the
shape-based matching approach by far. If the minimum score
is set low enough, the recognition rate of the modified Hough
transform surpasses that of the shape-based matching, however,
for higher values its recognition rate rapidly falls. Here, also
PatMax® shows very good results: the recognition rate is nearly
constant when increasing the minimum score from 10% to 60%
but also drops down during further increase.
3.3.2 Accuracy Since the Hausdorff distance does not return
the object position in subpixel accuracy, only the accuracy of the
five remaining candidates are evaluated in this section. To assess
the accuracy of the extracted model position and orientation a
straight line was fitted to the mean extracted coordinates of posi-
tion and orientation. This is legitimated by the linear variation of
the position and orientation of the IC in the world as described in
Section 3.2. The residual errors of the line fit, shown in the Fig-
ures 7 and 8, are an extremely good indication of the achievable
accuracy.
As can be seen from the Figure 7 the position accuracy of the nor-
malized cross correlation, PatMax®, the modified Hough trans-
form, the shape-based matching and the least-squares adjustment
and are very similar. The corresponding errors are in most cases
smaller than 1/20 pixel. The two conspicuous peaks in the error
plot of Figure 7 occur for all three approaches with similar mag-
nitude. Therefore, and because of the nearly identical lines, it is
probable that the chip was not shifted exactly and thus, the error
must be attributed to a deficient acquisition. Since the errors in y
during a vertical translation approximately have the same magni-
tude as the errors in x we refrain from presenting these plots.