The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B4. Beijing 2008
1193
smoothness and color to the observed D, then minimizing the
function.
The evaluation of segmentation relative to a training set is simply
a quantitative measure of the goodness of polygon matching. It
does not necessarily imply a good classification result. This is
particularly true in the event that a classification of primitives can
be used as a preliminary step to the ultimate assembly of objects
(see, for instance, Pichel et al. 2006). For example, consider an
evaluation of segmentation results relative to the ultimate
classification of an entire vehicle. This discounts the prospect of
first classifying vehicle parts such as windshield, hood, roof, etc.,
then assembling these parts into complete cars through dissolve
operations or other adjacency rules. However, the method
described here could easily be applied to such a scenario through
the provision of training sets for the individual car parts, then
evaluating the goodness of match between the segmentation and
the supplied primitives. These hierarchical relationships between
objects at different spatial scales could be more easily exploited
using OverSegmentation and UnderSegmentation. With any
software that produces nested segmentations at different scales (as
both ASTRO and eCognition do), the D measure could be
harnessed to compare predefined object primitives to a wide
variety of segmentations at different scales. In this way, optimal
scales for analysis could be identified by comparing the training
objects to different levels of the hierarchy.
The advantage of the measures we describe is that a quantitative
index can be generated relative to any set of training objects of
interest. The measures will also provide useful diagnostic
information for the efficacy of the segmentation relative to the
different object types. This characteristic of D is illustrated by
Figure 1, in which the performance of the different software is
shown to be very different when supplied with different kinds of
training objects.
In the event that two segmentation results have similar values of D,
the setup described here can be extended to incorporate additional
indices. However, the indices should be scaled to [0,1] and
increase the dimension of S, with the Euclidean norm D calculated
accordingly. The distribution of D in S is of great interest and
should be defined in order to determine the significance of
differences between segmentation results. Simulation studies are
needed to identify this distribution.
CONCLUSION
We have presented and demonstrated measures that facilitate the
identification of optimal segmentation results relative to a training
set. We propose that these measures are not only useful for the
selection of segmentations from an array of choices, but also have
utility in reporting the overall accuracy of segmentation, again
relative to the set of supplied training objects. This setup is useful
in the case where pre-defmed objects are to be located and
extracted (through a classification algorithm) from an image of
interest. The objective selection of a segmentation result (i.e. not
based on “expert opinion,” “visual interpretation” and the like)
necessitates such an approach. Additionally, the growing supply
of segmentation software means that inter-comparisons such as
that presented here could benefit from a set of quantitative, well
defined measures that communicate the effectiveness of the
software to find objects of interest. This paper presents an
approach that provides an initial basis for the consistent
comparison of segmentations resulting from varying parameters
and software.
REFERENCES
Antani, S., D.J. Lee, L.R. Long, G.R. Thoma. 2004. Evaluation of
Shape Similarity Measurement Methods for Spine X-ray Images.
Journal of Visual Communication and Image Representation.
Special issue: Multimedia Database Management Systems 15(3):
285-302.
Benz, Ursula C., Peter Hofmann, Gregor Willhauck, Iris
Lingenfelder, Markus Heynen. 2004. Multi-resolution, object-
oriented fuzzy analysis of remote sensing data for GIS-ready
information. ISPRS Journal of Photogrammetry & Remote
Sensing. 58: 239-258.
Bian, Ling. 2007. Object-Oriented Representation of
Environmental Phenomena: Is Everything Best Represented as an
Object? Annals of the Association of American Geographers.
97(2)267-281.
Borsotti, M., P. Campadelli, R. Schettini. 1998. Quantitative
evaluation of color image segmentation results. Pattern
Recognition Letters. 19: 741-747
Chabrier, Sebastien, Bruno Emile, Christophe Rosenberger,
Helene Laurent. 2006. Unsupervised Performance Evaluation of
Image Segmentation. EURAS1P Journal on Applied Signal
Processing. (2006): 1-12
Congalton, Russell G. 1991. A Review of Assessing the
Accuracy of Classifications of Remotely Sensed Data. Remote
Sensing of Environmment. 37: 35-46
Gong, P., and P.J. Howarth, 1990. Land-cover to land-use
conversion: A knowledge-based approach, ACSM-ASPRS Annual
Convention Proceedings, 18-23 March, Denver, Colorado, 4:pp.
447-456.
Holt, Ashley, E.Y.W Seto, Q. Yu, T. Rivard, P. Gong. Accepeted.
Optimization of segmentation parameters for object-oriented
target detection in remotely sensed imagery. Photogrammetric
Endgineering & Remote Sensing.
Krolupper, Filip and Jan Flusser. 2007. Polygonal shape
description for recognition of partially occluded objects. Pattern
Recognition Letters. 28: 1002-1011
Lee, E. T. 1974. The Shape-Oriented Dissimilarity of Polygons
and its Application to The Classification of Chromosome Images.
Pattern Recognition. 6: 47-60
Levine, M.D. and A.M. Nazif. 1982. “An experimental rule
based system for testing low level segmentation strategies.” In
Multicomputers and Image Processing: Algorithms and Programs.
K. Preston and L. Uhr, Eds. New York: Academic. Pp. 149-160.