The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B4. Beijing 2008 
1193 
smoothness and color to the observed D, then minimizing the 
function. 
The evaluation of segmentation relative to a training set is simply 
a quantitative measure of the goodness of polygon matching. It 
does not necessarily imply a good classification result. This is 
particularly true in the event that a classification of primitives can 
be used as a preliminary step to the ultimate assembly of objects 
(see, for instance, Pichel et al. 2006). For example, consider an 
evaluation of segmentation results relative to the ultimate 
classification of an entire vehicle. This discounts the prospect of 
first classifying vehicle parts such as windshield, hood, roof, etc., 
then assembling these parts into complete cars through dissolve 
operations or other adjacency rules. However, the method 
described here could easily be applied to such a scenario through 
the provision of training sets for the individual car parts, then 
evaluating the goodness of match between the segmentation and 
the supplied primitives. These hierarchical relationships between 
objects at different spatial scales could be more easily exploited 
using OverSegmentation and UnderSegmentation. With any 
software that produces nested segmentations at different scales (as 
both ASTRO and eCognition do), the D measure could be 
harnessed to compare predefined object primitives to a wide 
variety of segmentations at different scales. In this way, optimal 
scales for analysis could be identified by comparing the training 
objects to different levels of the hierarchy. 
The advantage of the measures we describe is that a quantitative 
index can be generated relative to any set of training objects of 
interest. The measures will also provide useful diagnostic 
information for the efficacy of the segmentation relative to the 
different object types. This characteristic of D is illustrated by 
Figure 1, in which the performance of the different software is 
shown to be very different when supplied with different kinds of 
training objects. 
In the event that two segmentation results have similar values of D, 
the setup described here can be extended to incorporate additional 
indices. However, the indices should be scaled to [0,1] and 
increase the dimension of S, with the Euclidean norm D calculated 
accordingly. The distribution of D in S is of great interest and 
should be defined in order to determine the significance of 
differences between segmentation results. Simulation studies are 
needed to identify this distribution. 
CONCLUSION 
We have presented and demonstrated measures that facilitate the 
identification of optimal segmentation results relative to a training 
set. We propose that these measures are not only useful for the 
selection of segmentations from an array of choices, but also have 
utility in reporting the overall accuracy of segmentation, again 
relative to the set of supplied training objects. This setup is useful 
in the case where pre-defmed objects are to be located and 
extracted (through a classification algorithm) from an image of 
interest. The objective selection of a segmentation result (i.e. not 
based on “expert opinion,” “visual interpretation” and the like) 
necessitates such an approach. Additionally, the growing supply 
of segmentation software means that inter-comparisons such as 
that presented here could benefit from a set of quantitative, well 
defined measures that communicate the effectiveness of the 
software to find objects of interest. This paper presents an 
approach that provides an initial basis for the consistent 
comparison of segmentations resulting from varying parameters 
and software. 
REFERENCES 
Antani, S., D.J. Lee, L.R. Long, G.R. Thoma. 2004. Evaluation of 
Shape Similarity Measurement Methods for Spine X-ray Images. 
Journal of Visual Communication and Image Representation. 
Special issue: Multimedia Database Management Systems 15(3): 
285-302. 
Benz, Ursula C., Peter Hofmann, Gregor Willhauck, Iris 
Lingenfelder, Markus Heynen. 2004. Multi-resolution, object- 
oriented fuzzy analysis of remote sensing data for GIS-ready 
information. ISPRS Journal of Photogrammetry & Remote 
Sensing. 58: 239-258. 
Bian, Ling. 2007. Object-Oriented Representation of 
Environmental Phenomena: Is Everything Best Represented as an 
Object? Annals of the Association of American Geographers. 
97(2)267-281. 
Borsotti, M., P. Campadelli, R. Schettini. 1998. Quantitative 
evaluation of color image segmentation results. Pattern 
Recognition Letters. 19: 741-747 
Chabrier, Sebastien, Bruno Emile, Christophe Rosenberger, 
Helene Laurent. 2006. Unsupervised Performance Evaluation of 
Image Segmentation. EURAS1P Journal on Applied Signal 
Processing. (2006): 1-12 
Congalton, Russell G. 1991. A Review of Assessing the 
Accuracy of Classifications of Remotely Sensed Data. Remote 
Sensing of Environmment. 37: 35-46 
Gong, P., and P.J. Howarth, 1990. Land-cover to land-use 
conversion: A knowledge-based approach, ACSM-ASPRS Annual 
Convention Proceedings, 18-23 March, Denver, Colorado, 4:pp. 
447-456. 
Holt, Ashley, E.Y.W Seto, Q. Yu, T. Rivard, P. Gong. Accepeted. 
Optimization of segmentation parameters for object-oriented 
target detection in remotely sensed imagery. Photogrammetric 
Endgineering & Remote Sensing. 
Krolupper, Filip and Jan Flusser. 2007. Polygonal shape 
description for recognition of partially occluded objects. Pattern 
Recognition Letters. 28: 1002-1011 
Lee, E. T. 1974. The Shape-Oriented Dissimilarity of Polygons 
and its Application to The Classification of Chromosome Images. 
Pattern Recognition. 6: 47-60 
Levine, M.D. and A.M. Nazif. 1982. “An experimental rule 
based system for testing low level segmentation strategies.” In 
Multicomputers and Image Processing: Algorithms and Programs. 
K. Preston and L. Uhr, Eds. New York: Academic. Pp. 149-160.