International Archives of Photogrammetry and Remote Sensing, Vol. 32, Part 7-4-3 W6, Valladolid, Spain, 3-4 June, 1999 
Fig. 1. Conceptual model of computer vision in remote 
sensing. 
The process of segmentation may imply a change of representa 
tion from the digital image data structure (pixel raster) to a list 
and/or graph data structure. Classification (matching) is then 
performed on the objects represented in the list (graph). The 
process of converting this list of classified objects into a 
thematic map may again involve a change of representation 
from the list data structure back to an image data structure. This 
process of "visualisation" is not explicitly shown in Figure 1. 
One of the main problems in the analysis model of Figure 1 is 
the interdependence of segmentation and classification: As the 
segmentation should produce segments (image objects) with a 
meaning in the real world, classification results may already be 
needed as input for a meaningful segmentation. On the other 
hand, the segmentation results are required for a good 
classification (e.g. using texture and shape parameters). In the 
purely sequential processing scheme of Figure 1, any 
problematic segmentation cannot be remedied in the 
classification step. If e.g. a segment delimited in the 
segmentation process encompasses areas of two different 
landcover types, the subsequent classification process is bound 
to fail. Segmentation and classification therefore have to be 
performed in an interrelated way. One approach is documented 
in Gorte (1998). 
This contribution presents a different strategy for the combined 
segmentation and classification for landcover mapping. 
2. SEGMENTATION 
2.1. Overview of segmentation methods 
Segmentation in general is the process of partitioning an image 
into segments (i.e. sets of adjacent pixels) having a meaning in 
the real world (see e.g. Fu and Mui, 1981, Haralick and 
Shapiro, 1992, Pal and Pal, 1993, for detailed surveys). For 
landcover mapping, the segments should represent different 
landcover categories. The criteria for grouping pixels into 
segments can be: 
• spectral and/or textural homogeneity of the pixels (globally 
or within one segment), 
• spectral and/or textural dissimilarity of the pixels of 
adjacent segments, 
• simple form of the boundaries of the segments, and 
• special thematic knowledge. 
Various methods exist for segmentation, each of them 
emphasising some of the above-mentioned aspects. 
Clustering in feature space can be considered as a simple type 
of segmentation. It is solely based on a global homogeneity 
criterion, not on homogeneity within the segments. Also, it does 
not pay attention to the criterion of dissimilarity of adjacent 
segments as well as to the criterion of simple boundaries. 
Clustering in feature space in most cases produces segments for 
which the sum of squared spectral distances to (global) cluster 
means is a minimum, whereas the sum of squared spectral 
distances to the segment means is not minimised. The main 
advantage of landcover mapping based on spatial clustering is 
in interactive mapping, when the landcover classes are not 
known beforehand. For automated procedures, the classes have 
to be defined in advance anyway. In this case, the only reason to 
perform segmentation by clustering may be the need for shape 
parameters of the segments in classification. 
Among the segmentation techniques working in image space, 
one may distinguish between region-based methods on the one 
hand and edge-based methods on the other hand. Region-based 
methods perform better in the case of noisy images, where it is 
more difficult to detect edges. Typical region-based 
segmentation techniques are split-and merge methods and 
region growing methods. It is also possible to regard region 
growing as a special case of segmentation based on merging. 
Region growing segmentation starts at seed pixels, which form 
the initial segments. At any stage of the process, all pixels 
adjacent to a segment S and not yet assigned to another segment 
are tested for similarity to the segment S. If this similarity is 
high enough, the pixel is allocated to S. The similarity may be 
defined either in terms of the difference between the new pixel 
to be tested and the adjacent pixel of segment S, or in terms of 
the difference between the new pixel and the mean of segment 
S. If there is no new pixel to be added to S, the growing of a 
new segment is started with a new seed pixel. Parameters of this 
region growing method are the method of selecting seed pixels 
and the threshold for defining the similarity (homogeneity) 
criterion. For realising texture homogeneity criteria, texture 
channels produced by filtering (e.g. with a variance filter) can 
be used. 
Edge-based segmentation methods start with the search for 
gradients as places of discontinuities in the image, assuming 
that these are the locations of the borders of the segments. 
Various edge detectors (gradient operators) are available, such 
as the Laplace operator or the Sobel operator. The edges are 
subsequently grouped, and border networks are constructed