The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B4. Beijing 2008
286
2.2. Evaluation
The need for evaluation methods has been published for
different vision applications (Unnikrishnan et al., 2007),(Wust
et al., 1998),(Jiang et al., 2005),(Zhang & Gerbrands, 1994).
Two different approaches exist. On the one hand are
dissimilarity measures between segmentation result and ground
truth (Borsotti et al., 1998),(Cardoso & Corte-Real,
2005) ,(Jiang et al., 2006),(Zhang, 1996). On the other hand
statistics without a priori knowledge are used (Chabrier et al.,
2006) ,(Rosenberger, 2006),(Roman-Roldan et al., 2001),
measuring intra- and inter-class disparity and uniformity of the
obtained regions. In our application we use the ground truth
approach, because the spatial accuracy and topological
correctness of visual delineation is still seen as very reliable
compared to computed segmentation boundaries.
2.3. Ground truth
Within NFI, manually measured features like the forest
boundary line are obtained by stereoscopic interpretation and
represent the ground truth. This manual delineation defines the
best available forest boundary depending on forest parameters
and visual clues and is managed by forest experts. Due to the
specific strict rules of the NFI forest definition, the delineation
is only available in linear pieces. Therefore result comparison
methods which rely on topological comparison of segmentation
results cannot be applied (Jiang et al., 2006),(Monteiro &
Campilho, 2006). Additional manual interpretation to achieve
closed interpretation regions can be done for few examples, but
the required effort for the whole NFI image database would be
excessive.
3. COMPARISON METHOD
We propose therefore a new optimization method, which is
based on spatial constraints between segmentation regions and
ground truth using city-block metric as distance criteria (Figure
1).
o
Groundtruth
Figure 1. Difference area in 4 directions (distinct colors for each
city block direction)
segmentation line, the error area will not reach the image
boundary. The concept of "hidden" segmentation line allows to
handle over-segmentation as additional error area, otherwise
over-segmentations would be ignored. With this metric spatial
accuracy and topological correctness can be handled with the
same approach.
With systematic variations of the parameter values, the
corresponding error areas represent the spatial deviation from
the perfect segmentation. This approach allows to optimize
parameters in a systematic manner but is also limited to a local
neighborhood of the whole parameter space.
The parameter range of values needs to be estimated by
experience or well-known values from publications. Expert
knowledge and experience is still needed to start with
meaningful seed values and ranges.
Despite this limitations, the proposed city block optimization
(CBO) is robust enough to select optimized parameter values
with a systematic and reproducible approach. Mainly for
application targets with limited ground truth data, the presented
spatial constraints allow to improve segmentation parameters
with a reasonable effort. Due to the high computational burden
for advanced segmentation algorithms like JSEG, an exhaustive
randomized evaluation of the parameter space is not applicable.
4. EXPERIMENTAL RESULTS
JSEG has two parameters of strong influence on the
segmentation results. The first one controls the color
quantization window and the other parameter controls the
thresholding of the region merging. Because the final region
merging has the strongest impact on the delineation of forest
areas, we focus on the optimization of threshold parameter th,
but vary the window size qw also.
The area between each segmentation region and the nearest
reference line will be calculated in the orthogonal directions +X
(West), -X (East), +Y(North) and -Y(South). The 4 areas
represent the overall spatial error difference, which needs to be
minimized. A single error area can be calculated as follows
(n=image columns/lines)
Figures 2-4 and 5-7 show two typical examples for forest
boundaries and both images have been processed with a
parameter range for threshold th from 0.25 until 0.95 (with an
increment of 0.05) and quantization windows qw ranging from
3x3, 5x5 and 7x7. Figure 2 and 5 show two ground truth images
representing a (simplified) manual forest delineation.
diff(x,y) = Zlxi-yil, i=l..n
If there is no segmentation line found between ground truth and
image boundary along the city block metric, the nearer image
boundary is treated as nearest "hidden" segmentation line
(results in rectangle-like shapes as in Figure 1). If there is a real
Figure 4 shows 5 distinct variations out of 45 processed
segmentations for a forest edge in the shape of a cove. The
ground truth shows that no gaps are defined as forest boundary
delineation. The smallest CBO-value of 26.1 (threshold=0.85)
confirms visually the closest correspondence with the ground
truth (less color - less error).