The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B4. Beijing 2008 
286 
2.2. Evaluation 
The need for evaluation methods has been published for 
different vision applications (Unnikrishnan et al., 2007),(Wust 
et al., 1998),(Jiang et al., 2005),(Zhang & Gerbrands, 1994). 
Two different approaches exist. On the one hand are 
dissimilarity measures between segmentation result and ground 
truth (Borsotti et al., 1998),(Cardoso & Corte-Real, 
2005) ,(Jiang et al., 2006),(Zhang, 1996). On the other hand 
statistics without a priori knowledge are used (Chabrier et al., 
2006) ,(Rosenberger, 2006),(Roman-Roldan et al., 2001), 
measuring intra- and inter-class disparity and uniformity of the 
obtained regions. In our application we use the ground truth 
approach, because the spatial accuracy and topological 
correctness of visual delineation is still seen as very reliable 
compared to computed segmentation boundaries. 
2.3. Ground truth 
Within NFI, manually measured features like the forest 
boundary line are obtained by stereoscopic interpretation and 
represent the ground truth. This manual delineation defines the 
best available forest boundary depending on forest parameters 
and visual clues and is managed by forest experts. Due to the 
specific strict rules of the NFI forest definition, the delineation 
is only available in linear pieces. Therefore result comparison 
methods which rely on topological comparison of segmentation 
results cannot be applied (Jiang et al., 2006),(Monteiro & 
Campilho, 2006). Additional manual interpretation to achieve 
closed interpretation regions can be done for few examples, but 
the required effort for the whole NFI image database would be 
excessive. 
3. COMPARISON METHOD 
We propose therefore a new optimization method, which is 
based on spatial constraints between segmentation regions and 
ground truth using city-block metric as distance criteria (Figure 
1). 
o 
Groundtruth 
Figure 1. Difference area in 4 directions (distinct colors for each 
city block direction) 
segmentation line, the error area will not reach the image 
boundary. The concept of "hidden" segmentation line allows to 
handle over-segmentation as additional error area, otherwise 
over-segmentations would be ignored. With this metric spatial 
accuracy and topological correctness can be handled with the 
same approach. 
With systematic variations of the parameter values, the 
corresponding error areas represent the spatial deviation from 
the perfect segmentation. This approach allows to optimize 
parameters in a systematic manner but is also limited to a local 
neighborhood of the whole parameter space. 
The parameter range of values needs to be estimated by 
experience or well-known values from publications. Expert 
knowledge and experience is still needed to start with 
meaningful seed values and ranges. 
Despite this limitations, the proposed city block optimization 
(CBO) is robust enough to select optimized parameter values 
with a systematic and reproducible approach. Mainly for 
application targets with limited ground truth data, the presented 
spatial constraints allow to improve segmentation parameters 
with a reasonable effort. Due to the high computational burden 
for advanced segmentation algorithms like JSEG, an exhaustive 
randomized evaluation of the parameter space is not applicable. 
4. EXPERIMENTAL RESULTS 
JSEG has two parameters of strong influence on the 
segmentation results. The first one controls the color 
quantization window and the other parameter controls the 
thresholding of the region merging. Because the final region 
merging has the strongest impact on the delineation of forest 
areas, we focus on the optimization of threshold parameter th, 
but vary the window size qw also. 
The area between each segmentation region and the nearest 
reference line will be calculated in the orthogonal directions +X 
(West), -X (East), +Y(North) and -Y(South). The 4 areas 
represent the overall spatial error difference, which needs to be 
minimized. A single error area can be calculated as follows 
(n=image columns/lines) 
Figures 2-4 and 5-7 show two typical examples for forest 
boundaries and both images have been processed with a 
parameter range for threshold th from 0.25 until 0.95 (with an 
increment of 0.05) and quantization windows qw ranging from 
3x3, 5x5 and 7x7. Figure 2 and 5 show two ground truth images 
representing a (simplified) manual forest delineation. 
diff(x,y) = Zlxi-yil, i=l..n 
If there is no segmentation line found between ground truth and 
image boundary along the city block metric, the nearer image 
boundary is treated as nearest "hidden" segmentation line 
(results in rectangle-like shapes as in Figure 1). If there is a real 
Figure 4 shows 5 distinct variations out of 45 processed 
segmentations for a forest edge in the shape of a cove. The 
ground truth shows that no gaps are defined as forest boundary 
delineation. The smallest CBO-value of 26.1 (threshold=0.85) 
confirms visually the closest correspondence with the ground 
truth (less color - less error).