The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B4. Beijing 2008
299
for more details). Since the definitions of the classes in the
change map do not match those required in Section 2.2, the
building label image is used for evaluation in this study.
3.3 Method 3 - (Champion, 2007)
The input data of the method are a DSM, a vegetation mask,
computed from CIR orthophotos and NDVI index, and a DTM,
automatically derived from the DSM using the algorithm
described in (Champion and Boldo, 2006). The workflow
consists of 2 stages: in a first step, geometric primitives,
extracted from the DSM (2D contours i.e. height discontinuities)
or from multiple images (3D segments, computed with
(Taillandier and Deriche, 2002)), are collected for each building
and matched with primitives derived from the existing vector
map. A final decision about acceptance or rejection is then
achieved per building. In the second step, the DTM is combined
with the DSM to process an above-ground mask. This mask is
morphologically compared to the initial building mask (derived
from the vector database) and the vegetation mask and new
buildings are extracted. The output of the method consists of a
change map, in which each building is labelled as unchanged,
demolished or new.
4.1.2 (Rottensteiner, 2007)
The evaluation of this method is illustrated in Fig. 1-b. Overall,
the changes are detected correctly. Compared to (Matikainen et
al., 2007), there are only three additional FNs. Five new
buildings are missed by the algorithm (Fig. 2-c), which is
caused by errors that occur in the DTM, by complicated
topographic features (cliffs). In presence of such features, the
DTM, derived from the DSM by hierarchic morphological
opening, is less accurate, which predictably limits the extraction
of new buildings that is partly based on the difference between
the DSM and DTM. Quantisation effects in the DSM (the
numerical resolution of height values is restricted to 20 cm)
prevented the use of surface roughness as an input parameter
for the Dempster-Shafer fusion process, which might have
helped to overcome such problems. However, the correctness of
the system is acceptable, which implies a limited number of
FPs. Compared to (Matikainen et al., 2007) and (Champion,
2007), no FPs are generated during the detection of new
buildings (e.g. Fig. 2-d). FPs are mostly caused by small
buildings, located in inner yards and in shadow areas: the
complexity of the urban scene and the relatively poor quality of
the DSM in shadow areas clearly and significantly deteriorate
the change detection correctness here.
4. RESULTS AND DISCUSSION
The evaluation outputs are summarized in Table 1. The
completeness and correctness are given for each test area and
for each approach, both on a per-building basis and on a per-
pixel basis. Values in bold indicates for which methods the best
results are achieved. In an optimal system, completeness and
correctness are equal to 1: in that case, there is no FN (no
under-detection) and no FP (no overdetection). To be of
practical interest, i.e. to consider the system effective and
operational, previous works on change detection (Steinnocher
and Kressler, 2006), (Mayer et al., 2006) expect a completeness
rate close to 1 (typically 0.85) and a high correctness rate
(typically 0.7). These recommendations are true for
completeness (the new map must be really new) but must be
modulated for correctness, with respect to the type of change. In
our opinion, the effectiveness of a system is mostly related to
the amount of work saved for a human operator. That also
corresponds to the number of unchanged buildings that are
correctly detected, because these buildings need no longer be
inspected. As a consequence, the correctness rate has to be high
during the verification of the database (i.e. for demolished
buildings). By contrast, a low correctness rate for new buildings
is less problematic: without the support of automatic techniques,
the entire scene has to be examined by a human operator and
therefore, a system that delivers a set of potential new buildings
is effective if the true changes are contained in the set and if the
number of FP s is not overwhelmingly large.
4.1 Marseille Test Area
4.1.1 (Matikainen et al., 2007)
The evaluation of this method is illustrated in Fig. 1-a. The
method best operates in term of completeness (0.98). Only two
changes are missed by the algorithm and are related to errors in
the initial “ground”/“above-ground” classification. Most false
alarms are caused by one of two problems, namely the
uncertainty of the classification in shadow areas (e.g. Fig. 2-a)
and errors in the DSM between buildings that cause a
misclassification of street areas as new buildings (e.g. Fig. 2-b).
Method
Completeness
Correctness
per
building
per
pixel
per
building
per
pixel
Marseille Test Area
Matikainen
0.98
0.99
0.54
0.79
Rottensteiner
0.95
0.98
0.58
0.83
Champion
0.94
0.94
0.45
0.75
Toulouse Test Area
Rottensteiner
0.85
0.90
0.49
0.53
Champion
0.80
0.95
0.55
0.85
Table 1. Completeness and Correctness achieved by the three
algorithms for both data sets.
4.1.3 (Champion, 2007)
The evaluation of this method is illustrated in Fig. 1-c. Five
FNs appear with the method. Two of them (in the north-western
comer of the scene - Fig. 2-e) occur during the first stage of the
algorithm. They are caused by extracted primitives that are
wrongly used to validate demolished buildings. Remaining FNs
are related to inaccuracies in the processed DTM and occur
where topography is particularly difficult. Here again, the
overestimation of the terrain height in the DTM prevents the
complete extraction of new buildings. Regarding FPs, those
that occur in the first stage of the algorithm are related to the
complexity of the scene: the extraction of pertinent primitives
for inner and lower buildings is more difficult and makes the
verification more uncertain. Most FPs that occur in the second
stage are related to building-like structures (walls that are
wrongly considered to be new buildings), errors in the
vegetation mask (omitted trees) and the same inaccuracies in
the correlation DSM (large overestimated areas in narrow
streets) that caused errors in the classification of (Matikainen et
al., 2007) (Fig. 2-f).
4.1.4 Remark concerning the aerial context
In the context of aerial imagery, there is not a visible
predominance of an approach over another one. The three of