The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B4. Beijing 2008 
299 
for more details). Since the definitions of the classes in the 
change map do not match those required in Section 2.2, the 
building label image is used for evaluation in this study. 
3.3 Method 3 - (Champion, 2007) 
The input data of the method are a DSM, a vegetation mask, 
computed from CIR orthophotos and NDVI index, and a DTM, 
automatically derived from the DSM using the algorithm 
described in (Champion and Boldo, 2006). The workflow 
consists of 2 stages: in a first step, geometric primitives, 
extracted from the DSM (2D contours i.e. height discontinuities) 
or from multiple images (3D segments, computed with 
(Taillandier and Deriche, 2002)), are collected for each building 
and matched with primitives derived from the existing vector 
map. A final decision about acceptance or rejection is then 
achieved per building. In the second step, the DTM is combined 
with the DSM to process an above-ground mask. This mask is 
morphologically compared to the initial building mask (derived 
from the vector database) and the vegetation mask and new 
buildings are extracted. The output of the method consists of a 
change map, in which each building is labelled as unchanged, 
demolished or new. 
4.1.2 (Rottensteiner, 2007) 
The evaluation of this method is illustrated in Fig. 1-b. Overall, 
the changes are detected correctly. Compared to (Matikainen et 
al., 2007), there are only three additional FNs. Five new 
buildings are missed by the algorithm (Fig. 2-c), which is 
caused by errors that occur in the DTM, by complicated 
topographic features (cliffs). In presence of such features, the 
DTM, derived from the DSM by hierarchic morphological 
opening, is less accurate, which predictably limits the extraction 
of new buildings that is partly based on the difference between 
the DSM and DTM. Quantisation effects in the DSM (the 
numerical resolution of height values is restricted to 20 cm) 
prevented the use of surface roughness as an input parameter 
for the Dempster-Shafer fusion process, which might have 
helped to overcome such problems. However, the correctness of 
the system is acceptable, which implies a limited number of 
FPs. Compared to (Matikainen et al., 2007) and (Champion, 
2007), no FPs are generated during the detection of new 
buildings (e.g. Fig. 2-d). FPs are mostly caused by small 
buildings, located in inner yards and in shadow areas: the 
complexity of the urban scene and the relatively poor quality of 
the DSM in shadow areas clearly and significantly deteriorate 
the change detection correctness here. 
4. RESULTS AND DISCUSSION 
The evaluation outputs are summarized in Table 1. The 
completeness and correctness are given for each test area and 
for each approach, both on a per-building basis and on a per- 
pixel basis. Values in bold indicates for which methods the best 
results are achieved. In an optimal system, completeness and 
correctness are equal to 1: in that case, there is no FN (no 
under-detection) and no FP (no overdetection). To be of 
practical interest, i.e. to consider the system effective and 
operational, previous works on change detection (Steinnocher 
and Kressler, 2006), (Mayer et al., 2006) expect a completeness 
rate close to 1 (typically 0.85) and a high correctness rate 
(typically 0.7). These recommendations are true for 
completeness (the new map must be really new) but must be 
modulated for correctness, with respect to the type of change. In 
our opinion, the effectiveness of a system is mostly related to 
the amount of work saved for a human operator. That also 
corresponds to the number of unchanged buildings that are 
correctly detected, because these buildings need no longer be 
inspected. As a consequence, the correctness rate has to be high 
during the verification of the database (i.e. for demolished 
buildings). By contrast, a low correctness rate for new buildings 
is less problematic: without the support of automatic techniques, 
the entire scene has to be examined by a human operator and 
therefore, a system that delivers a set of potential new buildings 
is effective if the true changes are contained in the set and if the 
number of FP s is not overwhelmingly large. 
4.1 Marseille Test Area 
4.1.1 (Matikainen et al., 2007) 
The evaluation of this method is illustrated in Fig. 1-a. The 
method best operates in term of completeness (0.98). Only two 
changes are missed by the algorithm and are related to errors in 
the initial “ground”/“above-ground” classification. Most false 
alarms are caused by one of two problems, namely the 
uncertainty of the classification in shadow areas (e.g. Fig. 2-a) 
and errors in the DSM between buildings that cause a 
misclassification of street areas as new buildings (e.g. Fig. 2-b). 
Method 
Completeness 
Correctness 
per 
building 
per 
pixel 
per 
building 
per 
pixel 
Marseille Test Area 
Matikainen 
0.98 
0.99 
0.54 
0.79 
Rottensteiner 
0.95 
0.98 
0.58 
0.83 
Champion 
0.94 
0.94 
0.45 
0.75 
Toulouse Test Area 
Rottensteiner 
0.85 
0.90 
0.49 
0.53 
Champion 
0.80 
0.95 
0.55 
0.85 
Table 1. Completeness and Correctness achieved by the three 
algorithms for both data sets. 
4.1.3 (Champion, 2007) 
The evaluation of this method is illustrated in Fig. 1-c. Five 
FNs appear with the method. Two of them (in the north-western 
comer of the scene - Fig. 2-e) occur during the first stage of the 
algorithm. They are caused by extracted primitives that are 
wrongly used to validate demolished buildings. Remaining FNs 
are related to inaccuracies in the processed DTM and occur 
where topography is particularly difficult. Here again, the 
overestimation of the terrain height in the DTM prevents the 
complete extraction of new buildings. Regarding FPs, those 
that occur in the first stage of the algorithm are related to the 
complexity of the scene: the extraction of pertinent primitives 
for inner and lower buildings is more difficult and makes the 
verification more uncertain. Most FPs that occur in the second 
stage are related to building-like structures (walls that are 
wrongly considered to be new buildings), errors in the 
vegetation mask (omitted trees) and the same inaccuracies in 
the correlation DSM (large overestimated areas in narrow 
streets) that caused errors in the classification of (Matikainen et 
al., 2007) (Fig. 2-f). 
4.1.4 Remark concerning the aerial context 
In the context of aerial imagery, there is not a visible 
predominance of an approach over another one. The three of