The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B4. Beijing 2008 
298 
(up-to-date) building databases were edited manually, by field 
surveying. Out-of-date databases were derived by simulating 
changes to the reference databases by inserting new and 
deleting some of the existing buildings. In Marseille, 107 
changes were simulated (89 new and 18 demolished buildings) 
and 40 changes (23 new and 17 demolished buildings) were 
simulated in Toulouse. The out-of-date databases were 
converted to binary image files (building vs. no building) 
having the same GSD as the input data. These binary building 
masks were distributed to the participants along with the CIR 
orthophotos and the DSMs. 
2.2 Evaluation Procedure 
Each group participating in the test was asked to deliver a 
change map in which each building is labelled either as 
unchanged, demolished or new. However, both the 
representation of the results of change detection and the output 
formats varied considerably between the individual algorithms. 
In addition, the definitions of the classes that are discerned in 
the change detection algorithms are not identical. Whereas the 
algorithm by Champion (2007) exactly matches the test 
requirements, this is not the case for the other two algorithms 
used in this study. In these two cases it was thus decided to use 
a building label image representing the updated building map 
for the evaluation. The evaluation consists of a comparison of 
the outcomes of each algorithm to ground truth (i.e. the initial 
reference database). Two quality measures are computed for the 
evaluation: the completeness, i.e. the percentage of the actual 
changes that are detected by an algorithm, and the correctness, 
i.e. the percentage of the changes detected by an algorithm that 
correspond to real changes (Heipke et al., 1997): 
Completeness = - e 0. 
TP + FN 1 
TP 
Correctness = e |0,ll 
TP + FP L J 
In Equation 1, TP, FP, and FN are the numbers of true positives, 
false positives, and false negatives, respectively. They refer to 
changes in the change map compared to actual changes in the 
reference. Thus, a TP is an entity reported as changed 
{demolished or new) that is actually changed in the reference. A 
FP is an entity reported as changed by an algorithm that has not 
changed in the reference. A FN is an entity that was reported as 
unchanged by an algorithm, but is changed in the reference. 
Finally, an entity reported as unchanged by an algorithm and 
also being unchanged in the reference is a true negative (77V). In 
this context, the entities to compare can be buildings, which 
results in per-building quality measures, or pixels in a rasterised 
version of the change map, which results in per-pixel quality 
measures. In the cases where it was decided to use a building 
label image representing the updated map for the evaluation, the 
rules for classifying an entity as a TP, a FP, a FN, or a 77V had 
to be defined in a slightly different way. Any existing (i.e. 
unchanged) building in the reference database is considered a 
TN if a predefined percentage (T h ) of its area is covered with 
buildings in the new label image. Otherwise, it is considered a 
FP, because it does not have a substantial correspondence in the 
new label image, which thus indicates a change. A demolished 
building in the reference database is considered a TP if the 
percentage of its area covered by any building in the new label 
image is smaller than T h . Otherwise, it is considered to be a FN, 
because the fact that it corresponds to buildings in the new label 
image indicates that the change detection algorithm has not 
found this building to have been demolished. A new building in 
the reference database is considered a TP if the cover 
percentage is greater than T h . Otherwise, it is considered a FN. 
The remaining large areas in the new label image that do not 
match any of the previous cases correspond to objects wrongly 
alerted as new by the algorithm and thus constitute FPs. 
3. CHANGE DETECTION APPROACHES 
3.1 Method 1 - (Matikainen et al., 2007) 
The building detection method of the Finnish Geodetic Institute 
(FGI) was originally developed to use laser scanning data as 
primary data. In this study, it is directly applied to input 
correlation DSM and orthophotos. The method includes the 
following stages: 
1. Pre-classification of DSM height points to separate ground 
points from above-ground points, using (Terrasolid, 2008). 
2. The region-based segmentation of the DSM into 
homogeneous regions and calculation of various attributes 
for each segment, using (Definiens, 2008) 
3. The classification of the segments into ground and above 
ground classes by using the pre-classification. 
4. The definition of training segments for buildings and trees on 
the basis of training data sets. 
5. The construction of a classification tree by using the 
attributes of the training segments (Breiman et al., 1984). 
6. The classification of above-ground segments into buildings 
and trees on the basis of their attributes and the classification 
tree. 
7. A post-processing to correct small, misclassified areas by 
investigating the size and neighbourhood of the areas. 
The output of the method also consists of a building label image 
representing the new state of the database, which is used for 
evaluation in this study. 
3.2 Method 2 - (Rottensteiner, 2007) 
The input data of this method consist of a DSM obtained by 
LIDAR or stereo-matching techniques. A geocoded NDVI 
image, the initial building data base, and a Digital Terrain 
Model (DTM) can optionally be used. If no DTM is available, it 
is derived from the DSM by hierarchic morphologic filtering. If 
the initial database is available, it can be used to introduce a 
bias that favours a classification consistent with the initial data 
base, because in most scenes only a small percentage of 
buildings will actually have changed. The workflow of the 
method consists of three stages. First, a Dempster-Shafer fusion 
process is carried out on a per-pixel basis and results in a 
classification of the input data into one of four predefined 
classes: buildings, trees, grass land, and bare soil. Connected 
components of building pixels are grouped to constitute initial 
building regions. A second Dempster-Shafer fusion process is 
then carried out on a per-region basis to eliminate regions 
corresponding to trees. The third stage of the work flow is the 
actual change detection process, in which the detected buildings 
are compared to the existing map. A very detailed change map 
is generated in this process. The output of the method consists 
of a building label image representing the new state of the data 
base and in change maps describing the change status both on a 
per-pixel and a per-building level (Refer to (Rottensteiner, 2007)