CMRT09: Object Extraction for 3D City Models, Road Databases and Traffic Monitoring - Concepts, Algorithms, and Evaluation
(a) Matikainen et al. (2007) (b) Rottensteiner (2008) (c) Olsen and Knudsen (2005)
Figure 1. Evaluation of change detection in Lyngby, for (a), (b) and (c) Green: TP; red: FN; orange: FP; blue: TN.
Unchanged
Demolished
New
(Champion, 2007)
Completeness [%]
93.5
88.9
95.2
Correctness [%]
99.8
18.4
63.5
(Matikainen et al., 2007)
Completeness [%]
94.7
100
97.6
Correctness [%]
100
23.7
75.6
(Rottensteiner, 2008)
Completeness [%]
94.1
100
94.0
Correctness [%]
100
22.0
96.3
Table 2. Completeness and correctness for the Marseille test
area, depending on the update status.
| Unchanged \ Demolished \ New
(Matikainen et al., 2007)
Completeness [%1
81.7
100
91.8
Correctness \%]
100
22.6
100
(Olsen and Knudsen, 2005)
Completeness [%1
87.8
100
93.9
Correctness [%]
100
30.4
82.1
(Rottensteiner, 2008)
Completeness [%]
95.9
100
87.8
Correctness [%)
100
56.8
91.8
Table 3. Completeness and correctness for the Lyngby test
area, depending on the update status.
Unchanged
Demolished
New
(Champion, 2007)
Completeness f%j
82.8
100
75.0
Correctness \%]
100
42.9
65.2
(Rottensteiner, 2008)
Completeness [%]
80.2
86.7
82.6
Correctness [%]
97.9
36.1
59.4
Table 4. Completeness and correctness for the Toulouse test
area, depending on the update status.
ranges from 18.4% with (Champion, 2007) to 23.7% with
(Matikainen et al., 2007). The situation is a bit better for new
buildings, with a correctness rate larger than 63% for all the
methods and even rising to 96.8% with (Rottensteiner, 2008). In
spite of such limitations, all the methods presented here are very
efficient in classifying unchanged buildings, for which the
completeness rates are higher than 93%, which indicates that a
considerable amount of manual work is saved and also
demonstrates the economical efficiency of these approaches in
the context of aerial imagery.
Analyzing Table 3 leads to similar conclusions for the LIDAR
context. The correctness rate for the reported demolished
buildings are again poor and only (Rottensteiner, 2008)
achieves less than 50% false positives. However, the methods
are very effective in detecting demolished buildings and achieve
a completeness rate of 100% for this class. Compared to the
outcomes obtained in Marseille, the main difference concerns
the new buildings, which appear to be more difficult to extract.
Thus, between 6.1% (Olsen and Knudsen, 2006) and 12.2%
(Rottensteiner, 2008) of the new buildings are missed. If these
percentages of missed new buildings can be tolerated, our tests
indicate that LIDAR offers a high economical effectiveness and
thus may be a viable basis for a future application. If these error
rates for new buildings are unacceptable, manual post-process is
required to find the missed buildings, at the expense of a lower
economical efficiency.
The situation is not quite as good with the satellite context
(Table 4). The method by (Champion, 2007) is very effective in
detecting demolished buildings (100%), but this is achieved at
the expense of a low correctness rate (42.9%). The same
analysis can be carried out with (Rottensteiner, 2008), but this
method even misses quite a few demolished buildings. It has to
be noted that, even though the completeness rates for
unchanged buildings achieved by both methods are relatively
low compared to those obtained in the Marseille and Lyngby
test areas, they also indicate that even under challenging
circumstances, 80% of unchanged buildings need not be
investigated by an operator. The main limitation appears to be
the detection of new buildings. As illustrated for an example in
Figures 3e and 3f, 17.4% and 25% of new buildings are missed
with (Rottensteiner, 2008) and (Champion, 2007) respectively,
which is clearly not sufficient to provide a full update of the
database and requires a manual intervention in order to find the
remaining new buildings.
In order to obtain deeper insights into the reasons for failure, in
the subsequent sections we will focus our analysis on some
factors that affect the change detection performance.
4.2 Impact of the Size of a Change
To analyse the performance of change detection as a function of
the change size, we compute the completeness and correctness
rates depending on this factor. For that purpose, new and
demolished buildings are placed into bins representing classes
148