CMRT09: Object Extraction for 3D City Models, Road Databases and Traffic Monitoring - Concepts, Algorithms, and Evaluation
Sample Distance (GSD) of all input data is 0.2 m. In Toulouse,
these images are Pléiades tri-stereoscopic satellite images. The
GSD of all input data is 0.5 m. Lastly, the DSM used in Lyngby
was derived from first pulse LIDAR data, and the digital
orthophoto was generated from a scanned aerial image, both
with a GSD of 1 m. For the three test areas, up-to-date vector
databases representing the 2D outlines of buildings were
available. They served as a reference in the test. In order to
achieve an objective evaluation, the outdated databases were
simulated by manually adding or removing buildings Thus, 107
changes (out of 1300 buildings in the scene) were simulated in
Marseille (89 new and 18 demolished buildings); 40 (out of
200) in Toulouse (23 new, 17 demolished) and 50 (out of 500)
in Lyngby (29 new, 21 demolished). The outdated databases
were converted to binary building masks having the same GSD
as the input data and then distributed to the participants along
with input data.
Each group participating in the test was asked to deliver a
change map in which each building of the vector database is
labelled either as unchanged, demolished or new. Because the
methods have been developed in different contexts, their
designs noticeably differ, for instance regarding the definitions
of the classes considered in the final change map - e.g. four
classes for (Champion, 2007) and six classes for (Rottensteiner,
2008) - and the format of the input data - e.g. vector for
(Champion, 2007) and raster for (Matikainen et al., 2007). As a
work-around, it was decided to use the building label image
representing the updated version of the building map (cf.
Section 3) for the evaluation of those methods that do not
deliver the required change map in the way described above.
Only the method by (Champion, 2007) delivered such a change
map, which was also directly used in the evaluation.
In order to evaluate the results achieved by the four algorithms,
they are compared to the reference database, and the
completeness and the correctness of the results (Heipke et al.,
1997) are derived as quality measures:
TP
Complétenos =
TF + FN
. TP
Correctness =
TP + FP
In Equation 1, TP, FP, and FN are the numbers of True
Positives, False Positives, and False Negatives, respectively.
They refer to the update status of the vector objects in the
automatically-generated change map, compared to their real
update status given by the reference. In the case where the final
change map is directly used for the evaluation, i.e. with
(Champion, 2007), a TP is an object of the database reported as
changed (demolished or new) that is actually changed in the
reference. A FP is an object reported as changed by the
algorithm that has not changed in the reference. A FN is an
object that was reported as unchanged by the algorithm, but is
changed in the reference. In the three other cases, where a
building label image representing the updated map is used for
the evaluation, the rules for defining an entity as a TP, a FP, or
a FN had to be adapted. In these cases, any unchanged building
in the reference database is considered a 77V if a predefined
percentage (T h ) of its area is covered with buildings in the new
label image. Otherwise, it is considered a FP, because the
absence of any correspondence in the new label image indicates
a change. A demolished building in the reference database is
considered a TP if the percentage of its area covered by any
building in the new label image is smaller than T h . Otherwise, it
is considered to be a FN, because the fact that it corresponds to
buildings in the new label image indicates that the change has
remained undetected. A new building in the reference is
considered a TP if the cover percentage is greater than T h .
Otherwise, it is considered a FN. The remaining areas in the
new label image that do not match any of the previous cases
correspond to objects wrongly alerted as new by the algorithm
and thus constitute FPs.
The quality measures are presented in the evaluation on a per-
building basis (rather than on a per-pixel basis), as the
effectiveness of a change detection approach is limited by the
number of changed buildings that is missed or over-detected
and not by the area covered by these buildings. As explained in
the Section 4, these quality measures are also computed
separately for each change class.
3. CHANGE DETECTION APPROACHES
The four methods tested in this study are concisely presented,
ordered alphabetically according to the corresponding author.
Champion, 2007: The input of the method is given by a DSM,
CIR orthophotos and the outdated vector database. Optionally,
the original multiple images can also be used. The outcome of
the method is a modified version of the input vector database, in
which demolished and unchanged buildings are labelled and
vector objects assumed to be new are created. The method starts
with the verification of the database, where geometric
primitives extracted from the DSM (2D contours, i.e. height
discontinuities) and, optionally, from multiple images (3D
segments), are collected for each object of the existing database
and matched with primitives derived from it. A similarity score
is then computed for each object and used to achieve a final
decision about acceptance (unchanged) and rejection (changed
or demolished). The second processing stage, i.e. the detection
of new buildings, is based on a Digital Terrain Model (DTM)
automatically derived from the DSM (Champion and Boldo,
2006), a normalised DSM (nDSM), defined as the difference
between the DSM and the DTM, and an above-ground mask,
processed from the nDSM by thresholding. Appropriate
morphological tools are then used to compare this latter mask to
the initial building mask derived from the vector database and a
vegetation mask computed from CIR orthophotos and an image
corresponding to the Normalised Difference Vegetation Index
(NDVI), which results in the extraction of new buildings.
Matikainen et al., 2007: The building detection method of the
Finnish Geodetic Institute (FGI) was originally developed to
use laser scanning data as primary data. In this study, it is
directly applied to the input DSM and CIR orthophotos. A
raster version of the database (for a part of the study area) is
used for training. The method includes three main steps. It starts
with segmentation and a two-step classification of input data
into ground and above-ground, based on a point-based analyisis
followed by an object-based analysis and using the Terrasolid 1
and Definiens 1 2 software systems. This is followed by the
definition of training segments for buildings and trees and the
classification of the above-ground segments into buildings and
trees. This classification is based on predefined attributes and a
classification tree (Breiman et al., 1984). A large number of
1 http://www.terrasolid.fi/. Last visited: 30 June 2009.
2 http://www.defmiens.com/. Last visited: 30 June 2009.
