International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol XXXV, Part B2. Istanbul 2004
computation by means of matching. Depending on the ap-
plication, another possibility is to consider a valid match
to be achieved as long as ground truth data and extracted
object information have any overlap.
For approaches relevant for DPW, testing must always be
done against real world data, and not against simulated
data. The question if ground truth data should be gathered
from the 3D reality, or if ground truth data should be man-
ually digitized from the image used also for the automated
interpretation is from our point easy to answer: If the goal
is to evaluate the whole production chain, the former is ap-
propriate. In many cases one just wants to know how much
worse than a human the automated system is. Then, bench-
marking against given manually digitized ground truth data
is the way to go. To avoid a bias from an operator, one can
match against the results of more than one operator such as
in (Martin et al., 2004).
Together with Emmanuel Baltsavias of ETH Zurich
we have recently set up a test on "Automated ex-
traction, refinement, and update of road databases
from imagery and other data” (http://www.bauv.unibw
muenchen.de/institute/inst10/eurosdr) under the umbrella
of EuroSDR (European spatial data research; formerly
known as OEEPE). On one hand, we want to learn the data
specification needs of important data producers, mainly na-
tional mapping agencies (NMA) and their customers. On
the other hand, existing semi- and fully-automated systems
for road extraction will be evaluated based on high quality
image data against given, manually digitized ground truth
data.
4 USER INTERACTION
To limit the scope, we do not deal with multi-spectral
classification, which is well understood and for which
powerful commercial products such as ERDAS IMAGINE
from Leica Geosystems or ENVI from Research Systems
Inc. are available. Closer to our intentions is eCognition of
Definiens GmbH as it deals with objects, not pixels. Be-
cause it aims more at similar applications as the former two
products assuming larger ground pixel sizes than DPW, we
will not treat it here either.
For general purpose DPW as well as GIS, automated func-
tionality for object extraction is very limited. According to
(Baltsavias, 2004), the only more widely known systems
actually useful for practice because offering the most au-
tomation are the systems InJect of INPHO GmbH (Gülch
et al., 1999) and CC-Modeler of CyberCity AG (Grün and
Wang, 2001). Though, both are limited with this respect,
that they are dedicated to building extraction only.
(Baltsavias, 2004) points out, that it is clear why full au-
tomation is not feasible today, but asks “why are important-
for-the-practice semi-automated approaches so rare?” We
will give some ideas why this is the case, but we will also
point on ways how to change it.
Basically, as pointed out above, automated object extrac-
tion is extremely difficult and therefore error-prone. Only
418
a limited number of the approaches developed over the
last two decades has been developed so far that they work
for a larger number of data sets and are ready for testing
(cf. Section 3). But even if there was a larger number
of approaches with reasonable performance in real world
tests, there is another issue which makes the preparation of
an approach for practice even more problematic than the
usual 1 : 10: 100 relation between proof of concept : sta-
ble prototype : product level: This is the dependence of the
user interaction on the performance level and the strategy
of object extraction in the system.
This means, that to build a highly effective interactive sys-
tem, the interaction needs to be tailored to a fixed level of
automation. If the level of automation improves, it is not
too unlikely, that the interaction of the system will have
to be considerably different, implying larger changes for
the software, but also possibly for the production chains of
the customers. Seen the other way around more positively,
(Baltsavias, 2004) recommends to design the control in-
cluding human interaction to build systems that are useful
for practice.
A reaction to the difficulties of fully-automated object ex-
traction is a restriction to problems where the computer
directly assists the user in real-time. This is the case for
InJect but only partly for CC-Modeler. For roads, this
idea has been promoted early, e.g., by (Grün and Li, 1994,
Heipke et al., 1994), but nowadays it seems that roads are,
e.g., in open rural areas, so easy to extract, that it is a good
idea to do it fully-automated. On the other hand, in urban
areas, but also in shadows or at complex crossings, they are
so difficult to extract, that only fully-automated non-real-
time-processing can deal with them today.
For practically relevant systems, we believe, that the hu-
man has to be in the loop. We also think that in many cases
it is beneficial to use one or two automated off-line pro-
cesses, probably preceded or interrupted, but in any case
followed by manual interaction. The generation of work-
flows defining the offline-phases, but also very importantly
the information to be given to the automated procedure by
a user interaction preceding it, is essential for the overall
performance.
There are several steps needed to develop a system use-
ful for practice, to be embedded into a DPW. The basis
are thorough theoretical understanding and testing. For an
efficient user interaction, the key is an appropriate trade-
off between completeness and correctness / reliability. It
is usually more costly in terms of user interaction time to
eliminate complex failures. Therefore, it is a good idea, to
use as a basis for human interaction a version where the
completeness is still high, but where very few complicated
errors, especially in terms of topology occur.
A related issue is self-diagnosis. In this context it is not the
same as in statistical modeling as it makes use of additional
knowledge about the strengths and weaknesses of human
interaction. For a semi-automated system it is extremely
important that the correct objects (green) are actually cor-
rect with a very high probability, so that they do not have to
Intern
be ch
tion
opera
result
more
ally,
be, t!
with i
An el
time !
auton
fort t«
of au
tomat
Final
into |
cific
streng
of (s
latter
tegrat
user I
and, «
Becai
on in
envir
ject e
in co
NMA
tility
opers
tion ¢
large
acade
The t
tion o
roads.
high 1
agery,
scann
tendei
agery
is a de
tional
etatioi
That «
trast t
Graph
and v
paper
where
and m
resolu
