1229
UAV-BASED AUGMENTED MONITORING - REAL-TIME GEOREFERENCING
AND INTEGRATION OF VIDEO IMAGERY WITH VIRTUAL GLOBES
H. Eugster and S. Nebiker
Institute of Geomatics Engineering
University of Applied Sciences Northwestern Switzerland (FHNW)
(hannes.eugster, stephan.nebiker)@fhnw.ch
ThS-23 UAV for Mapping
KEY WORDS: Mobile mapping system, Sensor orientation, GPS-assisted photogrammetry, Absolute Orientation,
Environmental monitoring
ABSTRACT:
Over the last few years low-cost micro and mini UAV systems equipped with light-weight geosensors such as video cameras have
started to appear on the market. These UAV systems are mostly instrumented with low quality INS/GPS sensors for position and
attitude control. Over the same time period web-based interactive 3D geoinformation solutions have evolved into virtual globe
technologies which have had a tremendous impact on the geospatial industry within just a few years. Up to now, UAVs and virtual
globes have rarely been combined. However, the real-time integration of live or recorded video streams from airborne platforms into
virtual globes opens up a wide range of new applications. Real-time infrastructure surveillance, forest fire monitoring, traffic
monitoring or rapid to real-time mapping are only a few examples. Today, the use of virtual globes for mission critical applications
is frequently hampered by the fact that the underlying base imagery data is often outdated and does no longer reflect the current
situation. Hence the real-time integration of image data based on video or still frame cameras into virtual globes bears a great
potential for dramatically increasing the benefits of 3D geoinformation solutions. This paper presents an approach to complement
virtual globes with up-to-date or even live geospatial content captured from mini- or micro UAV platforms.
1. INTRODUCTION
1.1 Motivation
Today a large number of geosensors and geosensor systems are
operational. The available spectrum of geosensors ranges from
simple static sensors which capture arbitrary physical
phenomena such as temperature or humidity to complex sensors
mounted on highly dynamic platforms like satellites or UAV
(unmanned aerial vehicle) systems. So far, this real-time sensor
data cannot yet be exploited in 3D geoinformation services.
However, if the available geosensor data are combined with 3D
geoinformation services, new application scenarios can be
realized. For such applications, mini- or micro UAV systems
are very promising cost-efficient platforms for capturing real
time close-range imagery. Possible application scenarios range
from decision support after natural disasters, such as
earthquakes, to the virtual piloting of UAV systems.
Our investigations differ from mini and micro UAV based
mapping applications in photogrammetry as for example
presented in (Eisenbeiss 2006) or (Annen and Nebiker 2007).
The focus of our research is placed on real-time data geo
registration and geodata integration into virtual globes.
Especially, we have to continuously geo-register and process
the available video data streams and low-quality non-metric
video cameras use for data capturing.
1.2 Goals and contents of the paper
This paper presents a prototype system solution, which allows
for a real-time integration of a video stream captured with a
mini or micro UAV system into a virtual globe technology.
Two different integration approaches will be presented, one
referred to as augmented monitoring, the other as virtual
monitoring. These two approaches will be presented in detail.
Additionally, the augmented monitoring approach will enable
the real-time mapping of arbitrary geo-objects and sharing this
information among numerous virtual globe clients with the use
of a collaboration framework. For the realisation of such a
solution the video geo-registration process based on low quality
and low cost IMU/GPS sensors used in mini or micro UAV
systems with an adequate geo-referencing accuracy is a major
challenge. Especially the system calibration of the camera
misalignment as well as the continuously changing systematic
attitude and position error of the MEMS based IMU and
navigation grade GPS receivers have a great influence on the
achievable geo-registration accuracy.
The paper commences with a technical overview of mini and
micro UAV systems, with a focus on their suitability for
mapping applications. Additionally, the UAV platform used for
these investigations is presented. Next, the state of the art of
virtual globe technologies is reviewed and put into perspective
with our own virtual globe technology i3D. The paper then
describes the entire real time data processing chain and the
developed software components. This workflow is subdivided
into the following steps. First, the flight data and video stream
transmission and synchronisation process is introduced.
Subsequently, the geo-referencing approach for geo-registering
the captured video streams is discussed. This is followed by a
description of the different strategies for integrating video
streams into the i3D virtual globe technology. The final section
of the paper presents results and experiences from a test case
project and provides an overview of new applications.
Furthermore, the mapping accuracies obtained from the above