1229 
UAV-BASED AUGMENTED MONITORING - REAL-TIME GEOREFERENCING 
AND INTEGRATION OF VIDEO IMAGERY WITH VIRTUAL GLOBES 
H. Eugster and S. Nebiker 
Institute of Geomatics Engineering 
University of Applied Sciences Northwestern Switzerland (FHNW) 
(hannes.eugster, stephan.nebiker)@fhnw.ch 
ThS-23 UAV for Mapping 
KEY WORDS: Mobile mapping system, Sensor orientation, GPS-assisted photogrammetry, Absolute Orientation, 
Environmental monitoring 
ABSTRACT: 
Over the last few years low-cost micro and mini UAV systems equipped with light-weight geosensors such as video cameras have 
started to appear on the market. These UAV systems are mostly instrumented with low quality INS/GPS sensors for position and 
attitude control. Over the same time period web-based interactive 3D geoinformation solutions have evolved into virtual globe 
technologies which have had a tremendous impact on the geospatial industry within just a few years. Up to now, UAVs and virtual 
globes have rarely been combined. However, the real-time integration of live or recorded video streams from airborne platforms into 
virtual globes opens up a wide range of new applications. Real-time infrastructure surveillance, forest fire monitoring, traffic 
monitoring or rapid to real-time mapping are only a few examples. Today, the use of virtual globes for mission critical applications 
is frequently hampered by the fact that the underlying base imagery data is often outdated and does no longer reflect the current 
situation. Hence the real-time integration of image data based on video or still frame cameras into virtual globes bears a great 
potential for dramatically increasing the benefits of 3D geoinformation solutions. This paper presents an approach to complement 
virtual globes with up-to-date or even live geospatial content captured from mini- or micro UAV platforms. 
1. INTRODUCTION 
1.1 Motivation 
Today a large number of geosensors and geosensor systems are 
operational. The available spectrum of geosensors ranges from 
simple static sensors which capture arbitrary physical 
phenomena such as temperature or humidity to complex sensors 
mounted on highly dynamic platforms like satellites or UAV 
(unmanned aerial vehicle) systems. So far, this real-time sensor 
data cannot yet be exploited in 3D geoinformation services. 
However, if the available geosensor data are combined with 3D 
geoinformation services, new application scenarios can be 
realized. For such applications, mini- or micro UAV systems 
are very promising cost-efficient platforms for capturing real 
time close-range imagery. Possible application scenarios range 
from decision support after natural disasters, such as 
earthquakes, to the virtual piloting of UAV systems. 
Our investigations differ from mini and micro UAV based 
mapping applications in photogrammetry as for example 
presented in (Eisenbeiss 2006) or (Annen and Nebiker 2007). 
The focus of our research is placed on real-time data geo 
registration and geodata integration into virtual globes. 
Especially, we have to continuously geo-register and process 
the available video data streams and low-quality non-metric 
video cameras use for data capturing. 
1.2 Goals and contents of the paper 
This paper presents a prototype system solution, which allows 
for a real-time integration of a video stream captured with a 
mini or micro UAV system into a virtual globe technology. 
Two different integration approaches will be presented, one 
referred to as augmented monitoring, the other as virtual 
monitoring. These two approaches will be presented in detail. 
Additionally, the augmented monitoring approach will enable 
the real-time mapping of arbitrary geo-objects and sharing this 
information among numerous virtual globe clients with the use 
of a collaboration framework. For the realisation of such a 
solution the video geo-registration process based on low quality 
and low cost IMU/GPS sensors used in mini or micro UAV 
systems with an adequate geo-referencing accuracy is a major 
challenge. Especially the system calibration of the camera 
misalignment as well as the continuously changing systematic 
attitude and position error of the MEMS based IMU and 
navigation grade GPS receivers have a great influence on the 
achievable geo-registration accuracy. 
The paper commences with a technical overview of mini and 
micro UAV systems, with a focus on their suitability for 
mapping applications. Additionally, the UAV platform used for 
these investigations is presented. Next, the state of the art of 
virtual globe technologies is reviewed and put into perspective 
with our own virtual globe technology i3D. The paper then 
describes the entire real time data processing chain and the 
developed software components. This workflow is subdivided 
into the following steps. First, the flight data and video stream 
transmission and synchronisation process is introduced. 
Subsequently, the geo-referencing approach for geo-registering 
the captured video streams is discussed. This is followed by a 
description of the different strategies for integrating video 
streams into the i3D virtual globe technology. The final section 
of the paper presents results and experiences from a test case 
project and provides an overview of new applications. 
Furthermore, the mapping accuracies obtained from the above