SENSOR POSE INFERENCE FROM AIRBORNE VIDEOS 
BY DECOMPOSING HOMOGRAPHY ESTIMATES 
E. Michaelsen', M. Kirchhof“, U. Stilla^ 
*FGAN-FOM Research Institute for Optronics and Pattern Recognition 
Gutleuthausstrasse 1, 76275 Ettlingen, Germany 
{michaelsen,kirchhof} Gfom.fgan.de 
^Photogrammetry and Remote Sensing, Technische Universitaet Muenchen 
Arcisstrasse 21, 80333 Muenchen, Germany 
Uwe.Stilla@bv.tum.de 
Commission III, WG HMI 
KEY WORDS: Geometry, Vision, Estimation, Navigation, Orientation, Infrared, Aerial, Video 
ABSTRACT: 
Airborne videos are gaining increasing importance. Video cameras are taking huge amounts of measurements for low costs. Their 
low weight and low requirement for energy makes them particularly attractive for small airborne carriers with low payload. Such 
carriers are discussed for military as well as for civil applications, e.g. traffic-surveillance. Often video cameras are used for 
documentation and reference in connection with other sensor systems. In addition to panchromatic or ordinary colour videos, 
nowadays also cameras operating in the thermal spectral domain gain attention. For the utilization of any stream of measurements 
taken from a moving platform the pose of the sensor in orientation and position has to be constantly determined. For airborne 
platforms often GPS and INS are used to acquire this information. However, the video stream itself provides also possibilities to 
estimate pose parameters. In this contribution we restrict our investigation to almost flat scenes but we allow oblique views both 
forward looking and side looking. The optical flow of the scene fixed structure on the world plane is estimated by a planar projective 
homography. This requires at least four point or line correspondences that can be traced over an appropriate number of frames. If the 
focal length is not changed and the camera has not been rotated, the proper transform will be restricted to a central collineation with 
five degrees of freedom. Two of these - giving the vertex or epipole - can be inferred directly from image correspondences. The 
remaining three are then estimated from the homography by solving a homogenous linear system. They give the axis or horizon, 
from which we obtain the rotational part of the pose, and a scale parameter for the speed to height ratio. Common level keeping 
flight manoeuvres where the epipole is close to the horizon lead to elations. Other manoeuvres - like e.g. landing - lead to 
homologies. The rotation-free calculations will also be appropriate if the camera rotation is known from another sensor. If the 
rotation between the frames is unknown the homography will be decomposed into a central collineation and an orthogonal rotation 
matrix. The five degrees of freedom of the collineation and the three degrees of freedom of the orthogonal rotation matrix sum up to 
eight, which is exactly the same number of degrees of freedom that a planar homography has. There is a set of analytic solutions to 
this equation system, of which the correct solution can be picked by heuristic considerations. We investigate the propagation of 
measurement errors through these calculations. Examples for such estimations are shown for thermal videos. Long focal lengths are 
unfavourable. The rotation-free decomposition gives more stability compared to the decomposition with rotation. 
1. INTRODUCTION special care. Push-broom cameras and CCD-line scanners are 
excluded. 
1.1 Unmanned Aircraft and Airborne Video 
i : ; s 1.2 Properties of the Thermal Spectral Domain 
New possibilities for a variety of tasks including traffic 
monitoring, disaster management, surveillance and military For many tasks operability at any time of the day and also 
applications come with the increasing utilization of unmanned under bad weather conditions is desired. Electromagnetic waves 
aircraft. These crafts can be built quite small and at low cost. in the thermal bands between 3um and Sum or between 8um 
The payload and power resources are limited, but almost always and l2um give the opportunity to measure the black-body 
they will feature one or several digital video cameras. This temperature radiation of the objects on the ground. The energy 
contribution investigates a possible utilization of this sensor that is measured comes from emission rather than reflection. No 
type for the pose estimation and thus navigation of the craft. external light source is needed. The transparency of the 
Automatic control of unmanned aircraft by vision alone may be atmosphere in these two thermal bands is equal or better than in 
one goal while another one may be the combination of this the visual band between 0.4um and 0.8um. For tasks like 
information source with other sensors like inertial systems, laser vehicle recognition or traffic surveillance thermal 
range finders, altimeters, speed sensors or GPS. Here we only measurements give the unique opportunity to determine the 
treat central perspective cameras that take the whole picture ^ operational status of objects. Running engines emit thermal 
through one aperture at one time instance. Devices using analog radiation. Today, the radiometric resolution and dynamic range 
video standard with two half-frames are included but need