IX-B3, 2012 
A MULTI-SENSOR APPROACH TO SEMI-GLOBAL MATCHING 
S. Gehrke?, M. Downey, R. Uebbing*, J. Welter?, W. LaRocque" 
* North West Geomatics Ltd., Suite 212, 5438 - 11? Street NE, Calgary, Alberta, T2E 7E9, Canada — 
{stephan.gehrke | michael.downey | robert.uebbing | john.welter} @nwgeo.com 
? Intergraph Corp., 19 Interpro Road, Madison, AL 35758, USA — william. larocque@intergraph.com 
Commission III, WG III/1 
KEY WORDS: Matching, Point Cloud, DEM/DTM, Surface, Multisensor, Aerial, High Resolution 
ABSTRACT: 
After we first presented the Semi-Global Matching (SGM) implementation for Leica ADS line-scanner data, the interest in applying 
this surface extraction to aerial frame imagery has increased. The reason is the combination of high-resolution geometry and multi- 
spectral information in the resulting point clouds. Such comprehensive point clouds or, more generic, information clouds (info 
clouds) allow for many different uses of the data, including applications that make currently use of LIDAR. 
The DSM extraction tool for the ADS is based on SGM, which enables the derivation of disparity maps and eventually point clouds 
at the very image resolution. This approach was now extended to support both frame sensors and line-scanners in order to provide an 
integrated workflow for different sensor types. This paper describes how SGM is used in a sensor-agnostic system, based on few 
specific pre- and post-processing steps, within the DSM extraction tool we developed. Results from the ADS line-scanner as well as 
from DMC-II and RCD30 frame data are presented. 
1. INTRODUCTION 
Dense surface extraction from aerial imagery is becoming an 
important feature of photogrammetric processing software. As 
of today, several commercial solutions are either available or 
announced. We first presented dense image matching for ADS 
line scanner data in 2010 (Gehrke et al., 2010); this DSM ex- 
traction was released shortly after with the Leica XPro 5.0 ADS 
ground processing software. Ever since, the interest in dense 
surface extraction using aerial frame imagery has increased. 
Based on that demand, the original application was expanded to 
also support frame imagery, aiming towards the flexible proces- 
sing of both aerial line-scanner and frame image data. 
Our DSM extraction is based on Semi-Global Matching (SGM), 
a dense image matching approach that allows for the derivation 
of disparity maps and eventually point clouds at the image reso- 
lution (Hirschmiiller, 2005, 2008; Gehrke et al., 2010, 2011). 
With the color data available from aerial sensors — for ADS and 
most frame sensors: RGB and near infrared — and derived point 
classification, the SGM-based point clouds are extended to 
information clouds (info clouds) that provide high-density and 
high-quality geometric and radiometric information for a broad 
variety of applications, including but not limited to fields that 
are currently using LIDAR data. See Gehrke et al. (2010) for a 
comparison of SGM-derived info clouds with LiDAR point 
clouds. 
The initial implementation for ADS comprises pre-processing 
including the required epipolar rectification, the disparity map 
computation based on SGM and various post processing steps to 
eventually provide an info cloud that is virtually error-free. The 
SGM core and most post-processing are carried out in disparity 
space, which is geometrically identical with the rectified image; 
it is independent from the type of sensor, and the transition from 
ADS to frame data processing is straightforward. For pre-pro- 
cessing and final projection into the info cloud, we implemented 
and adapted the sensor model functionality and, accordingly, the 
modeling of the epipolar geometry. 
The remainder of this paper describes the multi-sensor SGM ap- 
proach, with focus on the (few) sensor-specific processing steps 
that are required to apply our highly optimized and well-tested 
implementation to different types of input imagery. A number 
of results from ADS images as well as DMC-II and RCD30 data 
is shown. Note that a comparative evaluation of SGM results is 
outside the scope of this paper, especially because of the impact 
of a variety of parameters and constraints outside the very SGM 
processing. 
2. DATA SETUP AND PRE-PROCESSING 
One of the steps taken towards a multi-sensor approach was the 
standardization of the stereo model, i.e. the provision of image- 
ry in epipolar orientation. Due the memory-intense SGM pro- 
cessing, a tiling scheme is generally required. It has to be opti- 
mized considering the available stereo coverage — depending on 
frame image size, flight and processing configuration. 
2.1 Epipolar Rectified Imagery 
Straight, parallel epipolar lines are generally desired for image 
matching; it is a precondition in our SGM setup. Such a con- 
figuration is theoretically provided by a perfectly linear line- 
scan. However, the actual ADS flight is non-linear due to Earth 
curvature and atmospheric turbulence, resulting in (perceived) 
distortions and curved epipolar geometry in the level-0 data, 
which is merely a collection of individual scan-line’s images. 
Therefore, the original data from all view angles are rectified to 
a common plane. The result is continuous geometry throughout 
a very long ADS strip (level-1), i.e. redundant stereo coverage 
for hundreds of thousands of scan-lines with piecewise straight 
and parallel epipolar lines (Figure 1).