— TT 
Tree ———————— Creer rer. 
-_— 
-— 
ee 
Tr ——— Atta 
-_— 
Figure 1. Epipolar rectification for ADS strips. Left: level-0 
data with curved epipolar lines. Right: level-1 projection, resul- 
ting in piecewise straight epipolar geometry. 
Frame epipolar lines are inherently straight (after lens distortion 
correction). However, images of a stereo pair are generally not 
located on a common plane, resulting in non-parallel epipolar 
lines. Aiming for processing frame stereo pairs in any orien- 
tation (including across flight strips) and also for the standardi- 
zation of SGM input, we rotate frame images into the epipolar 
orientation and rectify them to a common plane (Figure 2). 
VAR 
Figure 2. Epipolar rectification for frame imagery of arbitrary 
orientation. Left: original image with non-parallel epipolar 
lines. Right: epipolar rotation and plane rectification. 
  
  
  
  
2.2 SGM Job Definition 
Based on the memory requirements of the SGM computation, 
aerial line-scanner and frame images have to be sub-divided for 
processing. We use tiles of up to 1024 image pixels square, with 
tile size and pattern adapted to the area to be processed — typi- 
cally the entire stereo overlap in case of a frame image pair. The 
long and continuous line-scanner image strips are divided into 
sections that can be computed in a reasonable amount of time, 
say less than one hour for each individual SGM job (depending 
on the disparity range, see section 5 for performance numbers). 
A limiting factor is also the amount of data in the info cloud, so 
this output could be displayed and further processed by third- 
party software. All SGM job results from a single line-scanner 
strip can be merged seamlessly to generate a very large, geome- 
trically and radiometrically consistent info cloud. 
  
  
  
  
  
  
Stereo Swath Job Job 
Sensor Overlap | Width Length Size 
[76] [px] [px] [Mpx] 
ADS40/80 100 12,000 8,192 100 
60 6,720 81 
DMC-II 140 12,096 
80 8,960 108 
60 8,486 132 
DMC-II 230 15,552 
80 11,316 176 
60 8,410 141 
DMC-II 250 16,768 
80 11,212 188 
60 4,040 36 
RCD30 9,000 
80 5,386 48 
  
  
  
  
  
  
  
Table 1. SGM job properties for selected line scanner and frame 
sensors, based on Sandau et al. (2000) and Z/I Imaging (2011). 
For the ADS40/80, the SGM job size covers the swath width of 
12000 pixels and approximately 8000 scan-lines along strip, 
resulting in about 100 Megapixels to be processed. This size is 
roughly in the order of the stereo overlap of medium and large- 
format frame images, leading to similar run times and info 
cloud sizes for frame jobs. Examples for different sensors and 
flight configurations are shown in Table 1. 
3. SGM AND POST PROCESSING 
The SGM approach is well-documented by Hirschmiiller (2005, 
2008); our implementation for ADS line-scanner imagery is 
detailed in Gehrke et al. (2010, 2011). Based on the standardi- 
zation in the above-described pre-processing, the SGM core al- 
gorithm as well as most steps of the disparity post processing 
are sensor-agnostic and, accordingly, required only minor adap- 
tations to process ADS and frame image jobs. Therefore, the 
description of our approach to disparity computation and post 
processing is kept brief in this context. 
3.1 Disparity Computation by SGM 
SGM aggregates pixel-based matching costs under considera- 
tion of smoothness constraints. It approximates the theoretically 
desired two-dimensional, global aggregation by a number of 
one-dimensional cost paths for each potential disparity or, re- 
spectively, parallax. A total of 8 paths (directions every 45?) is 
usually considered sufficient. Using 16 paths can improve the 
results but increases computation time, in our current CPU im- 
plementation by about 15% in total. The minimum aggregated 
cost, summed from 8 or 16 directions, at each image pixel leads 
to the disparity map for a stereo image pair. 
3.2 Outlier Elimination and Cleanup 
As a first step in the disparity verification, the roles of base and 
pair (or “left” and “right”) images in a stereo pair are swapped, 
and SGM is carried out both ways. Inconsistent disparities are 
eliminated; they indicate mismatches, which occur predomi- 
nantly in occluded areas (cp. disparity maps in Figure 3). 
Further verification is based on the assumption that small iso- 
lated patches, which significantly differ in height (or disparity) 
from their neighborhood, are most likely errors. For the required 
segmentation, neighboring pixels with significantly different 
disparities are assigned to different segments. Small segments 
are considered outliers and removed from the disparity map. 
Depending on the usage of the SGM results, an optional cleanup 
step can be applied to remove multiple heights at the same 
planimetric location. This can occur within complex buildings 
(e.g. under balconies) or underneath trees if viewed off-nadir, 
and would cause undesired ambiguities in derived 2.5D pro- 
ducts such as TINs or gridded DSMs. 
3.3 Data Reduction and Thinning 
Considering the large amount of data (cp. Table 1), a reduction 
might be desired or even required. This is carried out in two 
ways: by a combination of 2x2 neighboring disparities into one 
and/or by intelligent thinning. The disparity combination also 
removes noise and generally increases vertical accuracy; it re- 
duces the data by up to 75%. Thinning is based on local ranking 
of curvature. This ensures a globally even distribution of de- 
rived points while keeping the locally most significant ones. 
This allows for further data reduction, by 90-99%. 
34 1 
Both | 
into o 
repres 
signec 
cation 
integr: 
NIR b 
output 
  
Figure 
ADS 
Belov 
(left) 
merge 
Botto