The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B5. Beijing 2008 
Drawbacks are the limited range, the small spatial resolution and 
the absolute accuracy in the range of a few centimeters. Possi 
ble applications for RIM sensors could be in the field of human- 
computer-interaction (HCI; Du et al., 2005), robot vision (Gud- 
mundsson, 2006), automotive engineering (Zywitza et al., 2005) 
or human motion analysis (Westfeld, 2007b; Hempel and West- 
feld, 2008). 
(a) (b) 
Figure 1: Range imaging cameras, (a): SwissRanger SR-3000 
(url: http://www.swissragner.ch/, 2007). (b): PMD [vi 
sion] 19k (url: http://www.pmdtec.com/, 2007). 
Figure 2: Near-infrared intensity image (a) and colour coded 
range image (b). 
The data used in this article were captured by the SwissRanger 
SR-3000 (Mesa Imaging AG, Zurich, Switzerland; Figure la). It 
should be pointed out that alternative manufactures, like PMD- 
Technologies GmbH (Siegen, Germany; Figure lb) or Canesta, 
Inc. (Sunnyvale, CA, USA), offer commercially available prod 
ucts, too. The modes of operations are nearly the same, except for 
the chip design: Mesa Imaging AG uses combined CCD/CMOS 
technology, PMDTechnologies and Canesta, Inc. just use CMOS. 
A detailed survey of optical range measurement and solid-state 
imaging sensing is given in Lange (2000). 
3 RANGE IMAGE SEQUENCE ANALYSIS 
Photogrammetric motion analysis is a well-established part of 
close-range photogrammetry and allows the extraction of geo 
metric information from images with high precision and relia 
bility. In this context, least squares matching is a common tool 
for the computation of motion vectors from image sequences. 
3.1 State of the Art 
2- D LSM: 2-D least squares matching formulates the gray value 
relations between two or more corresponding image patches as 
non-linear observation equations (Ackermann, 1984; Forstner, 
1984; Grim, 1985). The goal is to determine six parameters of 
a 2-D affine transformation and - if necessary - a 2-parameters 
radiometric correction. Commonly used in spatial and/or tem 
poral matching tasks (e.g. conventional aero triangulation, DSM 
generation or motion analysis applications), 2-D LSM represents 
a multifunctional instrument for 2-D image analysis. 
3- D LSM: The basic 2-D LSM approach was extended to a 3-D 
algorithm working on voxel data and applied on flow tomogra 
phy sequences by Maas et al. (1994). Accordingly, 3-D LSM 
works with 3-D volume data and voxels rather than 2-D images 
and pixels. The 3-D affine transformation has 12 parameters, and 
the observation equations have to be formulated using gray value 
gradients in three directions. 
Least Squares Surface Matching: Based on a basic 2-D LSM 
approach, Maas (2000) computed correspondences between neigh 
boring or crossing airborne laser scanning strips by formulating 
LSM on a TIN structure. Grim and Akca (2004) proposed a 
3-D least squares surface matching algorithm (LS3D), which es 
timates the seven parameters of a 3-D similarity transformation 
between two or more 3-D surface patches by minimizing the Eu 
clidean distances. 
3.2 2.5-D Least Squares Tracking 
2-D LSM can basically be applied for tracking surface patches 
in RIM data sequences by using the Cartesian coordinates only. 
The proposed 2.5-D LST (least squares tracking) algorithm uses 
the original intensity and range information simultaneously. Due 
to the 2.5-D nature of the surface patches, this is referred to as 
2.5-D here. 
Functional Model: Intensity observations are used in the same 
manner as in conventional LSM: Template patch gvi and search 
patch gV2, taken from consecutive gray value images Zi and I2, 
provide gray value observations for the adjustment at each posi 
tion (x, y) resp. (x', y'). The geometric and radiometric relations 
between those patches can be formulated as 
gvi(x,y) - Vi(x,y) = r 0 + 7-1 • gv 2 (x',y') (1) 
Based on the same considerations, the relation between two patches 
rvi and rv 2 taken from range value images TZi and TZ2 become 
rv 1 (x, y') - v 2 (x, y) = do + di • rv 2 (x',y / ) (2) 
The geometric affine transformation model for both types of ob 
servations, intensity and range, is given by 
x — ao + a\x + a 2 y and y = 60 + b\x + b 2 y (3) 
In Equation 1, ro and r\ model brightness and contrast variations. 
In analogy to a radiometric gray value correction, range variations 
between template and search window can be formulated as a lin 
ear function, too. Thus, it is also possible to compute a 1-D depth 
shift do and a depth scale factor d\. Within a Gauss-Markov- 
Model (GMM), the parameters can be estimated by minimizing 
the sum of the squares of the error vectors vi and v 2 . 
Stochastic Model: The stochastic model describes the variances 
and covariances of the observations. In many cases, the setup of 
the stochastic structure of the observations (variance-covariance 
matrix; VC-matrix) is given by the a-priori definition of fixed 
weights. Information from the instrument manufacturer or from 
previous accuracy analyses provide the basis for the specification 
of the quality of the observations. Besides the parameters of the 
functional model, the standard error of unit weight can be esti 
mated for the stochastic model only. Thus, the evaluation of the 
quality of the observations using the variance of the unit weight 
is limited to homogeneous observations. 
2.5-D LST uses heterogeneous observations (intensity and range) 
and requires adjusted weights for each group of observations. In 
our work, the weights are computed by iterative variance com 
ponent estimation (VCE; e.g. Schneider and Maas, 2007). This 
approach provides the following advantages: 
• An automatic estimation of the variance components. 
934