Figure 1: Multiple image patches covering the same surface 
patch S 
rj 2 G(X, Y) — Tf [s (Tf (s, 9))] (1) 
with ri, à = 1,2,...,p the residual vectors, dimension m?, 
and G(X,Y) the gray level function of the surface patch S. 
Each image patch contributes m? equations leading to a to- 
tal of pm? observation equations. The parameters to be de- 
termined include the gray levels G( X, Y) and the elevations 
Z(X,Y ) of the surface patch, as well as the transformation 
parameters T and TE for every image patch. 
With equation 1 the task of reconstructing S from multi- 
ple image patches is formulated as a least squares problem 
where the gray level differences between the image patches 
and the surface are minimized by varying the surface shape 
Z(X,Y), the surface gray levels G(X,Y) and the exterior 
orientation of the image patches. This takes the concept of 
matching in object space a step further to include the deter- 
mination of the gray levels of the surface. Since the image 
patches p; have to be resampled with the geometric trans- 
formation TS (which in turn is a function of the unknown 
exterior orientation parameters!) the image patches must 
be larger (n x n) than the surface patch (m x m), hence 
m « n. 
2.1. Geometric Transformation 7*9 
There are several possibilities to model the geometric trans- 
formation between the image patches p; and the surface 
Z(X,Y). 
1. The collinearity equations are the most general trans- 
formation between surface and images. The transfor-. 
mation parameters comprise the exterior orientation 
elements of the images I;. 
2. Since the image patches are rather small the central 
projection may be approximated by a parallel pro- 
jection. In that case the transformation parameters 
would include the spatial direction of the projection (3 
angles), a translation of the image patch and a scale 
factor. 
3. It is also conceivable to approximate the surface by an 
analytical function and determine its parameters. 
4. As a further simplification of model 3 we approximate 
the surface S by a plane. The relationship between 
an image patch and S can now be expressed by a pro- 
jective transformation which in turn may be approxi- 
mated by an affine transformation. This would corre- 
spond to the classical case of least squares matching 
with shape parameters. 
2.2. Radiometric Transformation 7" 
Based on the assumption that S is a Lambertian surface a 
linear radiometric relationship of the form 
T* — ro t 71 (9: (2,9) (2) 
exists between the image patches. It may even be adviseable 
to perform the radiometric adjustment prior to the match- 
ing. Therefore, we exclude the radiometric transformation 
from the following considerations. 
3. GEOMETRIC TRANSFORMATION MODELS 
3.1. Central Projection 
We linearize the observation equation 1 under the assump- 
tion that T® is a central projection. Disregarding the radio- 
metric transformation and dropping the indices : for denot- 
ing i^ image patch equation 1 reads 
f G(X, Y) rU (Tes T,) (3) 
where g(T,, T,) needs to be linearized with respect to the 
exterior orientation parameters and Z. 
r = G(X,Y) - g(T9, T9) — S^ 2T. da; 
7 (09g(T,, T.) 0T, 
g( ^ 
V9 (Te, 1y) UT ) Aa; 
1-1 
al 
2T, da; 
with 
. 3g(z,y) 
gs scie df, 
- $g(z,y) 
gy T IT, (5) 
we obtain 
QT. 2T, 
r = G(X,Y) — ZU, 72) — > CES + va Aa; (6) 
1—1 a 1 
  
  
where 
the pz 
initial 
mined 
image 
ramet 
AZ, t 
G(X, 
3.2 P. 
If the : 
mined 
tem w 
weakly 
tersect 
the sai 
patche 
adjust. 
If the 
paralle 
projec 
the bu 
Fig. 2 
space. 
the trz 
tive ce: 
is then 
The an 
dle ray 
With t 
photo 
in the 
R, in 
multip! 
orienta 
If we r: 
become 
we dea 
relatior 
are the 
coordin 
Now wi 
transla 
sates fc 
formati 
describ