F(x4) + J(xk)Pk 
    
-F(x,) * 
Figure 1: The residual surface surface F(x) and the tangent plane at F(x ,) in observation space X". The vector px 
corresponds to the point F(x4) -- J(x«)py on the tangent plane which is closest to the origin O. The right figure is 
centered around F(x,) and shows that J(x,)p; is the orthogonal projection of —F(x,) onto the plane spanned by the 
columns in J (x). For illustrative purposes, L — I is assumed. 
space closest to the origin, as defined by the distance met- 
ric L. At any point F(x,) on the surface, the columns of 
the Jacobian J (x) span a tangent plane around F(x) as 
illustrated in Figure 1 (left). If the distance metric L = 1, 
the identity matrix, then the vector 
Jipx 2 J4(JIJ4) ! JI(-F4) (11) 
is the orthogonal projection of —F , onto the range space 
of Jx as illustrated in Figure 1 (right). If the distance met- 
ric L Z L, the projection is oblique instead of orthogonal. 
In both cases, the point Fy, + J; py is the point on the tan- 
gent plane closest to the origin as measured by L. For an 
in-depth discussion of oblique projections, see (Gulliksson 
and Wedin, 1992, Gulliksson, 1993). 
2.5 ALSM and the Gauss-Newton method 
By formulating the residual function F(x) as 
F(x) = g(z,y) — f(z,9), (12) 
where the picture function g(z, y) is formulated as an ex- 
plicit function of the parameters in x and the available im- 
age go(x,y), the linearization (8) will correspond to the 
linearization of Equation (1) and the Jacobian J will equal 
the design matrix A. Furthermore, by selecting the weight 
matrix W — P, the vector p, of Equation (9) will corre- 
spond to the least squares estimate X of Equation (3) and 
the ALSM update (4) will correspond to the Gauss-Newton 
update (10). 
Thus, under the aforementioned assumptions, the ALSM 
algorithm is congruent to the Gauss-Newton method ap- 
plied to Problem (5) with the residual function (12). 
3 MODIFICATIONS OF THE GAUSS-NEWTON 
METHOD 
At each iteration, the Gauss-Newton method (and the ALSM 
algorithm) calculates an update to the current iterate Xp 
based on the assumption that the function F (x) is approxi- 
mately linear around the point x ,. If this assumption does 
not hold, the method may diverge, oscillate, or converge 
  
  
  
  
  
Figure 2: Contour plot of the objective function f(x) in 
parameter space 3t" for the problem in Figure 1 with the 
corresponding points x; and xx + Pk marked. In this case, 
the function value at x; + pr is higher than the function 
value at xj. 
very slowly. This may be due to many reasons, e.g. the so- 
lution is far from the starting approximation xo, the prob- 
lem is over-parameterized, or the problem has a large resid- 
ual and large curvature at the solution point. The problem 
shown in figures 1 and 2 has a large residual and curvature 
near the solution, i.e. the linearity assumption holds only 
in a small region around x. 
3.1 Line search 
Consider the modified update 
Xk41 — X& t OkDk; (13) 
where p; is calculated as before and aj, > 0 is chosen as 
the first value in the sequence 1, 3, +, .. . such that the new 
point x44 satisfies the Armijo condition (see e.g. (Den- 
nis Jr. and Schnabel, 1983)) 
f (xx + appr) < f(xk) + por Fr WiDr, (14) 
um. 
Figure 4: 
—F, is « 
B betwee 
purposes 
  
|| 
| 
1 
IROE RUE e 
Won gm dox 
x 
  
OOO m= 
  
  
  
FL 
0 1/16 1/8 
Figure 3: 
tiple valu 
Pz in Fig 
accepted 
the corre 
would be 
aem. 5 
for = 
solid str: 
line to f 
for a giv 
is called 
and the : 
The Gau 
called th 
3.2 Ge 
The intei 
that we s 
large ste 
better pc 
search al 
we do n 
ing redu