ISPRS Commission III, Vol.34, Part 3A ,,Photogrammetric Computer Vision“, Graz, 2002
cannot be calculated a disparity at all. To such pixels only with
prior information or by some kind of interpolation a (often
inaccurate) disparity can be assigned. With respect to our
attracting forces that means the following: If there is an edge in
the left image between (i-1,j) and (ijj) and another one between
(i -1,j) and (ij) in the right image then there is a force K7(i,i’j)
originating from (7j) and attracting (i') and another force K,(i-
l1, -lj) acting from (i-1j) to (i-1j). This is necessary for
coping with occlusion.
Let’s consider the external force Kz(i,i’j). Then, first, that force
depends on the difference |g; (i,j)-gx(i'y)| or, more general, on a
certain mean value of that difference. That mean value should
be calculated only over pixels which are in the same image
regions as pixels (ij) (in left image) and (i) (in right image) in
order to exclude problems with occlusion. To guarantee this, the
averaging is performed only over image points (ijj) with
igi G)-gi)| € threshold and (i^) with |gs(i"J)-ga(i)| €
threshold, respectively. We denote that mean value as
Ag(i.i5j)- (g,(.7)-2,6.7])) ^ ae
Secondly, pure radiometric criteria are not sufficient. Therefore,
geometric deviations are taken into account too. To do that, we
consider two region border lines (one in the left image and the
other in the right image) which contain the points (i,j) and (i),
respectively. The situation is shown in figure 4.
Fig. 4. Borderlines in left image, in right image, and overlaid
We see that both borderlines are different and do not match. A
useful quantity for measuring that mismatch is the sum of the
border point distances along the horizontal lines drawn in figure
4. Be (ipj+K) a point on the left borderline and (i ’,j+k) a point
on the right borderline, respectively. For k — 0 the points are
identical with the points (/) and (ij). Be d, — |i - i^, Then, a
useful border point distance is
Ww
d n = i-a) (17)
kz-w
The distance d, accomplishes a certain coupling between
epipolar image rows which are no longer independent. This
sometimes can reduce mismatches efficiently.
With Ag and d, the total distance is
d(i,i; j)- 0 -d, (i.i; j)- 0 - Ag (i: j) as)
The smaller that distance between edge points (ijj) and (ij) is,
the bigger is the force Ky(i,i’,j). Therefore,
Kp(i,l'; ÿ) exp[-d(i,i'; j)] (19)
seems to be a good measure for the force Kz. The calculation of
K, is fulfilled analogously.
Now, the (external) force K.,(i’,) acting on point (i) can be
computed as the maximum of all forces Ky(i,i’,j) with different i
or as a weighted sum of these forces. Here, we take into account
only points (ij) with | i - i'| € Max disparity. The maximum
disparity used here is often known a priori. The introduction of
Max disparity is not necessary. One can also use distance
depending weighting and calculate the resulting force as
Kent (i, j) = > Ky Er J): fli —X (i, j))- sign(i —X (i, j))
(20)
with f{| i —x,(i’j)|) being a certain weighting function which
decreases with increasing distance |i - x,(i’;j)|. Here, we use the
special function
ss ix if lx| € Max _ disparity n
0 elsewhere
Up to now we have considered only forces which act only on
image points (^j) near edges. But we must assign a disparity to
each point of the right image. Therefore, the disparity
information from the edges must be transfered into the image
regions. Within the model presented here, it is useful to do this
by means of adequate forces, which connect the edge points
with interior points (i.e. points inside regions). Local forces of
spring type have been studied for that purpose. Let x(i',j) and
x(i 1) be two neighboured mass points which we assume to
be connected by a spring. Then, point x(i’+1,j) exerts the
following (attracting or repulsive) force on point x,(i’,j):
K pring E18, 7) = 8 [0,041 J) - x, Gj) -1]
(22)
The same force, but with the opposite sign, acts from x,(i’j) on
x(i 1j) according to Newton's law of action and reaction.
Experiments with those and other local forces (e.g. internal
friction) have not been fully satisfying up to now. Of course, the
stereo information is transferred from the edges into the regions,
but very slowly. One needs too many recursions until
convergence. Therefore, one result of these investigations is that
local forces are not sufficient. We need far-field interaction
between points x,(i'j) and x(i'^kj) which can easily be
introduced into our equations of motion. First experiments with
such forces have given some promising results but that must be
studied more detailed in future.
The algorithm is applied here to the standard Pentagon stereo
pair because that image pair is a big challenge because of the
many similar structures and the many occlusions. Figure 5
shows a section of the smoothed image pair (see figure 3 for the
whole left-hand image).
A - 178