broom
de and
ut the
orithm
ling to
1 other
d. The
'essing
'essing
>.
n von
en der
en für
ithmus
t lokal
ihrt zu
'amide
1g von
ignete
eitung
(Wide
e first
ystems
"uction
[WO Or
nounts
ailable
can be
Herbert Jahn
Here, a new method is presented which relies on a pixel-wise minimisation of a quadratic measure of the grey value
deviation of two images (e. g. nadir and forward images) under some constraints to estimate the disparity in both
directions. The 2D disparity vector is updated recursively in each image point of the nadir image until stability is
reached, i. e. the disparity change remains below some threshold. As constraints at the moment the ordering constraint
(Klette et al., 1998) is used. In image regions without occluded areas and with continuous surface the ordering
constraint guarantees a coupling of disparity values over some distance which is essential for stereo matching within
homogeneous areas where no stereo information is available. To transfer stereo information from edges, corners etc.
into homogeneous regions Gaussian smoothing of the image pair is applied.
In order to prevent too big gray value deviations of the stereo image pair a coarse gray value fitting (equal mean value
and standard deviation in both images) is applied as a pre-processing step. The initial condition of the non-linear
algorithm can be given by the disparity values s,(i,j) (in epipolar direction, derived by an epipolar matching algorithm),
$,(,7)-0 (perpendicular to epipolar direction). Using these values instead of s,(i,j)=0 diminishes possible trapping in
local wrong minima if big disparities occur in the image pair.
The resulting disparity images s,(i,j), s,(i,j) can be smoothed with a special edge preserving smoothing algorithm which
assumes that the disparity inside image segments changes smoothly (i. e. without discontinuities). That assumption
especially seems to be useful in order to assign disparities to partly occluded image regions. The used smoothing
algorithm is an adaptation of an edge preserving smoothing algorithm for images of gray values (Jahn, 1998) and other
features (Jahn, 1999).
In contrast to most other (sequential) methods the presented matching algorithm is parallel in space and sequential in
time. As in the human visual system where layered neural processing structures solve the matching problem (Hubel,
1995) the new algorithm also can be implemented in special Multi Layer Neural Networks or in recurrent Neural
Networks. Therefore, when suitable parallel processing hardware (with one neuron or processor element assigned to
each pixel) will be available then real-time stereo processing becomes possible.
Because the disparity is computed in each pixel of the nadir image a dense disparity map is generated and there is no
need for interpolation.
The method was tested with some airborne images with good success. Some results (disparity and matched gray value
profiles, disparity images) are presented. Of course, the quality of the results must be enhanced further, especially in
regions with occlusions.
The method will be described in chapter 2. Then in chapter 3 the results of some processed images are presented and
discussed. Finally, conclusions for further investigations are drawn.
2 THE METHOD
Let 2:(1,j), @r(1,j) (i=1,...N,3 j=1....,Ny) be an image pair registered with a left and a right camera, respectively. In case of
pushbroom line scanners the L-image corresponds to the nadir image and the R-image to one of the images taken by the
forward or backward looking CCD lines. Between both images the following relation (approximately) holds:
8,653) 4 5, s). (1)
Here, s, = s,(i,j) and s, = s,(i,j) are shifts (disparities) in x- and y-direction, respectively. They are considered as
functions of the coordinates (i,j) of the left image (the L-image is considered as a distinguished image having in mind
the pushbroom scanner application where the L-image is the nadir image).
With the coordinate transform i' 2 i + 5/2, j' =j + 5/2 (1) can be written in the equivalent form
no y 25€ ian S,
EL alsa = ER ro 112 Q)
which can be used for obtaining better stability of the algorithm. Of course, the disparity can also be assigned to a
centered (cyclopean) image as it is done sometimes (Belhumeur, 1996, Gimel’farb, 1999). This does not change the
method.
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B3. Amsterdam 2000. 437