2 MOTIVATIONS AND BASIC CONCEPTS
Three main goals motivated the design of the matching ap-
proach presented in this paper. These goals are emerged from
the aspects described in the previous section and the lack of
sufficient solutions for them.
1. Reliable matching with single points:
Failures in area-based matching are usually rooted in
insufficient information within the matching windows.
In DEM generation applications for exam ple, constraints
are used for obtaining reliable results. The object sur-
face is assumed to be smooth, or at least smooth
between discontinuities. If the disparity at a certain
point is considerably different from its surroundings
(i.e., the matching procedure converged to a wrong
location), or if a solution cannot be obtained, an eleva-
tion is calculated by interpolation from the neighboring
points. When only a single point is required at each
location, which is the case in automatic aerotriangula-
tion, it is not possible to use the smoothness constraint
to check the reliability of the results.
A possible solution is to use large matching windows,
which are more likely to contain sufficient informa-
tion. This however, leads to inaccuracies if conven-
tional area-based matching techniques are used. In
these techniques, the shape of the object surface is
assumed to be a horizontal plane (in straightforward
cross-correlation without shaping) or a tilted plane (in
least-squares matching using an affine transformation).
The idea in the proposed matching method is to re-
construct a small surface patch around each point to
be matched, via a hierarchical iterative procedure. The
purpose of this reconstruction is to have a better rep-
resentation of the ground within the matching win-
dow. Using the reconstructed ground surface the im-
age patches are warped and the matching is performed
between the warped patches. Since the geometric dif-
ferences are minimized, patches which are significantly
larger than those used by traditional methods are ac-
curately matched by a least-squares procedure, using a
rather simple transformation.
2. Avoiding numerical problems and correlation among
parameters:
Theoretically, the mathematical model of least-
squares matching may include the surface elevations
and the exterior orientation parameters (see e.g.,
Ebner et al. (1993)). However, when these paramet-
ers are introduced as unknowns in the LSM, numerical
problems may lead to a weak solution. The cause for
these problems are dependencies between the paramet-
ers. For example, such dependencies exist between the
elevations and some of the exterior orientation para-
meters.
A possible solution is to alternate alternating two
groups of unknowns within adjustment procedure. The
image patches at each tie point are warped and
matched as describe earlier. Then, the entire set of
tie points is used for determining the orientation para-
meters of all the photographs in the block by a stand-
ard block adjustment. By alternating these two steps
while refining the resolution of the images, the process
is converged to the correct location.
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B3. Vienna 1996
3. Avoiding correlation among observations:
When more than two images are used for matching,
a mathematical model based on simultaneous adjust-
ment should be used to take advantage of the addi-
tional information. One possibility is to use differences
between gray values from each possible combination
of a pair of images as observations [Agouris (1992)].
With this approach, there are correlations among the
observations. For example, if three image patches are
considered, and the differences between the first and
the second images, and between the first and the third
are involved, the differences between the second and
third images do not contribute any new information to
the solution.
The proposed matching method uses gray values as
observations, rather than gray-value differences. The
“true” intensities of the surface (referred to as the gray
values of the surface elements) are introduced as un-
knowns in the adjustment.
Two key concepts are used in the multiple-patch matching
in the object space, which was derived as a consequence of
the motivations described above: matching more than two
image patches simultaneously matching warped images (or
matching in the object space). The former is described in
detail in [Krupnik (1996). The second concept and its use
are described here.
When the surface around a certain tie point is known, it is
possible to rectify the image patches that cover this area to ob-
tain an orthophotograph. Since in the case described here the
surface is approximated but not known, the result of the recti-
fication process are “warped” image patches that are not real
orthophotographs. However, geometric differences between
warped image patches of the same area are significantly smal-
ler than those of the original image patches. The rationale of
matching warped patches is described below.
For the sake of simplicity, the idea is explained here for the
one dimensional case with two images only. The extension
to the two dimensional case with more than two images is
obvious.
Figure 1 shows a schematic description of the geometric situ-
ation. For a given point pe on the left image, the conjugate
point p; on the right image is sought. If point pr is an ap-
proximation for p;., an object point P, which does not lie on
the true surface, is obtained. Using the available surface el-
evations (which are not necessarily known a priori, but are
approximated and modified during the iterative procedure),
the image patches, centered on p; and p, are warped. The
corrected location found, by the matching of the warped im-
ages (P) is then projected back to the image (based, again, on
the available surface elevations), and a better approximation
is obtained for the matched point. Note that as the differ-
ences between the approximate and the true surfaces become
smaller, the correction will bring the location found by the
matching closer to the true location.
3 OUTLINE OF THE PROPOSED STRATEGY
Figure 2 shows a schematic description of the iterative pro-
cedure. Each iteration contains three main phases:
Warping and Matching Phase (Figure 3): Image patches,
centered on each tie point, are warped according to the
400
Figure
warpe
