Juliang Shao
correspondence can be better determined when the trifocal tensor (Hartley, 1995; Shashua, 1994) is estimated or the
exterior orientation of the camera station configuration is known, as is usually the case in photogrammetry. The Second
type of information is grey level intensity data. Grey level images are locally similar in corresponding patches and a
typical measure of matching quality is the correlation coefficient value, the image distortion being locally approximated
by an affine transformation (Lan and Mohr, 1997).
Such an approach proved early on in the present investigation to be successful for practical applications which utilised
controlled close-range imagery. The extension to aerial imagery was initially problematic, however, since the segment
matching process can inadvertently discard an excessive number of candidate segments. Missing correspondences can
be caused by object occlusion, noise or simply poorly chosen feature extraction thresholds. Thus, at the initial step, it is
unlikely that all valid matches will be fully obtained. With the proposed approach, however, considerable improvement
is achieved in the number of correct multi-image matches for non-noisy segments.
The initial set of recovered matches is used as a starting point in a refined matching process. As the set may contain
wrong matches, a relaxation process is applied to filter out unreliable correspondences. In multiple image matching,
explicit optimisation strategies using binary compatibility constraints for a removal of ambiguous line segment
correspondences are not usually employed. Instead, a more traditional approach considering geometry (eg epipolar line
geometry) and intensity information is likely to prove sufficient for the control of matching quality. Here, relaxation
labelling is applied at each stage as a post-filtering process.
Also at the initial stage, a sparse set of reliable matches is assumed to be provided. The next step is then an iterative
process where at each iteration, already matched features offer ‘evidence’ for either additional matches or additional
features that need to be searched for. No exact prediction of a correspondence can be made without knowing the 3D
position of the feature. However, if we accept the concept of a limited disparity gradient, matched features can be
employed to predict positions of unmatched features. This operation is performed using a local affine model where once
the affine coefficients are determined, a position can be predicted. Given a certain error space, a search can then be
carried out for a predicted feature. When a candidate is identified, a final verification step is performed using grey level
information and linear segment orientation attributes. If no feature is found at this location, a refinement to the segment
extraction is again performed, but with a lower threshold.
The proposed multi-image segment matching scheme has been experimentally evaluated using both aerial and close-
range imagery. The paper discusses an experimental application and reports on the analyses of the results obtained.
2 COMPUTING THE INITIAL MATCHES
Consider the relationship between image points and epipolar constraints in a multi-image configuration. Two points in
two images are admissible matches if the epipolar constraint is satisfied. For multiple images, this corresponds to a
network of constraints, as illustrated in Fig. 1 for the case of three images. Such a network is constructed not only using
the epipolar geometry, but also by employing signal similarity which is quantified through cross correlation and
gradient direction similarity.
Fig. 1: E12 and E21, E13 and E31, and E231 and E321 are the
corresponding epipolar lines. Two cliques of admissible matches are
possible: a, a2/, a31 and a, b21 and 531.
838 International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B3. Amsterdam 2000.
