ISPRS, Vol.34, Part 2W2, “Dynamic and Multi-Dimensional GIS", Bangkok, May 23-25, 2001
A NEW STEREO MATCHING APPROACH USING EDGES AND
NONLINEAR MATCHING PROCESS OBJECTED FOR URBAN AREA
Mitsuteru SAKAMOTO, Wei LU, Pingtao WANG
Asia Air Survey Co., Ltd. R&D Department
8-10, Tamura-cho, Atsugi-shi, Kanagawa 243-0016, Japan
Tel:+81-46-295-1886, Fax: +81-46-295-1934, E-mail: mi.sakamoto@ajiko.co.jp
Yukio KOSUGI
Frontier Collaborative Research Center, Tokyo Institute of Technology
4259, Nagatsuta-cho, Midori-ku, Yokohama 226-8503, Japan
Tel: +81-45-924-5466, E-mail: kosugi@pms.titech.ac.jp
KEYWORDS: Stereo Matching, Nonlinear mapping, Principle of Coincidence Enhancement, Building Detection, Edge Matching
ABSTRACT
Automatic recognition of buildings has been in demand for efficient digital mapping process or updating of existing information in
geographical information application. To deal with this subject, effective stereo matching technique that is applicable to urban area is
necessary. However conventional stereo matching techniques that make use of area-based, point-based or edge-based matching
cannot generate satisfactory results for urban area because of occlusion or inability of recovering from mismatching.
In this paper, a new stereo matching technique with combinational use of edges and nonlinear mapping is introduced. This approach is
based on process called Coincidence Enhancement Method (CEM) that consists of competition and consensus operation [4]. Edge
segments, derived from edge detection and matching process, are used as initial information or constraints for CEM process and thus
improvements of CEM is achieved.
Experiments on proposed approach in this paper, edge detection, edge matching, CEM and CEM with edge support were executed with
stereo aerial imageries objected for urban area. As a result, it was ascertained that proposed approaches of CEM with edge support
was efficient for improving of the matching results surrounding building’s boundary.
1. INTRODUCTION
Up till now mapping processes of ground objects with aerial
stereo image pairs still depend on professional operator and
needs tremendous time and costs. While new systems known as
Digital Photogrammetric Workstation (DSW) has been presented
past several years, which has more effective functions for low
cost mapping, it has not been realized to detect or map ground
objects automatically. Computer vision is most widely studied
technology and is becoming more and more promising thanks to
easier availability of auxiliary information, such as height
information from laser profiler finder or SAR. However, for urban
area where occlusion happens frequently, the most fundamental
technology such as stereo matching still face great difficulty, so
is edge detection process, which output is one of the most
crucial information for reconstruction of building model.
Edges that can be extracted with various kinds of filters are the
most fundamental information for automatic recognition of
manmade structures. If the structural boundary of a ground
object can be extracted in the form of edge information, it will be
not a much difficult job to reconstruct the 3 dimensional model of
the original structure. There have been numerous researches
regarding efficient and reliable edge detection. However,
because of the limitation of image media, none of the acclaimed
algorithms can produce complete and error free results.
Coincidence Enhancement (CE), being an extension of Hebb’s
rule, is a self-organizing process of neural network modeling.
This process can be modeled by the principle of competition and
consensus [4]. CE model can realize smooth projection between
input signals and output pattern, which means that when the
majority of the initial values are correct, the minority of
erroneous data can be absorbed. This effect is very useful in
reducing the wrong stereo matching result caused by local
minimum. The key factor for CE model to function correctly is to
ensure the reliability of the initial data, which is not an easy job
in the case of computer vision, especially when dealing with real
world images.
Recently, we have been developing a DSW system, and
constantly making improvement for this system [1]. Even though
this system has many superior functions, such as the ability of
handling imageries of several hundred mega bytes at high
speed even on personal computers of middle range cost,
various digitizing function and so on, almost all of the processes
still depend on manual operations. Our ultimate goal for the
system is to introduce as many as possible automatic
processing modules of high reliability to reduce the cost of
production and increase the quality of digitized information.
The algorithms proposed in this paper aim at improving the
precision and reliability stereo matching process by introducing
CE process that is constrained by reliable edge information. The
first section gives a general overview of the proposed
algorithms. The second section describes in detail the improved
algorithms for extracting reliable edge information. The third
section describes the enhanced CE process with the reliable
edge information functioning both as initial value and constraints
during enhance process. Experiments have been conducted and
the results show that the proposed algorithms are capable of
improving the precision and reliability of stereo matching
process.
2. OVERVIEW OF THE PROPOSED ALGORITHMS
2.1 Objectives and Tasks
Our objectives are to improve the reliability and precision of
stereo matching for automatic building recognition. The reliability
is improved by using highly reliable edge information that is
extracted through strict constraints. The precision is improved by
using neural network enhanced with edge constraints. The
background and outline of the two strategies are as follows.
Edges are fundamental components of building. In this study,
edge mainly servers for three purposes. One is for initialization
of global search for stereo matching, which is equal to general
registration in photogrammetry. The other is for constraining the
local matching in CEM, whose brief description will be given
below and the details will be given in section 4. The third one is
for maintaining collinear condition during mapping between
stereo image pairs.
Since edge information extracted by conventional approaches
tends to be noisy and unreliable, our strategy is to only detect of
