lume XXXIX-B3, 2012
|
NSTRUCTION
teo@mail.nctu.edu.tw
ka02001@hotmail.com
ictural lines extraction. The
ation, Speeded Up Robust
bined for block adjustment.
1 image space are extracted
tilt displacement of image
rity measurement. Finally,
ages are acquired by a non-
he accuracy of orientation
| matching is more flexible
approaches consider the
ints simultaneously. The
| image point of the master
jugate points are obtained
oint of the master image is
n the contrary, the object-
| point in the object space.
ed to the image spaces and
window will be calculated.
is not fixed in the object-
be a point or a line. The
' Which can provide more
‘e. As most of the façades,
posed of straight lines, the
the façade reconstruction
mn the line matching for
et al, (1999) employs
ng of aerial images based
etric similarity. McIntosh
me lidar data and aerial
gital surface model. The
raints like epipolar lines,
f gray value surround line.
rrestrial lidar and images
le reconstruction. Lidar
e provides linear features.
le to modelling the fagade
re images is a challenging
le variety of close-range
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B3, 2012
XXII ISPRS Congress, 25 August — 01 September 2012, Melbourne, Australia
images is quite large when compared to airborne vertical
images. Moreover, relief displacement caused by façade
structure is relatively large when the images are taken from
different look directions. Consequently, image matching cannot
find the conjugated feature correctly or find it incompletely.
The aim of this paper is to solve these problems. This paper
uses a coarse building model (ie. LOD 2 building model
without façade structure) to overcome the problem of scale and
tilt displacement in object-based matching. We correct the
distortion of close-range images using wall of LOD-2.
Moreover, multi-view and multi-widow matching strategies are
proposed to improve the reliability of image matching.
Figure 1(a) shows an example of a window taken by a hand-
held camera. The tilt displacement of window is caused by
different camera stations while the relief displacement is caused
by the depth of window. The boxes in Figure l(a) indicate
matrixes for image matching. The gray values of the matrixes
are different and may affect the correctness of matching. The
corrected images are shown as Figure 1(b). This paper uses
LOD 2 building model to correct the image displacement for
matching. The red boxes in Figure 1(b) indicate the corrected
matrixes for image matching. The selection of red box is better
than yellow box in Figure l(a) as it is able to improve the
similarity between images.
(a) original images for matching
(b) corrected images matching
Figure 1. An example of a façade structure from original and
corrected images.
The objective of this paper is to extract the façade structure
using multiple close-range images and LOD 2 building model.
In order to improve the level of detail of building models, this
research develops a façade linear extraction procedure using
multi-image matching. The major works are orientation
determination, line extraction, multiple images matching, and
3D line regression. In orientation determination, Speeded Up
Robust Features (SURF) is applied to extract tie points
automatically. Then, the tie points and control points are
combined for block adjustment. The line extraction combines
canny edge detector and Hough transform to obtain 2D straight
line in image space. In multiple images matching, the multiple
images are projected to LOD 2 building using different depths.
Then, the multiple windows are generated based on the target
features. The average of normalized cross correlation is
calculated from all object images. Finally, a least squares line
regression is used to obtain 3D façade structural lines.
2. METHODOLOGIES
The proposed method includes four major parts: (1) orientation
determination, (2) generation of linear feature, (3) multiple
images matching, and (4) generation of 3D line. The workflow
of the proposed method is shown in Figure 2. The explanations
of each step are stated as follows.
63
LOD 2 building Close range
model images
Y
; ; : Interior
Orientation modelling va orientation files /
Ÿ
Line extraction
Ÿ
Multiple Images
matching
Y
3D line regression
Façade
structural lines
Figure 2. Flowchart of proposed method.
2.1 Orientation Modelling
Assume that the interior orientation parameters are available.
Orientation modelling establishes the relationship between
multiple close-range images using tie points and control points.
Speeded Up Robust Features (SURF) (Bay et al, 2006) is
applied in automatic tie point extraction as it can overcome the
scale and rotation effects between close-range images. Then, a
large number of automatic-extracted tie points and sparse of
manual-measured control points are integrated in bundle block
adjustment. As the mismatching is unavoidable in tie point
matching, the tie points with large positioning error are
removed iteratively in bundle block adjustment.
2.2 Line Extraction
Canny edge detector (Canny, 1986) and Hough transform
(Hough, 1962) are used to extract the line features on building
facades. Canny edge detector extracts edges by pixel gradient
and double thresholds. After the Canny edge detector, the edges
of all objects such as the facade texture, trees and surface
features in the image are extracted. In order to specify the
facade structures in huge amount of edges, Hough transform is
applied to extract the straight lines. Hough transform converts
each pixel of edges into parametric space, where all pixels are
represented as curves. The peak of accumulated curves
represents the location of the line which appears most of the
times that is the significant straight lines of all edges.
2.3 Object-based Multiple Image Matching
The highly overlapped close-range images provides favourable
geometrical configuration with high redundancy. The high
similarity between contiguous stereo images is beneficial to the
reliable image matching. Hence, the 3D features generated from
image matching have a great potential in 3D modelling. The
aim of multiple image matching is to consider all the available
images for similarity measurement simultaneously. The
advantage is not only to increase the measurement from
different views, but also to ensure the correctness of matching.
There are two ways to perform multiple image matching. The
first one is an image-based method which utilizes the idea of
pass point between overlapped images. The matched points on
the first stereo pair are passed to the next stereo pair to ensure
the correctness. This process will stop when the matched points
meet the end of the image strips. The second method is an
