ISPRS Commission III, Vol.34, Part 3A „Photogrammetric Computer Vision“, Graz, 2002
| |
eo] | } |
| h | /
| I 1 |
| [. | | .
E geli ————————á
( ! Ÿ Y
s SÍ ST s e seed rown
777 $ 4 1-D projection | oundary |
4 rz initial intensity
r4 rji EN seed edge decreasing
own «T l1 DSL seed region
oundaries | | fit rectangle
(a) (b)
Figure 3: Oriented region growing (ORG). (a) one iteration of region grow-
ing; (b) determining the location of the grown boundary.
global thresholding algorithm is not useful for detecting microstruc-
tures in such images.
3.1 Oriented Region Growing (ORG)
We apply an oriented region growing (ORG) algorithm to the CTF
images for window structure extraction (Wang and Hanson, 2001).
This algorithm detects a generic class of objects that exhibit a regular
size, pattern, and orientation. One of its major advantages is that it
deals with global illumination variations and other types of noise. It
requires that a window be on average darker than the wall locally but
not necessarily globally.
In the current system, the symbolic microstructures are represented
as a set of disjoint 2-D rectangles, each having two vertical and two
horizontal edges. A large number of windows in urban areas fit well
into this representation. Window extraction is performed on the CTF
images, on which windows appear as dark, rectangular blobs on the
brighter wall surface.
Details of the ORG algorithm are shown in Figure 3(a) in the facade
image space. It runs iteratively, starting from a smaller rectangle
(called a seed) and growing outward into a larger one that best fits
the window blob. The growing processes are performed only in the
two vertical and horizontal directions. In each direction, a search
strip, e.g. s in 3(a), is established based on the seed. A zero second-
order derivative criterion, shown in 3(b), is applied to the intensity
profile, h(s), of the strip for determining the grown boundary. A
new rectangle is fit to the four boundaries, found in the four strips, to
form a larger rectangle, which is then treated as a new seed to initiate
another iteration of oriented growing. The iteration halts when the
region ceases to grow; the resulting rectangular region is taken as a
window candidate.
The ORG module requires only two user-provided parameters, the
lower and upper bounds of window size. It attempts to find all win-
dows of any size between the two bounds.
3.2 Periodic Pattern Fixing (PPF)
The ORG algorithm is a purely bottom-up process and may result in
missing candidates due to image noise. A top-down module (Fig-
ure 4) is designed to fix the missing candidates by applying a high-
level constraint about the microstructure pattern. This constraint
states that microstructures of similar size have a periodic pattern in
horizontal and vertical directions on the facade. Based on this con-
straint, a periodic pattern fixing (PPF) module is designed for re-
pairing periodic microstructure patterns. Structures of similar size
are grouped together; the horizontal and vertical periods of a mi-
crostructure group are then found using clustering algorithms based
on their neighboring distances. Missing candidates are then hypoth-
esized using interpolation or extrapolation.
In reality, the periodic pattern constraint may not always be strictly
CO 0000 00 0 0000 00 0 QUO] DU: ü
a 0 00 0800 = 000 0 = 00 00
(Jo 0000 0000 sums. DDDU DODD [assise DODU DODU
cl 0 oo 00 0 0 Dn 0 0 a line-pattern D: il
initial windows
find a period
and complete
loop until the line-pattern
no more windows
0000 00 0
d000 O000 0000 | -
UD00 0000 0600
0000 0000 0000
Figure 4: Periodic pattern fixing (PPF).
remove overlapping
windows in initial
windows
fill in
the gaps
satisfied on all buildings. To ensure that missing candidate hypothe-
ses are only filled in for windows that exist, a “bottom-up verifica-
tion" test is used to verify their existence in the LNF images before
interpolation/extrapolation. On each LNF image, a vertical and hori-
zontal edge detection algorithm is performed at locations of missing
candidates (if they are visible). A missing candidate is accepted for
filling in only if there are sufficiently many LNF images that support
the hypothesis.
33 Experiments
Ten facade images, representing the major buildings in Technology
Square, were used to test the symbolic window extraction algorithm.
Figure 5(a) shows a CTF image (512 x 256), where there are two
types of windows on the facade: twenty small ones aligned on the top
floor and 192 windows in a matrix pattern. The image is noisy: the
lower part of the facade has significantly less luminance and contrast
than the upper part; windows have different local background condi-
tions and different degrees of blurriness on their edges. In addition,
there are structures on the bottom floor with irregular reflectance.
In the experiments, the lower/upper bounds of window size are set
to 3 and 100 pixels, respectively, for both height and width. Fig-
ure 5(b) shows the results of the ORG algorithm. The majority of
the windows are extracted correctly; some windows are missing due
mainly to extremely low contrast in the CTF image; there are some
false extractions caused by the irregular reflectance on the bottom
floor. Figure 5(c) shows the PPF results, in which missing windows
are correctly filled in and irregular structures are properly removed.
Table 1 lists the extraction results in the experiments. Among the
1146 windows on the ten facades, 1119 of them have been extracted
correctly. Only 27 are missing, accounting for 2.496 of the actual
windows. Among the 1133 extracted structures, about 98.8% are
correct windows and only 1.2% (14 extracted structures) are false
positives. An examination of the images shows that the missing win-
dows are mainly caused by low contrast of the windows and their
blurred edges. The false positives are mostly due to the complex in-
tensity patterns caused by rectangular structures on the wall surfaces
that look like windows but are not.
The PPF module is based on the assumption that the windows pos-
sess a periodic pattern. This strong assumption did not cause signif-
icant false positives because of the bottom-up verification process.
However, it is worth noting that PPF is an optional module highly
dependent on a priori knowledge of the domain, scene, and/or spe-
cific object. For urban sites where windows do not show periodic
patterns on buildings, this option may not be used.
A - 384