ISPRS Commission III, Vol.34, Part 3A „Photogrammetric Computer Vision“, Graz, 2002 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
| | 
eo] | } | 
| h | / 
| I 1 | 
| [. | | . 
E geli ————————á 
( ! Ÿ Y 
s SÍ ST s e seed rown 
777 $ 4 1-D projection | oundary | 
4 rz initial intensity 
r4 rji EN seed edge decreasing 
own «T l1 DSL seed region 
oundaries | | fit rectangle 
(a) (b) 
Figure 3: Oriented region growing (ORG). (a) one iteration of region grow- 
ing; (b) determining the location of the grown boundary. 
global thresholding algorithm is not useful for detecting microstruc- 
tures in such images. 
3.1 Oriented Region Growing (ORG) 
We apply an oriented region growing (ORG) algorithm to the CTF 
images for window structure extraction (Wang and Hanson, 2001). 
This algorithm detects a generic class of objects that exhibit a regular 
size, pattern, and orientation. One of its major advantages is that it 
deals with global illumination variations and other types of noise. It 
requires that a window be on average darker than the wall locally but 
not necessarily globally. 
In the current system, the symbolic microstructures are represented 
as a set of disjoint 2-D rectangles, each having two vertical and two 
horizontal edges. A large number of windows in urban areas fit well 
into this representation. Window extraction is performed on the CTF 
images, on which windows appear as dark, rectangular blobs on the 
brighter wall surface. 
Details of the ORG algorithm are shown in Figure 3(a) in the facade 
image space. It runs iteratively, starting from a smaller rectangle 
(called a seed) and growing outward into a larger one that best fits 
the window blob. The growing processes are performed only in the 
two vertical and horizontal directions. In each direction, a search 
strip, e.g. s in 3(a), is established based on the seed. A zero second- 
order derivative criterion, shown in 3(b), is applied to the intensity 
profile, h(s), of the strip for determining the grown boundary. A 
new rectangle is fit to the four boundaries, found in the four strips, to 
form a larger rectangle, which is then treated as a new seed to initiate 
another iteration of oriented growing. The iteration halts when the 
region ceases to grow; the resulting rectangular region is taken as a 
window candidate. 
The ORG module requires only two user-provided parameters, the 
lower and upper bounds of window size. It attempts to find all win- 
dows of any size between the two bounds. 
3.2 Periodic Pattern Fixing (PPF) 
The ORG algorithm is a purely bottom-up process and may result in 
missing candidates due to image noise. A top-down module (Fig- 
ure 4) is designed to fix the missing candidates by applying a high- 
level constraint about the microstructure pattern. This constraint 
states that microstructures of similar size have a periodic pattern in 
horizontal and vertical directions on the facade. Based on this con- 
straint, a periodic pattern fixing (PPF) module is designed for re- 
pairing periodic microstructure patterns. Structures of similar size 
are grouped together; the horizontal and vertical periods of a mi- 
crostructure group are then found using clustering algorithms based 
on their neighboring distances. Missing candidates are then hypoth- 
esized using interpolation or extrapolation. 
In reality, the periodic pattern constraint may not always be strictly 
  
  
  
  
  
  
  
  
  
  
  
CO 0000 00 0 0000 00 0 QUO] DU: ü 
a 0 00 0800 = 000 0 = 00 00 
(Jo 0000 0000 sums. DDDU DODD [assise DODU DODU 
cl 0 oo 00 0 0 Dn 0 0 a line-pattern D: il 
  
  
  
initial windows 
   
find a period 
and complete 
loop until the line-pattern 
no more windows 
  
  
  
0000 00 0 
  
  
d000 O000 0000 | - 
UD00 0000 0600 
0000 0000 0000 
Figure 4: Periodic pattern fixing (PPF). 
remove overlapping 
windows in initial 
windows 
fill in 
the gaps 
  
  
  
  
  
  
  
  
satisfied on all buildings. To ensure that missing candidate hypothe- 
ses are only filled in for windows that exist, a “bottom-up verifica- 
tion" test is used to verify their existence in the LNF images before 
interpolation/extrapolation. On each LNF image, a vertical and hori- 
zontal edge detection algorithm is performed at locations of missing 
candidates (if they are visible). A missing candidate is accepted for 
filling in only if there are sufficiently many LNF images that support 
the hypothesis. 
33 Experiments 
Ten facade images, representing the major buildings in Technology 
Square, were used to test the symbolic window extraction algorithm. 
Figure 5(a) shows a CTF image (512 x 256), where there are two 
types of windows on the facade: twenty small ones aligned on the top 
floor and 192 windows in a matrix pattern. The image is noisy: the 
lower part of the facade has significantly less luminance and contrast 
than the upper part; windows have different local background condi- 
tions and different degrees of blurriness on their edges. In addition, 
there are structures on the bottom floor with irregular reflectance. 
In the experiments, the lower/upper bounds of window size are set 
to 3 and 100 pixels, respectively, for both height and width. Fig- 
ure 5(b) shows the results of the ORG algorithm. The majority of 
the windows are extracted correctly; some windows are missing due 
mainly to extremely low contrast in the CTF image; there are some 
false extractions caused by the irregular reflectance on the bottom 
floor. Figure 5(c) shows the PPF results, in which missing windows 
are correctly filled in and irregular structures are properly removed. 
Table 1 lists the extraction results in the experiments. Among the 
1146 windows on the ten facades, 1119 of them have been extracted 
correctly. Only 27 are missing, accounting for 2.496 of the actual 
windows. Among the 1133 extracted structures, about 98.8% are 
correct windows and only 1.2% (14 extracted structures) are false 
positives. An examination of the images shows that the missing win- 
dows are mainly caused by low contrast of the windows and their 
blurred edges. The false positives are mostly due to the complex in- 
tensity patterns caused by rectangular structures on the wall surfaces 
that look like windows but are not. 
The PPF module is based on the assumption that the windows pos- 
sess a periodic pattern. This strong assumption did not cause signif- 
icant false positives because of the bottom-up verification process. 
However, it is worth noting that PPF is an optional module highly 
dependent on a priori knowledge of the domain, scene, and/or spe- 
cific object. For urban sites where windows do not show periodic 
patterns on buildings, this option may not be used. 
A - 384