ISPRS Commission II, Vol.34, Part 3A „Photogrammetric Computer Vision“, Graz, 2002
(b1)
(a2) (b2)
(a3) (b3)
tures, but not unmodeled ones. A median-based technique developed
by (Coorg and Teller, 1999) repairs unmodeled occlusions; however,
the method may cause blurred or disrupted boundaries of structures.
We describe a new method to obtain a realistic facade texture map,
removing occlusions and effects of illumination variations. This
method takes as input a coarse geometric model of buildings and a set
of images taken from nodes at different locations and associated with
reasonably accurate, but not exact, camera pose information. We as-
sume that the light source for the urban images are normal sunlight
(i.e. nearly white) and thus only the luminance in the color space is
considered in this paper. We also assume that the building facades
are close to the Lambertian surface model (Foley et al., 1990).
As a preprocessing step, the input images are rectified into facade
images, i.e. images under orthographic projection of a facade. This
happens only to a subset of the input images in which the facade
is (at least partially) visible. For each image, the facade visibility
and rectification is calculated based on the camera geometry at the
node where the image is taken. Figure 1(d) shows some sample fa-
cade images in our experiments. To facilitate texture fusion for re-
moving degradation effects (e.g. occlusions), the facade images are
normalized by linear gray-level stretching; the resulting luminance-
normalized facade images (or LNF images) have the same average
luminance and thus are comparable to one another.
2.1 Weighted Averaging
The core of our method is a weighted-average algorithm that gen-
erates a consensus texture facade image (or CTF image) for each
facade. The luminance value of pixel [, j] in the CTF image of a
facade is a weighted average of all LNF images of that facade:
Yerrli, Jl = S^ Yrurlo 7] * w' [à j], (1)
T
eG» "n
Figure 1: Texture recovery. (a) environment mask [al: camera position, a2: LNF image, a3: mask]; (b) obliqueness mask [b1: ca
image, b3: mask]; (c) correlation mask [c1: a version of CTF image, c2: LNF image, c3: mask]; (d) sample original facade images of
image without deblurring; (f) CTF image after iterative deblurring.
mera position, b2: LNF
this wall; (e) initial CTF
NE ES Q)
in which Ywr is LNF image 7, Ycrr is the fused CTF image, and
w^ is the weight factor determined by three masks described below.
A mask is an image whose pixel value indicates the relative impor-
tance of the corresponding pixel in the LNF image. The three masks
measure three different physical attributes.
Environment Mask is a binary mask that specifies whether a pixel is
occluded by a modeled object (Figure 1(a)). It is computed using the
camera geometry and the 3D coarse model: Mg [i, j] is set to 0 if
pixel [4, j] is occluded; otherwise, it is set to 1.
Obliqueness Mask is a grey-scale mask that represents the oblique-
ness ofa facade as seen from the camera (Figure 1(b)), also computed
from the geometry:
Moli, j] = cos 0" (i, j), (3)
in which 0” (i, j) is the camera viewing angle at [i, j] on the facade
measured from the normal of the facade.
Correlation Mask is a grey-scale mask intended to deweight the ef-
fects of unmodeled occlusions and local illumination variations. To
compute this mask, an initial CTF image is needed, and the mask is
calculated using a standard linear correlation between the LNF image
and the initial CTF image (Figure 1(c)):
Cov; jj [Yrwr: Yer FJ
M7 pl ANA
cli, jl Var; ; [Y Np) Var, ilYerr]
(4)
in which Cov;,; and Var;,; are based in an image window, centered
at [4, j], of a predetermined size (8 x 8 in our experiments). In prac-
tice, the weighted-average algorithm is carried out iteratively (Sec-
tion 2.2), and in each iteration a new CTF image is used to calculate
MC. The initial CTF image is obtained by the first iteration, in which
only MF and MÇ are used.
A - 382
