Figure 4: Images used for the radiance retrieval, from left and
right cameras, for 6 vehicle locations (all images displayed with
the same settings).
the sky radiance coming from all points of the upper hemisphere,
using images acquired by a ground-based mobile-mapping vehi-
cle, with an accurate geolocation, but affected with radiometric
artifacts. The method aims at working for all weather conditions.
This problem is ill-conditioned since the true signal due to sky
radiance and the noise signal due to radiometric artifacts cannot
be demixed.
3 SKY RADIANCE ESTIMATION
In this section, we propose two different approaches to estimate
the radiance map from the images: aggregation of pixels detected
as sky (including the clouds), and estimation parametric model
with blue sky pixels (avoiding clouds). These two methods can
be mixed in order to reduce the influence of artifacts and try to
enhance the accuracy of the sky radiance estimation. The corre-
sponding results are presented in section 4.1.
3.1 Extraction and aggregation of sky pixels
The first task is to extract the pixels that see the sky in the images,
in order to set them apart from pixels of building or vegetation.
We use simple thresholds on color values, set by experiments on
the data set. These thresholds must lead to a good balance be-
tween accepting clouds and rejecting highly illuminated walls.
The reflexions on cars are not a problem because pixels below
the horizon are not used, but reflexions on windows are usually
above the vehicle; a morphological filtering is used to remove
most of the small areas detected among the buildings. The thresh-
olds used are as follows, where the Cmin threshold depends on the
camera response and its bit depth :
R—B
MAX(R,G,B) — MIN(R,G.B)
<c, and MAX(R,G,B) > Cmin
(1)
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B3, 2012
XXII ISPRS Congress, 25 August — 01 September 2012, Melbourne, Australia
a) b) C)
Figure 5: Result of sky--clouds (a) and blue sky (b) detection.
The main errors remaining are overdetection of large windows
and of bright walls above the horizon, as well as a lack in detec-
tion of dark clouds. The sky areas affected by strong bloom are
also badly detected, because of their having a high red component
(figure 5).
The aggregation of the detected sky pixels is made by a simple
bilinear interpolation on a regular grid, averaging the values from
different images if they overlap. It leads to artifacts in the envi-
ronment map: discontinuities are visible between the areas ex-
tracted from different images, for the reasons mentioned in sec-
tion 2.2. Furthermore, there are holes in the environment map, for
the directions where no sky is visible or detected in any image;
this occurs principally for the areas close to the horizon. We limit
the extent of these holes by using images taken while the vehicle
goes through a crossroads, so we have a maximum solid angle of
visible sky. Then, an interpolation is performed to fill the holes.
This interpolation leads to more artifacts in the areas with no sky
pixels extracted (figure 8). The discontinuities can be reduced by
image processing techniques of blending, for instance by solv-
ing a Poisson equation, using only reliable pixels in the data term
(Bhat et al., 2008). However, the discontinuities mainly affect
the visual quality, and are not a problem in themselves for the
light simulation. The problem is to retrieve an environment map
with values closest as possible to the sky radiance at the time of
acquisition.
3.2) Estimation of a parametric model
Another possible approach is to use a parametric model to re-
construct the environment map. The most widely used model is
the Perez model (Perez et al., 1993), that uses 5 parameters a, b,
c, d, e to describe the fall-off of the light around the sun posi-
tion, plus 1 parameter L to set the zenithal radiance. However,
this model is limited to the description of low spatial frequency
phenomena, and therefore cannot model the clouds. The detec-
tion now avoids the clouds and extract only the background blue
sky pixels, by using hue and saturation thresholds proposed by
(Schmitt and Priese, 2009):
190° < hue < 240° and saturation > 0.2 (2)
The overexposed pixels, detected before the flat-field correction,
are rejected. The parameters estimation is then performed by
non-linear least square fitting, and gives coherent results as long
as the sky pixels extracted are scattered homogeneously in the
hemisphere. However, this estimation is affected by the radio-
metric effects mentioned in 2.2. Predominantly, the bloom effect
in the images for which the sun is close to their field of view,
leads to an overestimation of the values of the estimated Perez
model (figure 8).
€) T9 m |
medo .- ga Hé AA lan ag ya —"-4— CS M //7 Uu M o
