«IMBiiìf* . ' :
AN UNSUPERVISED HIERARCHICAL SEGMENTATION
OF A FAÇADE BUILDING IMAGE IN ELEMENTARY 2D - MODELS
Jean-Pascal Burochin, Olivier Tournaire and Nicolas Paparoditis
Université Paris-Est, Institut Géographique National, Laboratoire MATIS
73 Avenue de Paris, 94165 Saint-Mandé Cedex, France
{firstname.lastname} @ign.fr
Commission III/3, III/4
KEY WORDS: street level imagery, façade reconstruction, unsupervised hierarchical segmentation, gradient accumulation, recursive
split, model matching
ABSTRACT:
We introduce a new unsupervised segmentation method adapted to describe façade shapes from a single calibrated street level image.
The image is first rectified thanks to its vanishing points to facilitate the extraction of façade main structures which are characterized
by a horizontal and vertical gradient accumulation which enhances the detection of repetitive structures. Our aim is to build a hierarchy
of rectangular regions bounded by the local maxima of the gradient accumulation. The algorithm recursively splits horizontally or
vertically the image into two parts by maximizing the total length of regular edges until the radiometric content of the region hypothesis
corresponds to a given model (planar and generalized cylinders). A regular edge is a segment of a main gradient direction that effectively
matches to a contour of the image. This segmentation could be an interesting tool for façade modelling and is in particular well suited
for façade texture compression.
1 INTRODUCTION
1.1 Context
Façade analysis (detection, understanding and reconstruction)
from street level imagery is currently a very active research do
main in the photogrammetric computer vision field. Indeed, it
has many applications. Façade models can for instance be used
to increase the level of details of 3D city models generated from
aerial or satellite imagery. They are also useful for a compact
coding of façade image textures for streaming or for an embed
ded system. The characterization of stable regions in façades is
also necessary for a robust indexation and image retrieval.
1.2 Related work
Existing façade extraction frameworks are frequently specialized
for a certain type of architectural style or a given texture appear
ance. In a procedural way, operators often step in a pre-process
to split correctly the image in suitable regions. Studied images
indeed are assumed to be framed in such a way that they exactly
contain relevant information data such as windows on a clean wall
background.
Most building façade analysis techniques try to extract specific
shapes/objects from the façade: windows frame, etc. Most of
them are data driven, i.e. image features are first extracted and
then some models are matched with them to build object hypothe
ses. Some other model-driven techniques try to find more com
plex objects which are patterns or layouts of simple objects (e.g.
alignments in ID or in 2D). Higher level techniques try to gener
ate directly a hierarchy of complex objects composed of patterns
of simple objects usually with grammar-based approaches. Those
methods generally devote their strategy to a special architectural
style.
1.2.1 Single pattern detection Strategies to extract shape hy
potheses abound in recent works. (Cech and Sara. 2007), for in
stance, propose a segmentation based on a maximum a posteriori
labeling. They associate each image pixel with values linked with
some configuration rules. They extract a set of non-overlapping
windowpanes hypotheses, assumed to form axis-parallel rectan
gles of relatively low variability in appearance. This restriction
does not take into account lighting variations. With a supervised
classification-based approach, (Ali et al., 2007) extracted win
dows width an adaboost algorithm. In the same fashion, (Wenzel
and Forstner, 2008) minimize user interaction with a clustering
procedure based on appearance similarity.
Assuming the regularity of the façade, (Lee and Nevatia, 2004)
use a gradient profile projection to locate window edges coordi
nates. They first locate valley between two extrema blocks of
each gradient accumulation profile and they roughly frame some
floors and windows columns. Edges are then adjusted on local
data information. Their results are relevant for façades whose
background does not contain any contours such as railings, bal
conies or cornices.
1.2.2 ID or 2D grid structures detection (Korah and Ras
mussen, 2007, Reznik and Mayer, 2007) use linear primitives to
generate rectangle hypotheses for windows. A Markov Random
Field (MRF) is then used to constrain the hypotheses on a 2D
regular grid. (Korah and Rasmussen. 2007) generate their rect
angular hypotheses in a similar way as (Han and Zhu. 2005):
they project on image 3D planar rectangles. (Reznik and Mayer.
2007) learn windows outline from training data and use as hy
potheses for window corners characteristic points.
1.2.3 Façade grammars A façade grammar describes the
spatial composition rules of complex objects (e.g. grid structure)
and/or simple objects to construct a façade. Approaches based
on grammars succeed in describing only façades corresponding
to the grammar. Nevertheless, to obtain a detailed description a
specific grammar is required per type of architecture (e.g. Haus-
manian in the case of Parisian architecture). The drawback is that
many grammars are necessary to describe the variety of building
architectures in a general framework.
