In: Paparoditis N., Pierrot-Deseilligny M.. Mallet C. Tournaire O. (Eds), IAPRS. Vol. XXXVIII. Part ЗА - Saint-Mandé. France. September 1-3. 2010 
A FORMULATION FOR UNSUPERVISED HIERARCHICAL SEGMENTATION 
OF FACADE IMAGES WITH PERIODIC MODELS 
Jean-Pascal Burochin, Bruno Vallet, Olivier Tournaire and Nicolas Paparoditis 
Université Paris-Est. Institut Géographique National, Laboratoire MAT1S 
73 Avenue de Paris. 94165 Saint-Mandé Cedex, France, {firstname.lastname}@ign.fr 
Commission II1/4 
KEY WORDS: street level imagery, facade reconstruction, unsupervised hierarchical segmentation, gradient accumulation, recursive 
split, model matching, periodicity detection 
ABSTRACT: 
We introduce an unsupervised segmentation method to build a hierarchical representation of a building facade from a single calibrated 
street level image. The process recursively splits horizontally or vertically the rectified image along dominant alignments until the 
radiometric content of the region hypothesis corresponds to a given model. This paper propose two main novelties: first we describe 
an advanced split energy formulation to separate dominant alignments breaks. Then we introduce a model that express periodicity in 
facade texture. This segmentation could be an interesting tool for facade modeling and is in particular well suited for facade texture 
compression. 
1 INTRODUCTION 
1.1 Context 
Facade analysis (detection, understanding and field of reconstruc 
tion) from street level imagery is currently a very active field of 
research in photogrammetric computer vision due to its many ap 
plications. Facade models can for instance be used to increase 
the level of details of 3D city models generated from aerial or 
satellite imagery. They also are useful for a compact coding of 
facade image textures for streaming or for an embedded system. 
The characterization of stable regions in facades is also necessary 
for robust indexation and image retrieval. 
We work exclusively on a single calibrated street-level image. We 
voluntarily did not introduce additional information such as 3D 
imagery (point clouds, etc.) because for some applications such 
as indexation, image retrieval and localization, we could just have 
a single photo acquired by a mobile phone. 
1.2 Previous work 
Existing facade extraction frameworks are frequently specialized 
for a certain type of architectural style or a given texture appear 
ance. In a procedural way, operators often step in a pre-process 
to split correctly the image into suitable regions. Studied images 
indeed are assumed to be framed in such a way that they exactly 
contain relevant information data such as windows on a clean wall 
background. 
Most building facade analysis techniques try to extract specific 
shapes/objects from the facade: windows frame, etc. Most of 
them are data driven (Ali et al., 2007, Lee and Nevada, 2004, 
Haugeard et ah, 2009), i.e. image features are first extracted and 
then some models are matched by them to build object hypothe 
ses. Some other model-driven techniques such as (Korah and 
Rasmussen. 2007). (Reznik and Mayer, 2007) or (Han and Zhu, 
2005) try to find more complex objects which are patterns or lay 
outs of simple objects {e.g. alignments in ID or in 2D). Higher 
level techniques such as (Alegre and Dellaert, 2004). (Müller et 
ah, 2007) and (Ripperda, 2008) try to generate directly a hierar 
chy of complex objects composed of patterns of simple objects 
usually w ith grammar-based approaches. Those methods gener 
ally devote their strategy to a special architectural style. 
Finally, (Burochin et ah, 2009) propose a facade segmentation 
independent of its architectural style. This framework first sepa 
rates a facade from its background and neighboring facades, and 
then identifies intra-facade regions of specific elementary texture 
models that all facades have in common. A recursive segmenta 
tion is applied only directed by dominant alignments. Neighbor 
ing facades and main intern shapes are correctly separated with 
out any semantic a priori. But detected models do not concents 
repeated structures, that are tipical properties of man-made ob 
jects. such as described in (Wenzel et ah, 2008). 
1.3 Contribution 
Most of the aforementioned approaches are specialized in a par 
ticular kind of architecture. Few of them have addressed very 
complex facade networks such as the ones encountered in Eu 
ropean cities where the architectural diversity and complexity is 
large (Hausmannian buildings for instance or other complex ar 
chitectures with balconies or decoration elements). Our work is 
upstream from most of these approaches and improves on the re 
cent framework proposed by (Burochin et ah, 2009) (our contri 
bution is mentioned in red on figure 1). We summarize this pre 
vious approach in section 2. Section 3 explains our new split en 
ergy formulation that better separates dominant alignment breaks. 
Then section 4 introduces a third model: the periodic one. We 
eventually present some results in section 5 and we discuss them. 
2 GENERAL STRATEGY 
In this section we describe the general strategy of (Burochin et ah, 
2009). This strategy requires horizontal and vertical image con 
tour alignments. Thus images first are rectified in the facade plane 
using two vanishing points extracted as described in (Kalantari et 
ah, 2008). The segmentation relies on a recursive split process 
and on a model based analysis of each subdivided regions. If the 
considered region does not match any of the proposed models, it 
is split into tw'o sub-regions w'hich are later analyzed as illustrated