A JOINT PIXEL AND REGION BASED MULTISCALE MARKOV RANDOM FIELD FOR 
IMAGE CLASSIFICATION 
     
  
Tiancan Mei* *, Lin Zheng? , Sidong Zhong* 
a School of Electronic Information Wuhan University, Wuhan, China - (mtc, 00200535, zsd)@whu.edu.cn 
Commission III, WG III/3 
KEY WORDS: Image segmentation, Region based MRF, Pixel based MRF, High resolution image, Multiscale analysis, Watershed 
transform 
ABSTRACT: 
MRF model is recognized as one of efficient tools for image classification. However, traditional MRF model prove to be limited for 
high resolution image classification. This paper presents a joint pixel and region based multi-scale MRF model for high resolution 
image classification. Based on initial image segmentation, the region shape information is integrated into MRF model to consider the 
pixel and region information simultaneously. The region shaped information is used to complement spectral signature for alleviating 
spectral signature ambiguity of different classes. The paper describes the unified multi-scale MRF model and classification algorithm. 
The qualitative and quantitative comparison with traditional MRF model demonstrates that the proposed method can improve the 
classification performance for regular shaped objects in high resolution image. 
1. INTRODUCTION 
Image classification is a process to categorize all pixels in an 
image into one of several predefined classes. Each pixel is 
assigned a class label based on the spectral signature of the 
pixel and the relationship with its neighbors. The performance 
of image classification is of paramount importance for many 
subsequent image based applications. Information extraction 
based on remote sensing image is an important application of 
image classification. Once the classification result is obtained, 
the categorized data can be used to produce thematic map of 
land cover present in the image. With the availability of high 
resolution remote sensing image, image classification has been 
applied in several diversity applications, such as road network 
extraction, urban planning and natural disasters prevention. 
The classification of moderate spatial resolution remote sensing 
image is mostly carried out based on the statistical separability 
of each classes (Melgani F., et al, 2000). The performance of 
pixel level classification is deteriorated when high resolution 
image is under processed. The improvement of spatial 
resolution increases the internal spectral variability within the 
same land cover classes and decreases the spectral variability 
between different classes. As a result, pixels will be 
misclassified due to the ambiguity between different classes. 
Despite the characteristic of high resolution image increase the 
complexity of classification problem, it provides geometrical 
information to be considered in the process of classification. So, 
the critical part of high resolution image classification is how to 
combine the spatial and spectral information to mitigate the 
spectral ambiguity between different classes. Previous work on 
high resolution image classification has been done by several 
authors. Unsalan (Cen Uslan, and Kim L Boyer, 2004) proposed 
a technique to identify different classes by using statistical 
property of extracted straight lines and spatial coherence 
constraint. Lee (Sanghoon Lee, Melba M. Crawford, 
2005)presents a multi-stage image classification method. This 
method first segments image by making use of spatial 
  
* Corresponding author. 
contextual information that characterizes the geophysical 
connectedness of image structure. Then the segment results are 
classified into distinct states by sequential region merging. 
Bruzzone (L.Bruzzone, and L.Carlin, 2006)proposes a 
supervised method to classify high resolution image. Adaptive 
multilevel spatial context driven feature extraction is first 
performed, and then the SVM classifier is used to obtain 
classification results. In (L. Zhang, et al, 2006), a pixel shape 
index is introduced to represent geometrical information and 
SVM is used to classify the image. Z.Lei(Z.Lei, et al, 2011) 
introduce the conditional texton forest to take use of spatial 
contextual information to perform the land cover classification 
for very high resolution image. Bellens(Rik Bellens, et al, 2008) 
uses morphological profile to extract the geometrical 
information as complement to spectral information. 
Bouziani(Mourad Bouziani, et al, 2010)proposes a rule based 
method for high resolution image classification by using 
spectral, geometric, and contextual information. 
The aforementioned previous work on high resolution image 
classification shows that the combination of geometrical 
information and spectral information are critical part of high 
resolution image classification technique. Although great efforts 
have been taken on this topic, few of the proposed methods can 
meet the end-user requirement on accuracy and efficiency. For 
this reason, there still remain many open issues that allow 
further investigation. 
Since the complexity of high resolution image classification, 
besides the geometrical information, it is necessary to take into 
account the contextual information about the image under 
analysis. The Markov random field(MRF) is a powerful tool to 
model the contextual information about the image. Bayesian 
approach is a simple, yet effective tool to integrate both the 
contextual information and statistical model of image in the 
classification process. In fact, the Bayesian approaches have 
been widely applied to perform image classification. 
At the early stage of MRF used for image classification, it is 
defined on the pixel level(Besag. J, 1986), because the 
neighboring relationship among pixels is regular on the 2D