58 
A SUPER RESOLUTION RECONSTRUCTION ALGORITHM TO MULTI-TEMPORAL 
REMOTE SENSING IMAGES 
Pingxiang Li a, *, Jixian Zhang b, Huanfeng Shen c, Liangpei Zhang a 
The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan 
University, Wuhan, Hubei, China, 430079-pxli@lmars.whu.edu.cn, zlp62@public.wh.hb.cn. 
b Chinese Academy of Surveying and Mapping- zhangjx@casm.ac.cn 
c School of Resource and Environmental Science, Wuhan University-shenhf@whu.edu.cn 
KEY WORDS: Super resolution, Remote Sensing, Multi-temporal images, Registration, Joint framework, Maximum a posterior. 
ABSTRACT: 
In this paper, we propose a super resolution (SR) image reconstruction algorithm to multi-temporal remote sensing images. The aim 
is to reconstruct a high resolution (HR) image by fusing the non-redundant information among the low resolution (LR) remote 
sensing images which captured on different dates by the same sensor. To increase the robustness of the image registration and SR 
reconstruction, we combine the two processes together using the maximum a posterior (MAP) framework. In the solution procedure, 
a cyclic optimization method is employed to solve the desired SR image, registration parameters and outliers which strongly depart 
away from the observation model. We test the proposed algorithm using real multi-temporal MODIS images. The experimental 
results and comparative analyses verily the effectiveness of this algorithm. 
gk = 
where I is the 
respectively the ga 
which balance the 
atmosphere conditit 
3. JOINT IV 
It has been prov< 
estimation) and Si 
Barnard et al. 199' 
al. 2007), and a 
implement the im< 
joint framework. I 
parameter vector 
m = {m v m 2 ,....m i 
posteriori (MAP) 
registration paramc 
Z,n 
Applying Bayes’ ru 
z,m = arg 
1. INTRODUCTION 
Image fusion is the combination of two or more different 
images to form a new image by using a certain algorithm. In the 
field of remote sensing, image fusion technique is commonly 
employed for sharpening low resolution (LR) images using one 
or more high resolution (HR) images, such as the fusion of 
multi-spectral images with SAR or panchromatic image. As a 
special case of image fusion, super resolution (SR) image 
reconstruction refers to a process that produces a HR image 
from a sequence of LR images using the non-redundant 
information among them. Generally, the non-redundant 
information roots in the sub-pixel displacements among the 
observed images. 
The multi-frame SR problem was first formulated by Tsai and 
Huang (Tsai and Huang 1984) in the frequency domain using 
discrete Fourier transform (DCT) . Consequently, many kinds 
of spatial domain approaches have been developed. Typical 
spatial domain approaches include non-uniform interpolation 
(Ur and Gross 1992), iterative back projection (IBP) (Irani and 
Peleg 1991), projection onto convex sets (POCS) (Stark and 
Oskoui 1989; Tekalp, Ozkan et al. 1992; Patti, Sezan et al. 1994; 
Patti, Sezan et al. 1997), maximum likelihood (ML) (Tom and 
Katsaggelos 1994), maximum a posteriori (MAP) (Schultz and 
Stevenson 1996) (Hardie, Tuinstra et al. 1997), hybrid ML 
/MAP/POCS (Elad and Feuer 1997), and adaptive filtering 
(Elad and Feuer 1999). 
Although the SR technique has been greatly developed in the 
last decades, it has been applied to generic camera images, 
medical images and video sequence much more commonly than 
remote sensing images. Although some papers in the literature 
provided the SR results of satellite images, most of them used 
synthetic images and assumed known motion parameters. Shen 
et al. (Shen, Ng et al. 2007, accepted) proposed a super 
resolution image reconstruction algorithm to multi-temporal 
remote sensing images. They implemented image registration 
and SR reconstruction separately. This paper extends the 
previous method by employing a joint MAP framework to 
simultaneously estimate the image registration parameters and 
the HR image. This algorithm reinforces the interdependence 
among the motion estimates and HR image in a mutually 
beneficial manner. 
2. IMAGE OBSERVATION MODEL 
For the SR problem, a typical image observation model 
assumes that imaging process involves warping followed by 
blurring and down-sampling to generate LR images from a HR 
image. Let us denote the underlying HR image in vector form 
by Z- [z x ,Z 2 ....,Z LyN ^ LiNi f , where L x N x xL 2 N 2 is the HR 
image size. Letting Lj and L 2 denote the down-sampling 
factors in the horizontal and vertical directions respectively, 
each observed LR image is of size A, x N 2 • Thus, the LR image 
can be represented as g k =[g kA ,g kX ...,g k ^ N J < 
where k = 1,2,.... P, with P being the number of the LR images. 
The typical image observation model can be represented as 
(Elad and Feuer 1997; Park, Park et al. 2003): 
g k =DB k M k z + n k (1) 
where M k is the warp matrix, B k represents the blur 
matrix, D is a down-sampling matrix, and n k represents the 
noise vector. 
In remote sensing imaging, the sun zenith angle and 
atmospheric absorption and scattering affect the amount of 
radiance received by the sensor. These effects in many cases 
can be simply modeled as a linear system (Conel 1985; Roberts, 
Yamaguchi et al. 1985), with which the following image 
observation model can be obtained 
* Corresponding author. 
= ari 
Since p(z,m\g) h 
rewritten as 
Z,m = argm 
Z, 
Assuming the mode 
and variance a 2 , th 
be given by 
P(gi\z,nh) = TT e 
where O k is a diag 
outliers that strong 
order to preserve 
reconstructed imag 
for p{z). The Hube 
Stevenson 1996) 
p(z)= l 
where C is a const 
clique within the 
d c ( z x ,y) is a spati 
often formed by 
and p(-) is the poter 
Pi 
where p is a thresh 
linear regions (S