The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B7. Beijing 2008 
1109 
allowed digital number range of the file) do not change 
image quality for remote sensing applications. 
• Visual evaluation results can be strongly 
influenced by image display conditions. The same image 
can be interpreted as having different qualities, if the 
display conditions are not the same. Therefore, it is 
important to assure a consistent display condition for 
images compared to achieve a convincing visual 
comparison result. 
• Significant disagreement exists in the 
quantitative measurements of the seven indicators. 
Images having the same quality for remote sensing 
applications are indicated as having significant quality 
difference. This proves that the indicators are not 
capable of providing convincing image similarity 
measurements. 
In the image fusion quality evaluation by Alparone et al.,(2004), 
the SYN result is clearly the best according to the Q4, SAM and 
CC measurements, as well as the visual comparison. Although 
the SYN result does not have the best ERGAS value, it should 
not be overly concerned because according to Alparone et 
al.,(2004) ERGAS failed in measuring spectral distortion. 
However, in the final ranking of Alparone et al.,(2004), the 
authors’ fusion algorithms GLP-SDM and GLP-CBD were 
ranked as the best, instead of the SYN results. This 
demonstrated that the authors themselves did not trust the 
measurement values, and personal preference played an 
important role in the ranking. 
In the fusion quality evaluation of the IEEE GRSS 2006 Data 
Fusion Contest, the inconsistency and irregularity of the 
evaluation has suggested the difficulty of using the seven 
quantitative indicators to provide convincing quality 
measurements. Otherwise, there would have been no need to be 
selective in the contest evaluation for showing that the judge’s 
GLP-CBD algorithm was the best and first class in the fusion 
contest, and the obvious, misfused patches or areas would have 
been detected. 
In conclusion, the discrepancy between the visual evaluations 
and quantitative analyses in the three cases discussed in this 
paper demonstrate that the seven quantitative indicators (MB, 
VD, SDD, CC, SAM, ERGAS, and Q4) cannot provide reliable 
measurements for quality or similarity assessment between 
remote sensing images. 
ACKNOWLEDGEMENTS 
The author thanks Mr. Z. Xiong and Mr. J. D. Mtamakaya for 
their kind support in data and material preparation. The author 
also thanks the City of Fredericton, NB, Canada, for providing 
the original Ikonos Pan and MS images, and the IEEE GRSS 
2006 data fusion contest committee for the original QuickBird 
Pan and MS images. 
Aiazzi, B., L. Alparone, S. Baronti, A. Garzelli, and M. 
Selva,2006. MTF-tailored multiscale fusion of high- 
resolution MS and Pan imagery. Photogrammetric 
Engineering and Remote Sensing, Vol. 72, No. 5, pp. 591— 
596. 
Aiazzi, B., L. Alparone, S. Baronti, and A. Garzelli„2002. 
Context-driven fusion of high spatial and spectral 
resolution data based on oversampled multiresolution 
analysis. IEEE Transactions on Geoscience and Remote 
Sensing, Vol. 40, No. 10, pp. 2300-2312. 
Alparone, L., B. Aiazzi, S. Baronti, A. Garzelli,2003. 
Sharpening of very high resolution images with spectral 
distortion minimization. Proceedings of 2003 IEEE 
International Geoscience and Remote Sensing Symposium 
(IGARSS 2003), pp. 458- 460. 
Alparone, L., B. Aiazzi, S. Baronti, A. Garzelli, and P. 
Nencini„2004. A Global Quality Measurement of Pan- 
Sharpened Multispectral Imagery. IEEE Geoscience and 
Remote Sensing Letters, Vol. 1, No. 4, October 2004. pp. 
313-317. 
Alparone, L., L.Wald, J. Chanussot, C. Thomas, P. Gamba, L.M. 
Bruce,2007. Comparison of Pansharpening Algorithms: 
Outcome of the 2006 GRS-S Data-Fusion Contest. IEEE 
Transactions on Geoscience and Remote Sensing, Vol. 45, 
No. 10, Oct. 2007, pp. 3012-3021. 
Buntilov, V. and T. Bretschneider,2000. Objective Content- 
Dependent Quality Measures for Image Fusion of Optical 
Data. International Archives of Photogrammetry and 
Remote Sensing, Vol. 33, 2000. 
Gamba, P., J. Chanussot, and L. M. Bruce,2006. TECHNICAL 
COMMITTEE REPORTS: Contest Organized by the Data 
Fusion Technical Committee at IGARSS 2006. IEEE 
Geoscience and Remote Sensing Society Newsletter, 
December 2006, pp. 11-16. 
Ji, L., and K. Gallo,2006. An Agreement Coefficient for Image 
Comaparison. Photogrammetric Engineering and Remote 
Sensing Journal, Vol. 72, No. 7, pp. 823-833. 
Li, J.,2000. Spatial Quality Evaluation of Fusion of Different 
Resolution Images. International Archives of 
Photogrammetry and Remote Sensing, Vol.,33, 2000.Piella, 
G., and H. Heijmans,2003. A new quality metric for image 
fusion. Proceedings of IEEE International Conference on 
Image Processing, Vol. 3, pp. 173-176. 
Wald, L., T. Ranchin, and M. Mangolini,1997. Fusion of 
satellite images of different spatial resolutions: Assessing 
the quality of resulting images. Photogrammetric 
Engineering and Remote Sensing, Vol. 63, No. 6, pp. 691— 
699. 
Wang, Z, D. Ziou, C. Armenakis, D. Li, and Q. Li,2005. A 
Comparative Analysis of Image Fusion Methods. IEEE 
Transactions on Geoscenc and Remote Sensing, Vol. 43, 
No. 6, pp.1391-1402. 
Wang, Z., A.C. Bovik, H. Sheik, and E. Simoncelli,2004. Image 
quality assessment: From error visibility to structural 
similarity. IEEE Transactions on Image Processing, Vol. 
13, No. 4, pp. 600-612. 
Wang, Z., and A.C. Bovik,2002. A Universal Image Quality 
Index. IEEE Signal Processing Letters, Vol. 9, No.3, 
pp.81-84. 
Willmott, C. and K. Matsuura,2005. Advantages of the mean 
absolute error (MAE) over the root mean square error