The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B7. Beijing 2008 
5. PROBLEMS IN THE IEEE GRSS 2006 DATA FUSION 
CONTEST 
5.1 Background 
Giga bites of testing data were made available to the contest 
participants for image fusion. The testing data contain two 
types of images: (1) QuickBird and (2) simulated Pleiades Pan 
and MS images. The Pleiades Pan images were simulated using 
green and red channels, which did not cover the designed 
spectral coverage of Pleiades Pan, 500-850 nm, analogously to 
Ikonos and QuickBird Pan (Alparone et al.,2007). The data 
volume of QuickBird images occupies over 80% of the total 
data volume provided to the participants for the contest. The 
fusion results generated by the participants were sent to one of 
two official contest judges, Dr. L. Alparone. 
5.2 Contest results 
The contest evaluation concluded that the fusion results of the 
Generalized Laplacian Pyramid Decomposition Featuring a 
Modulation Transfer Function Reduction Filter and a Context 
Based Decision Injection Rule (GLP-MTF-CBD), also called 
GLP-CBD, outperformed the other competing algorithms for 
most of the criteria [MB, VD, SDD, CC, SAM, ERGAS, and 
Q4] (Gamba et al.,2006). An IEEE Certificate of Recognition 
was granted to the GLP-CBD developers at the IEEE IGARSS 
2006 conference in August 2006. 
The paper on 2006 data fusion contest outcome (Alparone et al., 
2007), published in IEEE Transactions of Geoscience and 
Remote Sensing, provided results of quantitative analysis and 
visual evaluation. The visual analysis stated: 
• “GLP-CBD: Image is nice as a whole. Colors 
should be better synthesized. This would enhance the 
legibility of the image. Details are there, except for the 
most colored (blue, red). Errors in colors lead to 
interpretation errors. Contours should be sharper. There 
is no bias, except for Strasbourg outskirts. Unacceptable 
for detailed visual analysis.” 
• “UNB-Pansharp: Image is too noisy. There are 
many artifacts. Colors are not well synthesized as a 
whole and locally. Green trees are not green enough. 
Red or blue cars are absent. Shapes are not well defined; 
they are sometimes underlined by black lines. Too large 
bias is observed. There is lack of variance as a whole. 
At times, unacceptable. In best cases, unacceptable for 
detailed visual analysis.” 
5.3 Irregularity of the evaluation 
After the contest award in August 2006, numerous requests 
were sent to the contest committee for an opportunity to review 
some fusion examples by the contest participants. The requests 
were rejected and the participants were asked to wait for the 
publication of the paper on the contest outcome. Finally, the 
evaluation examples were provided to the participants for 
review in late December 2006. 
outcome paper in 2007 (Alparone et al.,2007) cannot be 
displayed here. However, readers can still see some difference 
by comparing the GLP-CBD QuickBird fusion result published 
in the IEEE GRSS Newsletter (Gamba et al.,2006) and that in 
the contest outcome paper (Alparone et al.,2007), even though 
In the reviewing of the fusion results used in the contest 
evaluation, it was found that the QuickBird fusion results 
produced by UNB-Pansharp were not evaluated in the contest, 
even though giga bytes of QuickBird fusion results of UNB- 
Pansharp were sent to the judge, Dr. L. Alparone, together with 
fusion results of the simulated Pleiades data. 
Two subsets of the UNB QuickBird fusion results are given in 
Figure 4. Readers can compare the original QuickBird Pan and 
MS images with the fusion results to evaluate whether the 
visual analysis of the IEEE fusion contest outcome by Alparone, 
et al. 2007 (see above) is objective, or not. Internal evaluation 
among the contest participants clearly agreed that the results 
produced by UNB-Pansharp are superior to those of GLP-CBD. 
Literature review after the fusion contest, especially after the 
publication of the contest outcome (Alparone et al.,2007), 
proves that the judge Dr. L. Alparone is also a co-developer/co- 
author of the top winning GLP-CBD algorithm (Alparone et al., 
2003; Aiazzi & Alparone et al., 2002; and Aiazzi & Alparone et 
al.,2006). 
A request for permission to use the GLP-CBD fusion results 
provided to the 2006 contest participants for publications was 
denied. The original answer to the participants is quoted below 
to avoid misinterpretation: 
“In particular, on the Quickbird fused image I noted few 
small areas where a proper spatial enhancement did not occur 
because of statistical instabilities of the adaptive injection 
model. Such fusion inaccuracies appear only in few small 
areas and cannot change the global evaluation of my 
algorithm. However, il [if] one extracts the misfused patches 
and compares only them with those of other algorithms, he 
might be erroneously lead to believe that GLP-CBD is not 
the best algorithm among those compared in the Contest. 
After the contest I realized of the inconvenience by watching 
the fused images in the DFC [data fusion contest] web site 
and I fixed it. On the other side, the DFC site should contain 
the images that were evaluated for the Contest and cannot be 
changed. Therefore, if you want use GLP-CBD fused data 
for any publications, I will be pleased to provide fused 
versions with the fixed algorithm, which performs identically 
to the earlier one except on the above mentioned small areas. 
So, I do not give you, or any other may request it, the 
permission of using the GLP-CBD fused data found in the 
DFC contest, because such data refer to the Contest only and 
do not reflect the current progress of my activity, as it should 
appear in an unbiased future publication.” 
In the comparison between the GLP-CBD QuickBird fusion 
results received by the contest participants in 2006 and that 
published in the IEEE contest outcome paper (Alparone et 
al.,2007), it is found that the GLP-CBD QuickBird fusion result 
in Alparone et al.,(2007) is clearly better than the one received 
by the participants—misfused patches and blurred areas are 
clearly reduced. 
Because the request for permission was rejected, the 
comparison between the GLP-CBD QuickBird fusion result 
used in the contest in 2006 and that published in the contest 
the images displayed are very small and do not cover the same 
area. To see the misfused patches, readers can see the GLP- 
CBD QuickBird fusion result published in the IEEE GRSS 
Newsletter (Gamba et al.,2006) and pay attention to the area 
circled in Figure 4 of this paper. The difference leads to a