In: Wagner W„ Szbkely, B. (eds.): ISPRS TC VII Symposium - 100 Years ISPRS, Vienna, Austria, July 5-7, 2010, IAPRS, Vol. XXXVIII, Part 7B
recently developed operational MODIS Collection 005 retrieval
algorithm. Results obtained over the entire globe during the first
six months of year 2005 showed that the proposed ensemble of
neural networks was significantly more accurate for all the
considered accuracy measures (Radosavljevic et al, 2010a).
3.3 Uncertainty Analysis of AOD Retrieval
Three years of MODIS data collocated with 201 AERONET
sites over whole the globe have been used for uncertainty
analysis of AOD retrieval. The average negative log-predictive
density (NLPD) of the true targets was used as a measure of the
quality of uncertainty estimation. Committees of 30 neural
networks were trained on the subset of sites which have data in
both training and testing years. The obtained results allowed
analysis of uncertainty of AOD retrieval at a given site over
time and also uncertainty comparison at multiple sites. As an
example, we compared uncertainty of AOD retrieval at Beijing
site in China vs. Muana Loa site in Hawaii to conclude that
properties of aerosols are much more stable at Muana Loa than
in Beijing. By further investigation we found that this discovery
is consistent with domain experts’ expectations as this site
serves for calibration statistics of AERONET instruments.
Our analysis of seasonal uncertainty levels over three years also
revealed existence of different interesting patterns. For example,
we compared sites with the highest and the lowest average
uncertainty over the seasons. The highest uncertainty levels
occur in Asia over all seasons, in Africa during the winter and
fall, and in the central part of South America during the
summer. These levels reach extreme values in summer while for
other seasons are almost equal. On the other hand, the lowest
levels of uncertainty appear in North America and Europe
during winter, summer, and fall, and in South America during
the spring (Ristovski et al, 2009).
3.4 Selection of Sites for Ground Based Observations
Data used in sites reduction experiments were distributed over
entire globe at 217 AERONET sites during years 2005 and
2006. We performed training on 2005 data and used 2006 data
for testing. We considered a scenario when current operational
AERONET sites have to be reduced by 33% or 66%. In all
experiments, we started from a set of 30 AERONET sites and
applied the proposed method and the two alternatives to identify
a subset of 20 or 10 AERONET sites to be retained. The two
alternatives included a random selection of sites as well as an
approach based on spatial distance among the sites was also
considered. Sites were selected such that their spatial coverage
was maximized. In our experiments, the proposed accuracy-
based selection achieved consistently better results than the
alternatives. Also, accuracy of the proposed site reduction
method did not change much even after removing 20 of the 30
AERONET sites. Interestingly, on average, the spatial selection
strategy performed slightly worse than the random selection
strategy. According to presented results we concluded that the
proposed accuracy-based sites reduction method is superior to
spatially-based and random selection alternatives
(Radosavljevic et al, 2009).
Our extensive experiments on globally distributed data over 90
AERONET sites from the years 2005 and 2006 provide strong
evidence that sites selected using the algorithm proposed in
Section 2.5 improve the overall AOD prediction accuracy at a
faster rate than those selected randomly or based on spatial
diversity among sites (Das et al, 2009). The evaluation data
spanned the entire world, ranging from January of 2005 to
December of 2006. A committee of 20 neural networks was
used to estimate the model uncertainty, each having 10 hidden
nodes. Initial training set was created with 700 training points
from 10 randomly selected AERONET sites. We assumed that
the AOD values corresponding to the remaining training points
are unknown. Then we proceeded to select t = (1, 2, ... 20}
sites in twenty independent experiments using the proposed site
selection algorithms. The R-squared accuracy was computed on
the test data before and after the selection of prospective sites.
Our results show that uncertainty-based site selection gives
significantly higher accuracy over random and spatial distance-
based selection (especially when only a few sites are to be
selected) and marginally higher accuracy than temporal
distance-based method. Performance improves further if
correlation among the sites is taken into consideration. A
comparison between accuracies uncertainty-based and all three
other methods which selects site based on combination of
uncertainty, spatial and temporal distance shows that there is
some improvement in performance due to inclusion of spatial
and temporal correlation metrics over purely uncertainty-based
selection. Although the improvement in accuracy is small, it is
not negligible, keeping in mind the huge variability of AOD
over the entire earth.
At the continent scale, we observed that different continents
favor different selection algorithms. For Europe, random
selection works quite well because a large number of unlabeled
sites are from Europe and therefore random selection favors
sites from Europe. In North America, despite the large number
of candidate sites, the random selection was not very successful.
The accuracies for North America generally followed the
accuracies obtained on overall test set. In South America and
Africa, spatial selection performed better than other methods.
Especially in South America, it was able to attain a significantly
higher accuracy (Das et al, 2009)
3.5 Discovering Correctable AOD Retrieval Error
Our approach described in Section 2.5 was applied on 3,646
collocated MODIS and AERONET observations within the
continental United States. The results showed that neural
networks are more accurate than the operational MODIS
algorithm over the observed locations. A study of differences
between neural networks and the MODIS algorithm revealed
interesting findings. For example, NN are more accurate than
MODIS when the retrieval is contaminated by clouds, snow, or
water (i.e. NDVI <= 0). These and other discoveries have been
found to be mostly consistent with expert knowledge and
revealed some new insights into the MODIS algorithm
performance (Vucetic et al, 2008).
3.6 Unifying Multiple Retrievals by Structural Regression
Experiments were conducted on MODIS data at 50x50 km 2
resolution collocated with observations at 217 AERONET sites
during years 2005 and 2006. In nested spatio-temporal 5-cross
validation experiments we determined parameters
corresponding to the operational C005 and neural network
(.NN) retrievals over five continents (Asia and Australia were
treated as a single region). In our experiments CRF achieved
significantly better accuracy than either NN or C005 alone.
In follow-up experiments when using indicator functions for 5
continents (Asia and Australia were treated as one continent)
CRF achieved better accuracy than either NN or C005 alone