In: Wagner W., Székely, B. (eds.): ISPRS TC VII Symposium - 100 Years ISPRS, Vienna, Austria, July 5-7, 2010, IAPRS, Vol. XXXVIII, Part 7B 
651 
Currently, over 30 models have been registered in the ROMC 
and several scientific publications use ROMC-generated graphs 
to provide independent and traceable proof of the quality of a 
canopy RT model. Of particular interest here is the fact that the 
ROMC provides an indication of model skill, defined as: 
Skill = 100- 
(1 + Ry 
(- 
■*mod 
vV 
+ 
l ref 
mod y 
^ mod 
V ^ re f 
+ 
<J, 
ref 
mod J 
where R is the correlation coefficient, X is the mean value, and 
O is the standard deviation of N simulations provided by the 
candidate model ( mod ) and ROMC reference data set ( re/ ), 
respectively. The skill metric depends on N and reaches 100 for 
a perfect match with the ROMC reference data. 
With the availability of the ROMC it became feasible for RAMI 
to address new issues. As such, RAMI proposed to utilize 
‘credible’ 3D Monte Carlo models to provide benchmark 
solutions against which the shortwave radiative flux 
formulations in the land surface schemes of SVATs and GCMs 
could be evaluated. This proposal was endorsed during the first 
Pan-GEWEX meeting in late 2006 and led to the launch of the 
RAMI4PILPS suite of experiments in 2008 (where 10 
modelling groups from around the world participated). In 
parallel, the fourth phase of RAMI was launched in 2009 with a 
completely new set of test cases, some of which, were based on 
detailed field inventories and exhaustive in-situ and laboratory 
measurements of actual forest stands. In addition, RAMI-IV 
expanded also the range of model simulations beyond that of 
passive optical space sensors to include also waveform LiDAR 
instruments and devices typically used during in-situ validation 
campaigns of remotely sensed products. Figure 2 provides an 
overview of the evolution of the RAMI activity with depictions 
of the canopy architecture of various test cases. 
The strategy of RAMI benefited 1) model developers, who were 
able to debug their software codes and receive indications as to 
where future development efforts were most needed, 2) users of 
canopy RT models, who can now make better choices regarding 
the selection of canopy RT models, and 3) the RT modelling 
By its very nature, RAMI and the I3RC (its sister activity 
dealing with clouds: http://i3rc.gsfc.nasa.gov/), are both 
dynamic and evolving activities. As a result, the benchmarks, 
reference data sets and evaluations issued by the RAMI process 
must be considered snapshots describing the state of the art at 
the time of the exercise, and not as a final, absolute and 
definitive judgment on the worthiness and performance of any 
particular model. In fact, it is through its systematic approach to 
RT model verification that RAMI contributes to the quality 
assurance of space derived information. 
3. OPPORTUNITIES AND CHALLENGES 
3.1 Expanding the scope of RAMI 
Through its systematic benchmarking efforts the various phases 
of RAMI have allowed to 1) identify ‘credible’ canopy RT 
models, 2) generated ‘surrogate truth’ reference data sets, 3) 
automate the model verification process via quasi-real time 
web-based benchmarking facilities, and 4) gradually increased 
the complexity and realism of the simulated plant environments. 
This allows RAMI to envisage the expansion into new thematic 
areas (soils, coastal zones, urban areas), spectral regions 
(thermal, SAR), and specific instruments (both space and in-situ 
based). Similarly, the benchmarking of RT model simulations 
under truly ‘controlled experimental conditions’ - such as are 
nowadays achievable in reference laboratory facilities - should 
be addressed in the future. This would both strengthen the 
credibility of the 3-D Monte Carlo models that were used to 
generate the ROMC reference dataset, and also enable the set 
up of a traceable quality assurance system to relate the 
performance of simpler canopy RT models - via the above 3-D 
Monte Carlo models - to a series of absolute reference 
standards of the real (as opposed to virtual) world. 
Ultimately, however, it is the accuracy of the retrieved state 
variable values that counts in many RT model applications. 
During RAMI-1 it had already been proposed to address the 
inversion of RT models against predefined sets of spectral and 
angular observations, similar to those provided by the current 
fleet of space borne sensors. In this way, it was hoped, that in 
this manner the impact of the various structural and radiative 
canopy model assumptions could be thoroughly assessed since 
assess shortwave 
RAMI-1 RAMI-2 RAMI-3 ROMC RAMI4PILPS 
(1999) (2002) (2005) (2007) (2008) 
realistic scenes 
RAMI-IV 
(2009) 
Figure 2. Evolution of the RAMI initiative, 
community which, through its continuing support and active 
encouragement of RAMI, was able to increase its visibility and 
maturity. About 60-65% of all currently existing canopy RT 
models have voluntarily participated in the RAMI initiative. 
the uncertainties of the available surface BRFs were known a 
priori. This approach had to be abandoned though due to a 
sever lack of participants. Now, with the identification of 
‘credible’ canopy RT models new sensor-specific (top-of-