If it is considered that at both information levels Scoring was
restricted to very gross identifications, then the error rate (even
the smaller rate of 25%) is high. The component-modifier method pro-
vided & more stringent score than did the component method, in that
both the component and modifier had to be correct for credit. Actually,
scoring at the component-modifier level cannot be considered refined,
for such scoring required identification only to the level of aircraft
bomber, for example, omitting such finer discriminations as weight,
number of engines, and whether or not it was a jet. If the component-
modifier level of precision is desired, then the scoring cannot be con-
sidered too stringent.
Completeness of Information Extraction. Another aspect of photo-
interpreter performance is the level of completeness of the information
extracted. An interpreter may provide information that is perfectly
accurate but it may be only a very small portion of the information to
be obtained from a given photograph. The 'ground truth" key does not
necessarily provide a fair reflection of extractability of information,
in view of the fact that Picture quality may not permit the extraction
of information on the objects actually photographed. To determine the
extent of completeness of information extracted, an "expectancy key! was
therefore devised. The 'expectancy key' reflects the total number of
different components (or component-modifier combinations) accurately
identified by any of the interpreters. A separate key was established
for each photographed target, on the assumption that those objects cor-
rectly identified by at least one interpreter are extractable.
Table 1
ACCURACY OF INFORMATION EXTRACTION IN PERCENT AT TWO
INFORMATION LEVELS: COMPONENT AND COMPONENT-MODIFIER SCORES
(FOR 16 EXPERIENCED PI's)
Component Component-Modifier
Target % right % wrong % right % wrong
1 81 19 45 55
2 75 25 56 Lh
3 63 ST 56 lj,
L 92 08 39 61
5 66 3h 5l. l6
6 Ol 16 Lo 51
T 95 15 61 39
8 65 35 16 5)
9 64 36 ll 59
Av. 75 25 50 20