ISPRS Commission III, Vol.34, Part 3A »Photogrammetric Computer Vision“, Graz, 2002 
a nxp matrix X. Similarly let y*, …, y4 denote the sub- 
jective features, representing the marks of the q evaluators, 
where ¢ = 10, when we also take the photo-interpreters 
votes into account. All the subjective data is organized in 
anxq matrix Y. 
To compare the two sets, we calculate linear combinations 
of the measurements in set 1 and set 2 : 
(€ ZNaz QıX +... + OpXP (15) 
n =Yb= by'+..+b,y? (16) 
where the projection vectors (a and b) are to be determined 
by maximizing the squared canonical correlation between 
€ and rj under the constraint that £ and 7 are unit norm vec- 
tors. £ and 7) are called canonical features. Yf the matrices 
X' X and Y'Y are invertible, one finds that b is the eigen- 
vector of the mattix M .— (Y'Y) 1!Y'X(X'X)1x'y 
related to the biggest eigenvalue, 82. Similarly, a is the 
eigenvector of.N — (X'X)-! X'Y(Y'Y)-1Y'X associ- 
ated with the same eigenvalue. When the first couple of 
canonical features (£1,771) has been found, we proceed then 
to the following couple (£25,772) so that their correlation is 
the next largest in order, while at the same time £1 and €» 
(respectively 71 and na ) have zero cross-correlation, and so 
on for £s, 1]3 ... From a geometrical point of view, on finds 
that (£1) is the cosine of the smallest angle between spaces, 
respectively generated by the columns of X and Y. The 
canonical analysis problem can be compared to the prob- 
lem of multiple regression (for more details see (Saporta, 
1990)). 
To choose r features among the p calculated, we used the 
redundancy criterion proposed by Thorndike (Tinsley and 
Brown, 2000). Let’s call as intraset loadings the correla- 
tions of the observed variables in set P with its canoni- 
cal features and as interset loadings the correlations of the 
observed variables in set P on the canonical components 
of set C. Drawing on principal component thinking, we 
can set an analogy between the eigenvalues of PCA and 
the sum of squared intraset loadings of the variables on 
a canonical component. The latter is the amount of vari- 
ance of the set that is accounted for by the canonical vari- 
ate of that set. This quantity, divided by the number of 
variables, produces the proportion of variance in the set 
that is accounted for by the component, denoted Vo;, for 
the j*^ component of set C. Let's recall at this moment 
that the squared canonical correlation (87) is the propor- 
tion of a canonical component's variance accounted for by 
the paired component in the other set. Therefore the pro- 
portion of variance in set P accounted for by the j^^ com- 
ponent of set C is: Redp, — Vp; * BE 
5 RESULTS 
5.1 PCA of the subjective features 
We formed a heterogeneous jury of 32 evaluators differ- 
ing in their expertise and familiarity with images. Some 
of them were not from image processing field (like secre- 
taries ...), others are doctoral students, technicians, pro- 
fessors, yet others were infrared or radar images domain 
experts. In addition each of 4 photo-interpreters have 
marked 2 groups of images. In the final analysis each im- 
age has ended up receiving 10 marks. They were summed 
up in 4 matrices (corresponding to the four groups defined 
in Section 4.1) of 40x8 elements. An element y? corre- 
sponds to the mark given by the jth evaluator (j = 1, ..., 8) 
to the 4^ image ($ = 1, ..., 40). 
Since not every image was marked by every evaluator, the 
PCA on the subjective features had to be carried out in 
groups. We have observed that the first PCA axis carries 
about 80 % of the inertia for all groups. It represents, in 
fact, the baseline of consensus of the evaluators. We should 
note that the data from the photo-interpreters has not taken 
place in this computation, but their mark vectors have been 
subsequently projected on the principal axes for validation 
and are also highly correlated with the first axis. The sec- 
ond component can be interpreted as portraying the differ- 
ential behavior of the evaluators, that is their tendency for 
severe or tolerant voting (a “severe” evaluator gives much 
easily a negative mark). In figure 5 we show the coordinate 
axes ul, u2 of the first two largest eigenvectors of group |, 
and the projection of the group's mark vectors. All three 
groups 1, 3, 4 have similar projections, whereas the group 2 
behavior differs on the vertical axis. One explanation could 
be that this group 2 has a relatively larger proportion of in- 
experienced evaluators. Indeed the second component axis 
for this group seems only differentiating inexperienced and 
experienced evaluators. Let's note that the principal axes 
do not have necessarily the same interpretation from group 
to group. 
Figure 5: Projection of mark vectors of the 8 evaluators 
(v1 to v8) and the 2 experts (v9 and v10) for group 1 on 
the principal PCA axes. 
5.2 Canonical analysis of the objective and subjective 
features 
As in the preceding PCA case of subjective scores, we im- 
plement the canonical analysis per group. Our goal is to 
find the subset of features which is the most correlated with 
the baseline of consensus of the evaluators found with the 
PCA. 
As we do not know a priori the cardinality r of the subset, 
we carry out an exhaustive canonical analysis. For each 
value r € [1, p], we try all possible combinations of r fea- 
tures chosen among p, and we run the CA on this subset 
A - 203