size of the 
ize of those 
arni 
    
Map 
f 
t Neuron 
[pixel] 
[pixel] 
  
  
Self-organizing map e 
of eyebrow 
  
number of 
eyebrow 
css m dil (re? | Victory neuron 
  
  
    
   
Self-organizing map ET a T 
itp] li ALLL 
       
if Tr Victory neuron 
of eye bir ki hf =; + — - - | number of eye 
i SA di 
  
  
Self-organizing map ESSET 
of mouth 
  
Victory neuron 
number of mouth 
  
  
» Time 
(a) The movements of the victory neuron for each facial segment 
uinouJ 
jo uouneu ÁJ0]9IA 
  
(b) The movements when strongly expressed 
0 
uinouJ 
jo uouneu ÁJ039IA 
     
ae 20 
10 nn 
e ^, 40 0 euro 
Yo "mpg "ion Moyen 
rà 
(c) The movements when strongly expressed 
Figure 6 The Classification of Facial Expressions 
  
[Map of Mouth [8040 [pixel] | 401 [pixell | 
5. RESULTS 
Figure 4 shows the self-organized maps used for 
recognition. Figure 5 shows the movement of victory 
neuron when the same kinds of facial expressions are 
input. The tracks of the victory neuron have similar 
shapes for three times. Figure (a) shows the movements 
of the each victory neuron. Figure (b) shows the 
movements of victory neuron in 3-dimensional space. 
Figure (c) shows the movements of the victory neuron 
while the facial expression was strongly expressed. 
Figure 6 shows the movement of victory neuron when 
different kinds of facial expressions are input. The tracks 
of the victory neuron have features peculiar to each facial 
expression. 
6. DISCUSSION 
6.1 Validity of Recorded Facial Expressions 
The facial expressions we recorded in this experiment 
ware based on the emotions we asked our subjects to 
express. To confirm that the facial expressions 
accurately expressed the emotions we've wanted, we sent 
questionnaire to 15 subjects. The latter ware instructed 
to watch a video image of each facial expression and facial 
expression they are watching. Table 3 summarizes our 
findings. Since four out of six types of emotions we 
interested (happiness, anger, surprise, fear) got high 
recognition rates, we concluded that emotions except 
sadness and fear ware effectively expressed. The results 
of our classification using self-organizing maps suggest 
that sadness and disgust are placed in the same category. 
The facial expressions of sadness ware classified by 
nether the self-organizing map or our human subjects. 
This suggests that sadness is a difficult emotion to express, 
and that facial expressions of sadness are not very 
pronounced. To test these hypotheses, we will need to 
develop better ways to record facial expressions. 
Table 3 The results of questionnaires 
  
  
  
  
  
  
  
  
  
The Kinds | Recognition rate of facial expressions[%] 
of Facial Subjects 
expression A B C D Average 
Happiness 100 100 100 60 90 
Anger 47 47 53 27 43 
Surprise 93 80 87 60 80 
Disgust 67 80 47 67 65 
Fear 20 33 20 0 18 
Sadness 13 20 27 13 18 
  
  
  
  
  
  
  
  
6.2 The Learning Ability of Self-organizing Maps 
The possibility of classifying facial expressions depends 
on how well self-organizing maps are learned. As shown 
in Figure 3, in some cases neurons placed far from in the 
output layer have similar features. When features are 
extracted using self-organizing maps, although image 
sequences are hardly changing, there are some cases 
where the movements of the victory neurons are so large 
that image sequences are perceived to be changing. 
Thus it is difficult to estimate changes image sequences 
from changes in the victory neuron. This problem might 
447