and peanut
recorded M-
tion of the
vided into 2
| phases were
ere from no.
sis to say.
ncluded that
as to help
r one was to
cue and the
sorting out
n phase ( BF
t regularly,
-phase with
phase showed
which was
Then, we
: of crushed
ncluded that
id mandible
rogress to
inction was
he change of
À text-book
pp. 97-156.
y the tongue
at, 89, pp.
nfluences of
S in humans.
oodstuffs of
nvestigation
|l pp 79-91.
'ogestutzte
genfunktion
'orthop, 54,
11trasound
swallowing.
International Archives of Photogrammetry and Remote Sensing. Vol. XXXII, Part 5. Hakodate 1998
RECOGNITION OF BASIC ACTIONS FOR THE DETECTION
OF THE PERPLEX SITUATIONS USING BOTH PUPIL POSITIONS
Takayuki KAMITANI and Yoji MARUTANI
Assistant Professor, Professor
Department of Information Systems Engineering
Osaka Sangyo University
Nakagaito 3-1-1, Daito-shi, Osaka, 574-8530
E-mail: kamitani@ise.osaka-sandai.ac.jp, marutani@ise.osaka-sandai.ac.jp
JAPAN
Comission V, Working Group 4
KEY WORDS: DP Matching, Image Sequence, Motion Analysis, Perplex Situation, Perplexed Behavior
ABSTRACT
This paper presents a method for recognizing the perplex situations based on the detection of basic actions by obtaining
positions and an angle of pupils in the image sequence.
Firstly, the perplexed behaviors were picked up by observing the subjects in word processor work. As the result of the
observation, it was made clear that the perplexed behaviors were shown in the motion of the head and the keyboard
operation rather than the facial expressions. The typical head motions in the perplex situations are such as shaking,
tilting, bending backward, bending forward to gaze on the CRT display, keeping still and changing the glance between the
CRT display and the keyboard frequently. Secondly, the head motions in x, y, Z and O directions were captured in real
time by chasing both pupils through the use of image processing system. In the system, the processing speed is
enhanced by reducing image data. The behaviors are converted to the series of vectors which have the moving velocity
of head in x, y, z and Ó directions as the elements. Thirdly, the distances between unknown input motion patterns and
the template patterns of the vector sequences were calculated by DP matching. As the result of DP matching, it was
made clear that each head motion was recognized. The proposed method is available for not only the specified person
but also unspecified persons. The method can be applied to the development of the software which responses
automatically when the operator falls into the perplex situations.
behaviors by obtaining positions and an angle of both
pupils in the image sequence. In Sect. 2, we describe the
Now, we live in a computer assisted society and cant observation of the perplexed behaviors. In Sect. 3, we
separate the computers from our life. However, even the propose the method of recognizing the basic actions of the
computer which seems convenient never can be handled perplexed behaviors using image processing and DP
easily. Especially beginners feel difficulty in operating. matching.
With the coming of the age when everyone has to handle
the computer, computer allergy has come to spread. For
these reasons, it is desirable to make the computer carry 2. OBSERVATION OF PERPLEXED BEHAVIORS
out the kindness action to the operator who has fallen into
the perplex situations. According to Ochiai et al. (Ochiai 2.1 Hardware configuration and observation method
et al, 1994), few operators demand positively the
computer a help for escaping the perplex situations. Itis ^ To investigate the operators perplexed behaviors, the
necessary to make the computer detect the operator's ^ subjects being at word processor work are observed.
perplex situations. The previous approaches of mind Figure 1 illustrates the setup for observing the perplexed
state recognition have been done by the use of the facial ^ behaviors.
expression analysis (Choi et al., 1991, Kitamura et al.,
1993, and Matsuno et al., 1993). These approaches The subject's face, the front and the side of his body, and
have been based on the theory that the facial muscles ^ a picture plane of the CRT display (PC-KM153 : NEC Co.)
expand and contract corresponding to the state of mind. ^ were observed by 4 television cameras (CCD-TR308 :
However, the facial expression analysis requires the ^ Sony Co.), while he was doing the word processor work by
processing of huge volumes of the CG data. Also, there the personal computer (PC-9821Ap : NEC Co... The
is the problem that the analysis is extremely difficult when subject handles the word processor software (Ichitaro
the face isn't toward the camera. For these reasons, we Ver.4 : JUSTSYSTEM Co.) which operates on MS-DOS
researched the simple and practical method of =… (Microsoft Co.). Camera 1 is placed on the CRT display
recognizing the perplex situations in real time by image and captures his front-view face. Camera 2 captures
processing (Kamitani and Marutani, 1994- 97e). upper half of his body from the front half right. Camera 3
captures the upper half of his body from the right side.
In this paper, we present a method for recognizing the — Camera 4 captures the picture plane of the CRT display
perplex situations based on the detection of perplexed from the front half left. The images and the sounds of
1. INTRODUCTION
435
