International Archives of Photogrammetry and Remote Sensing. Vol. XXXII, Part 5. Hakodate 1998 
Function of Object from Observing Human Action 
-Toward improving the operationality of virtual tools- 
Kazuhiro Annoura, Kazuaki Tanaka, Jiang Yu Zheng, Norihiro Abe, Shoujie He 
Faculty of Computer Science and Systems Engineering 
Kyushu Institute of Technology 
noura @ sein.mse.kyutech.ac.jp 
Hirokazu Taki 
Faculty of System Engineering 
Wakayama University 
Commission V, Working Group SIG 
KEY WORDS: magnetic sensor, virtual reality, human action, function of object 
ABSTRACT 
As it is not easy for a user in virtual environmente use virtual tools, improving the feasibility of manipulating them is neces- 
sary to help him correctly achieve his intenstion. Tools are used to change the current state of an object into desirable one, 
so the several constraints are imposed on their movement, a grasping method and an object class to be manipulated. 
Consequently, it is possible for the system to predict user's intent by watching his behavior. We propose a method to 
measure the movement of real tools when they are exploited for a particular purpose and to store them so that they are 
retrieved according to their constraints. Inferring user's intention through the observation of his behavior, the system plays 
back one of the stored movements. This allows any users to exploit a virtual tool in virtual environment in the similar way in 
the real one. 
1. INTRODUCTION 
An artificial reality or a virtual reality have been attracted 
attention as techniques for having us intuitively understand 
problems which it is hard for us to experience in daily life. 
For making it easy for us to intuitively comprehend events 
occurring in virtual environments, a man machine interface 
plays an important role to secure fast and correct communi- 
cation between us and a computer. 
Displays, keyboards or mouses are well known as interface 
tool for bridging between a person and a computer, but a 
mouse does not allows a person with no experience in a 
virtual space to manipulate the 3 dimensional space at his 
own will. A number of purpose-built input/output devices 
have been developed in virtual reality studies to facilitate 
the 3 dimensional manipulation. Input devices such as a 
handle or an accelerator used in drive simulators are typical 
examples. These purpose-made devices generally make it 
difficult to generalize a system for controlling devices in- 
cluding problems of their sizes, costs and popularization 
into families. An approach of building a knowledge base of 
tools was proposed to divide input devices from a core sys- 
tem by restricting them to 3 dimensional position sensors 
and data gloves (Funahashi et al., 1997). In this system, the 
process of manipulating a tool in a virtual environment is 
realized by visualizing a gesture of a data-glove, but there 
is somewhat difference between the real behavior and the 
virtual one. 
Tools are used to manipulate an object for attaining a par- 
ticular purposive operation, this allows a system to antici- 
pate an operator's behavior. This means that it is possible 
to store in advance the behavior of hands observed at the 
time when he manipulated a real tool. 
When a system perceives that a person is going to use a 
tool in order to attain its original goal, by playing back the 
behavior corresponding to the tool he will get real opera- 
tional feeling. 
Consequently, if a data glove approaches to a tool and he is 
going to grasp it with a grip which makes the operation pos- 
sible, the system will get his virtual hand to grasp the virtual 
tool and generate a sequence of motion from the stored 
behavior. Of cause, the size of the virtual tool is not always 
equal to the stored one, then some translation is necessary 
for generating the motion. 
To make clear the effect of the method proposed in this 
paper, we select a claw hammer as an example which is 
usable both as hammer and as nail extractor. A nail and a 
board is given as the objects of the tool (Figure 1). 
  
Fugure1. An example of a tool and object. 
  
In robo 
try to hi 
tion ob 
trimmir 
The hu 
methoc 
sional | 
detect 
measu 
need n 
The pt 
helps « 
which c 
we do ı 
ora ni 
into ht 
themse 
pose ir 
betwee 
As you 
a hami 
have n 
functio 
to a gi 
definiti 
which 
given ( 
A visic 
object 
son us 
meası