A FILTERING-BASED APPROACH TO EYE-IN-HAND RoBOT VISION 
Antonio Maria Garcia Tommaselli 
DCart-FCT-Unesp - Pres. Prudente 
e.mail: ueppr@brfapesp.bitnet 
Brazil 
Clésio Luis Tozzi 
DCA-FEE-Unicamp 
e.mail: clesio@dca.fee.unicamp.br 
Brazil 
COMMISSION V 
ABSTRACT 
The paper addresses the problem of camera calibration and object location in 
robotics eye-in-hand applications. The 
proposed solution uses a 
modified functional 
model based on straight features for camera calibration and introduces Kalman filtering 
techniques for improvement of the results. A reduction of the computational effort of 
features extraction at image processing level is obtained by the feedback of estimated 
parameters of camera location. Results concerning precision and computational effort 
are presented and discussed. 
KEY WORDS: Robot Vision, Machine Vision, Kalman Filtering, Camera Calibration, Space 
Resection, Eye-in-Hand, Object loration. 
1. INTRODUCTION 
Current application of industrial robots, when 
used without external sensorial feedback, are limited 
by the uncertain models of the robots and unknown 
environments. Precision and flexibility of the whole 
system are increased when vision and others sensors, 
such as laser, sonar and tactile sensors are used. 
Vision systems have been used mainly for 
recognizing, locating and inspecting stationary parts. 
However, visual information may be used to identify and 
locate objects or as a feedback to the robot control 
systems. 
According to Weiss et al (1987), the actual robot 
geometry may be slightly different from the robot model 
and, therefore, the actual end-effector position may 
differ from the desired one. In order to solve this 
problem position and orientation (pose) of the 
manipulator end-effector obtained by vision sensors can 
be used as a feedback signal to control robot in real 
time (Feddema et al, 1991). 
Vision systems can be introduced in the robot 
either by attaching a camera over the wrist 
(eye-in-hand) or in a remote position (eye-off-hand). 
In the  eye-in-hand system the problem is the 
determination of camera location and orientation each 
time a robot movement is made. In the eye-off-hand 
configuration, otherwise, camera location and 
orientation is known and tracking and reconstructing 
the wrist position becomes the problem. 
1.1 Camera Calibration 
The problem of calculating camera position and 
orientation is called camera calibration or space 
resection. In Photogrammetry, besides the six position 
and orientation parameters, the problem of calibration 
involves additional inner parameters, which describe 
the internal camera geometry. In order to obtain the 6 
external parameters, control points are used in most of 
photogrammetric and vision approaches. Given a set of 
image coordinates and corresponding world coordinates 
of control points, the well known collinearity 
equations and Least Squares Method can be used in order 
to get an optimal estimate for the parameters. This 
approach is iterative and requires linearization of the 
collinearity equations, which is time consuming and 
improper for real time applications. In order to avoid 
the computational cost caused by collinearity 
linearization, some alternatives have been proposed 
which adopt linear models: Abdel-Aziz, Karara (1974), 
Lenz and Tsai (1988), Fischler and Bolles (1981). 
Once the problem of parameters estimation is 
solved, remais the problem of feature extraction and 
correspondence of the control points in the image and 
object space, which is the bottleneck in the Machine 
Vision process. Most of the authors avoid this problem 
by considering correspondence as a foregoing step in 
their approaches. 
Alternatives for features correspondence have been 
developed using, instead of points, more meaningful 
features, such as, straight lines, curved lines, 
rectangular ^ shapes, junctions, etc. Straight lines 
present advantages over other features considering 
that: 
images of man-made environments are plenty in 
straight lines; 
straight lines are 
features and the 
easier; 
straight line parameters can be 
subpixel accuracy. 
The use of alternative features has received more 
attention in recent years and more and more methods 
have been proposed: Masry (1980), Lugnani (1980), 
Tommaselli and Lugnani (1988), Mulawa and Mikhail 
(1988), Liu and Huang (1988a and 1988b),Salari and Jong 
(1990), Mitiche, Faugeras and Aggarwal (1989), Mitiche 
and Habelrih (1989), Dhome, Richetin, Lapresté and 
Rives (1989), Halarick (1989), Chen, Tseng and Lin 
(1989), Wang and Tsai (1990), Lee, Lu and Tsai (1990), 
Echigo (1990), Chen and Jiang (1991), Chen and Tsai 
(1991). 
It is important to observe that methods for camera 
calibration in Machine Vision must take into account 
parameter estimation and error analysis in order to 
avoid unreliable solutions. 
detect than point 
problem becomes 
easier to 
correspondence 
obtained with 
1.2 Filtering 
Filtering techniques offer two great advantages 
when applied to the dynamic space resection problem in 
eye-in-hand systems: firstly, parameter estimation can 
be obtained using past observations without storing 
them; secondly, for each observation an state estimate 
is generated. This recursive approach can be used to 
feedback the feature extraction step, in order to 
reduce the search space both in image and Hough space 
and, therefore, to diminish computational effort. 
  
  
  
  
  
  
  
  
  
  
  
   
  
  
   
    
  
  
  
  
    
  
  
   
   
    
   
    
  
    
   
   
    
    
   
   
    
   
     
    
   
       
   
     
      
   
   
  
  
     
   
   
   
  
  
     
    
     
    
    
    
   
   
      
EN A hme A pups [f) m b (RR AG pee fh f5 oT NY o5 09 M ER A 
— 
£5 f) "r4 "D mM ^65 
mn tn er "3 € Ay MA eon IN 
(5, 6 6 "05 et