onvey it into 
VE 
llows: 
gh a control 
ide how to 
e program to 
| purpose of 
In this case, 
nents object 
monitor, and 
position with 
lition, due to 
cation using 
utomatically 
, the models 
nly provides 
r arms are 
and realistic 
lied a virtual 
at real-time 
/ironment so 
perator. The 
can perceive 
s directions. 
e-delay. 
Iti-strategies 
r example, a 
cular active 
| as a means 
ition fusion, 
provided by 
provided by 
ject position 
ce both drift 
also in pose. 
ion contour 
S to SIVE 
) calibrating 
features; 2) 
inted on the 
hnique; 3) 
f the robotic 
calibration; 
sing line 
. 
landmarks, 
space robot 
for dynamic 
SIVE environment, and it is also very difficult to find 3D and 2D 
corresponding points. We have to adopt straight lines instead of 
points as our feature landmark, and utilize geometric information 
of the straight lines to realize the camera calibration. The reasons 
selecting the straight line as landmark are: 
(a) The straight line is a common, important natural landmark; 
(b) The straight line is most easily detected than the point; 
(c) The straight line contains a lot of important geometric 
information, such as parallel, orthogonal, intersection at a point. 
(d) The straight line can be found in many objects, such as 
building, robot arm, industrial parts, cone, polyhedron. 
A) Camera calibration model 
Without considering optical distortion, the camera model is 
really a pinhole model. As shown in Fig. 3. Assuming that (x,y,z) 
present the coordinates of a visible point P in a fixed reference 
systemO — XYZ , and (Xc»Yc»Zc) represent the coordinates of 
the same point in a camera-centered coordinate system 
0 —X,.X.Z, [Fig. 3]. The origin of the camera-centered 
coordination system coincides with its optical axis, the image 
plane, which corresponds to the image sensing array, is assumed 
to be parallel to the (Xc»Yc) plane, f represents a (effective) 
focal length of the camera. The relationship between the world 
and the camera-centered coordinate systems is given by 
X X. 
Z ZC 
Where R = (a; ,b;,c;)^ , i-1,2,3 is a rotation matrix defining the 
: : ; ides J 
camera orientation, A is a scale factor, T — (t;) „4=1,2,3 isa 
translation vector defining the camera position. 
9 
ZC 
TIU 1 2 
c A. 
y r Oo, 15 
2 6 P7 
Z 
vv u 
X Oc z 
XC M 
yc X 3 4 
Fig. 3. Camera calibration model. Fig. 4. Camera calibration using 
straight line groups [Feng 1986]. 
  
  
  
  
  
  
  
  
  
  
We now define, in image plane, the original coordinate 
system (O",u,v), where O' represents the principal point of the 
image plan and the u and v axes are chosen parallel to the X, and 
Ye axes. The image plane coordinates of the point P are given 
by following equations [Feng 1986] 
u=-f# 
t Q) 
vef 
Z 
If (r,c) are used to represent the coordinates of pixels in the 
digital image plane with row, column, and (75, Co) represents the 
row and column of the principal point o' in the image plane. 
Equation (2) may be represented by 
r-r,=su=-fs 
(3) 
cc, =sv=-f, 
Where 5,,5, are scale factors of sensors array in horizontal and 
vertical directions. Equation (3) represents distortion-free 
camera model (also called pinhole model). 
B) Resolution Strategies 
B.1) Estimation of the rotation parameters 
Let us suppose there are three pairs of parallel straight lines in 
objective space (Fig. 4). Line 1-2 is parallel to line 3-4, and 
further parallel to X axis in the world coordination system; line 
5-6 is parallel to line 7-8, and further parallel to Z axis; line 9-10 
is parallel to line 11-12, and further parallel to y axis. 
For any a point on line 1-2, we can obtain the following 
equation from equation (1) and (2): 
rb 
Zt 3 
Supposing (4j,Vj) and (42,V2) are the projective coordinates 
of the point 7 and 2 in image plane respectively, then the above 
equation can be rewritten by: 
= Constant 
  
(v, m )f 1 1 1 
(a, a, a,) (u, —U, ); ug gisiM voie a (4) 
MI ues 09 
where ug, vo are the image coordinates of the principal point. In 
the same way, if (u,, v4) and (U,,V4) denote the position of 
the point 3 and 4 in the line 3-4, we can obtain 
(mom) a Ales P) 
zs = a, + Ku, 
a U “) d di 2 a, = aR f (5) 
2 Q, * Ru 
y,-v, 0 0| |u-» 0 O0 
P=| 0 uy, Uu|-| O UL Up (6) 
0 V3 Va 0 V V2 
u4-u4, 0 0| |4- 0 O 
Q=| O vi vj-| O vi val g- 
0 wu 0 My Map 
iy rly, es 4 
Vj V5 V3 — V4 
(7) 
  
  
The orthogonal constrain of nine elements in rotation matrix 
R is given by 
al*el*alzl (8) 
Thus, we obtain the equation about 4 
+R 2 2 
a a. D, = (Qi + Ryu) +(P, - Rye) «au» ©) 
1 
Similarly, we can obtain the relations between the angle 
parameters (b, ,bz ,b3) and the parallel straight line 9-10 and 11- 
12, which are parallel to Y axis, as well as the (€; ,C2 , C3) and the 
parallel straight line 5-6 and line 7-8, which are parallel to Z 
axis. 
_ 22 + Rauo b; _Ravwv-R b, Raf: (10) 
D, D, D, 
+R 2 
_ 23 + R3ug ew Ryvo - P cos Rif (11) 
D; D; D; 
Where: P; ,Q; , R5 , D, , P,Q4, R4, D; are similar to P,, Qj, Rj, Dj. 
by 
€i 
B.2) Determination of the interior parameters (u, »Vo» £3 
Considering orthogonal constrain of the nine components in 
rotation matrix R , we obtain 
-(Q,R, +Q,R,) (RP, +R,P) -R,P, u, Jann) 
- (Q,R, - Q,R)) (R,P, * R,P,) - RP, Yo = Q;Q, * P,P, 
-(Q;R,*Q,R,) (R,PB,*R,P, -R,P, Us + VHS} 0,0, + P,P, 
(12) 
B.3) Determination of the translation vector t 
So far, we have known the rotate matrix (R=(a;,b;,c;), 
i 2 1,2,3), the interior parameters (4, ,v, , f£) . If the length of 
203