> run On a 
e. We are 
m for the 
)n system. 
ly in front 
1e grabber 
eras at the 
as 
ojection is 
in front of 
ytorealistic 
relatively 
the face is 
A random 
s to avoid 
enough to 
ls. The use 
the lateral 
ired by the 
  
The system is calibrated using a 3-D reference frame with 
coded target points whose coordinates in space are known (see 
figure 4). These are fully automatically recognized and 
measured in the images (Niederoest, 1996). The results of the 
calibration process are the exterior orientation of the cameras 
(position and rotations: 6 parameters), parameters of the interior 
orientation of the cameras (camera constant, principle point, 
sensor size, pixel size: 7 parameters), parameters for the radial 
and decentering distortion of the lenses and optic systems (5 
parameters) and two additional parameters modeling 
differential scaling and shearing effects (Brown, 1971). A 
thorough determination of these parameters modeling 
distortions and other effects is required to achieve high 
accuracy in the measurement. 
   
coooQoQQQ 
 ©000060 
Figure 4. Calibration frame with coded targets 
2.2 Matching process 
Our approach is based on multi-image photogrammetry using 
images acquired simultaneously by synchronized cameras. The 
multi-image matching process is based on the adaptive least 
squares method (Gruen, 1985) with the additional geometrical 
constraint of the matched point lying on the epipolar line. 
Figure 5 shows an example of the result of the least squares 
matching (LSM) algorithm: the black boxes represent the 
patches selected in the template image (left) and the affine 
transformed in the search images (center and right), the epipolar 
lines are drawn in white. 
  
Figure 5. Geometrical constrained LSM; 
left: template image, center and right: two search images 
The automatic matching process produces a dense and robust 
set of corresponding points, starting from few seed points. The 
seed points may be manually defined in each image, generated 
semi-automatically (defining them only in one image) or fully 
automatically. The manual mode is used for special cases where 
the automatic modes could fail; the seed points have to be 
selected manually with an approximation of at least 2 pixels in 
each image: LSM is then applied to find the exact position. In 
the semi-automated mode the seed points have to be selected 
manually only in the template image; the corresponding points 
in the other images are established automatically by searching 
for the best matching results along the epipolar line (see figure 
6). This mode is the most convenient for normal cases of static 
surface measurement: it is fast but leave the operator the choice 
where to set the seed points. The fully automatic mode is useful 
in cases with dynamic surface measurement from multi-image 
video sequences, where the number of multi-image sets to be 
processed could be very large. In this case, Foerstner interest 
point operator (Foerstner and Guelch, 1987) is used to 
automatically determine in the template image marking points 
where the matching process may perform robust results; the 
corresponding points in the other images are then established 
with the same process as for the semi-automatic mode. 
template image search image 
    
    
purior 
Figure 6. Semi automated seed point definition 
After the definition of the seed points, the template image is 
divided into polygonal regions according to which of the seed 
points is closest (Voronoi tessellation). Starting from the seed 
points, the set of corresponding points grows automatically until 
the entire polygonal region is covered (see figure 7). 
zoom 
"S 
$-—64— 6 -—0—- € Es 2 
A 
eee reo 
| 7 
O. seedpoints A KE nur 
M 
* 
/ 
e matched points 
9-—Q—- e—e e—e 9 / 
/ 
y. 
  
  
Ne s y 
T 
Figure 7. Search strategy for the matching process 
The matcher uses the following strategy: the process starts from 
the seed point, shifts horizontally in the template and in the 
search images and applies the least squares matching algorithm 
in the shifted location. If the quality of the match is good, the 
shift process continues horizontally until it reaches the region 
boundaries; if the quality of the match is not satisfactory, the 
algorithm computes the matching again, changing some 
parameters (e.g. smaller shifts from the neighbor, bigger sizes 
of the patches). The covering of the entire polygonal region of a 
seed point is achieved by sequential horizontal and vertical 
shifts. The process is repeated for each polygonal region until 
the whole image is covered. 
To evaluate the quality of the result of the matching, different 
indicators are used: a posteriori standard deviation of the least 
squares adjustment, standard deviation in x and y directions, 
displacement from the start position in x and y directions and 
distance to the epipolar lines. Thresholds for these values can be 
defined for different cases, according to the texture and the type 
of the images. 
Before beginning the three dimensional processing, filters can 
be applied to the 2-D matching data to minimize the number of 
possible errors. The Voronoi tessellation produces an irregular 
grid (see figures 8 and 9, left) of points in the template image, 
therefore, the set of matched points has first to be uniformed to 
a regular grid before the application of any filters. This is 
achieved by matching all the points shifted to the regular grid 
(see figures 8 and 9, right). 
  
9 € 
e o 
^74 9 e e À 
9959 
e 9 o\» 
e .» o 9 
. 99 
e e e o o e 
€ e o e o o 
e eo. 0° o 
eo... 0 0 
090 0 0 0 0 0 
. 000000 
© 0 0 0 0 0 0 
. 0000 00 
® 0 0 0 0 0 0 
. 0 000900 
. 0000000 
  
  
Figure 8. Regularization of the matched point grid 
-243- 
BRET LI Re