'er and less
5. Also, one
ve is sharper
the minimum
jreater preci-
as been suc-
ones under a
lakahara and
-SSAD
.SAD3
'SAD2
-SAD1
—— 6
ns
n of Images
1 equation (2)
ral, however,
ectly aligned,
o rate image
..n} which are
AD, (s,t, C) has
Jo (091 (4)
ordinates (s,t)
5,t) (Figure 4).
87, Kimura et
ugeras, 1992)
IRIN Dg]
rence with
cation
are Imple-
machine con-
he first step IS
the Laplacian of Gaussian (LOG) filtering of the input
images. This filtering enhances the image features and
removes the effect of intensity variations among images
due to difference of camera gains, ambient light, etc. The
second step is the computation of SAD and SSAD with
geometric rectification and correction to produce the SSAD
function. The third and final step is the identification and
localization of the minimum of the SSAD function to deter-
mine the inverse depth. Uncertainty is evaluated by ana-
lyzing the curvature of the SSAD function at the minimum.
The total amount of computation per second required for
the SSAD calculation is estimated as:
N^xW^xDx(C-1) xPxF (5)
where N? is the image size, W? the window size, D the dis-
parity range, C the number of cameras, P the number of
operations per one SD calculation and F the number of
frames per second. We have estimated p as 14 operations
including image sampling in the subpixel precision and cal-
culation of difference. If we set N 2 256, W 2 11, D- 30, C
= 6, and F = 30, then the total computation would be 465
giga-operations. However, the most important aspect of
the multi-baseline stereo algorithm is that it takes advan-
tage of the redundancy contained in multi-stereo pairs. As
a result it is a straightforward algorithm which is appropri-
ate for hardware implementation.
Images
I Laplacian of Gaussian (LOG) E
de
SAD and SSAD Computation
with Geometric Correction and Rectification
- =
| Minimum Detection 4
~
Disparity Map
Figure 5: Outline of stereo method
4. ARCHITECTURE OF THE STEREO MACHINE
Figure 6 illustrates the architecture of the system. It con-
sists of five subsystems: 1) multi-camera stereo head
(five-eye camera head); 2) multi-image frame grabber; 3)
Laplacian of Gaussian (LOG) filters; 4) image rectification"
and parallel computation of SSAD; and 5) subpixel local-
ization of the minimum of the SSAD in the C40 DSP array.
The machine was built with off-the-shelf components (See
Figure 1). The main devices used in the machine include
PLDs, high-speed ROMs, RAMS, pipeline registers, com-
mercially available convolvers, digitizers and ALUs. All of
the system was designed and built at CMU except for the
video cameras, the C40 DSP array and the real-time pro-
cessor board.
' The SSAD subsystem stores the calibration function lo- ly
and Jy-J, of equation (4) in RAM in the form of tables.
Using these tables, the SSAD hardware calculates abso-
lute differences in the rectified coordinates (see Figure 4).
The tables are obtained at the time of calibration and are
loaded when the machine starts up.
These subsystems are connected to a VME Bus and con-
trolled by a VxWorks real-time processor. System soft-
ware, running on a Sun workstation, enables users to
exploit the machine's capabilities through a graphical inter-
face (Figure 7).
d = Ele EERE
Fram Multiple Images with
Different Baselines
LOG Y v x
T d 3 #
Data icc MES | #2| #31099 | «e| v, ;
Camera Head
Y yy SY 99 Ba
+» LOGtoSAD I/F | LOG Outputs
Y Y Y Y
SSAD Computation — Curve of SSAD
Absolute Difference SSAD
| SAD and SSAD |
| Minimum Finder td
VME Bus
v
«——| C40 VF & Graphics Function. |
A
Subpixel Disparity
| Detection
7} #8
Depth map Sy
Depth M
C40 Communication Port J ep ap
VxWorks Ethernet Sun
Real-time :
Processor ” Workstation
*
Figure 6: Architecture of stereo machine
Figure 7: Graphical interface window of the system
software
5. REAL TIME Z-KEYING: A NEW APPLICATION OF
THE STEREO MACHINE
Besides robotic applications, such as autonomous vehi-
cles, there are many other applications for the stereo
machine. The capability of producing a dense 3D repre-
413
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B5. Vienna 1996
