SURFACE BASED OBJECT RECOGNITION AND INSPECTION 
BY PHOTOMETRICALLY EXTENDED BUNDLE ADJUSTMENT TECHNIQUE 
A.B ORUN, Researcher 
Marmara Research Center, Information Technologies Institute 
P.O Box 21, 41470 Gebze, Kocaeli-Turkey 
Prof. Dr. A.ALKIS 
Yildiz Technical University 
Geodesy and Photogrammetry Department 
Besiktas, Istanbul- Turkey 
Commission Ill, Working Group 
Key Words : Photogrammetry, Object recognition, Photometry, Surface inspection, CCD cameras 
ABSTRACT : 
Object or product recognition and inspection has been an active and popular area of various industrial applications in 
last two decades. The demands on more effective algorithms are increasing by the needs for recognition and 
inspection of more complicated industrial products. The Extended Bundle Adjustment technique presented in this paper 
is aiming to integrate geometrical object surface data (which are determined using photogrammetrical bundle 
adjustment technique) with the surface reflectance properties (measured by a remote CCD camera) to yield more 
efficient object recognition results. This combined technique is assumed to be more practical for the industrial 
environment and may give better result than Photometric Stereo technique, which was first formulated by Woodham 
and required almost perfect laboratory conditions. Since the Extended Bundle Adjustment technique based on 
perspective geometry of the stereo image pairs is already able to yield both the orientation properties of the object 
surface and the surface reflectance data separately , it may be the more convenient method to extract object surface 
information, unlikely to Photometric Stereo whose surface geometry and reflectance data are highly correlated to each 
other. 
1. INTRODUCTION 
The most common techniques of object surface 
determination are basically guided by shape-from- 
shading tasks which are principally referred to the 
object surface reflectance properties. In these 
techniques, the patch of object surface can be 
analysed with the contribution of light reflectance from 
the surface. Within this technique, scene radiance 
information plays an important role to extract surface 
slant and tilt values (surface orientation parameters) 
which already means 3-D surface information itself. An 
example of shape-from-shading technique can be 
given as Photometric Stereo, which was first 
formulated by Woodham (1980). Since the other 
similar shape-from-shading techniques (Harrt and 
Carlotto, 1989 ; Ikeuchi, 1980) depend on almost 
same principles , here as an example, only 
Photometric Stereo technique has been compared to 
Photometrically Extended Bundle Adjustment method. 
The idea of Photometric Stereo, basically refers to the 
determination of relationships between the object local 
surface (patch) orientation and its reflectance by 
varying the direction of incident illumination between 
the successive images. In Photometric Stereo, the 
reflectance characteristics of an object surface must be 
known. An ideal surface is to be used such as 
Lambertian with the perfectly controlled illumination 
condition to gain surface parameters. Each image is 
taken from the different directions while the viewing 
direction is constant. The object surface intensity value 
at any surface point for three different views may be 
described as : 
616 
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B3. Vienna 1996 
Li (xy) = R1(p1,91); (1) 
I2 (Xy) * R2(p2,92); 
Is (x,y) = Rs (ps,q3) ; 
Here at least three images are required to solve p; and 
qi of the same surface. The function Ri (pi , qi) 
characterises the reflectance map and p,q are surface 
gradients along x,y directions. In Photometric Stereo 
technique, so many unknown parameters should 
accurately be measured in the laboratory environment, 
such as ; surface reflectance pixel values, incident 
angles (between the light source and surface normal), 
etc. Briefly, Photometric Stereo technique needs the 
following features ; 
1-Image projection used is assumed to be 
orthographic. 
2-Light source should be a single distant point source. 
3-Light source displacement to provide different 
incident angles should be very accurate in degree or 
if differently located sources are used, they should 
have exactly equal illumination. 
4-The object surface should be perfectly diffuse 
(Lambertian, etc.) surface. 
(Woodham, 1979). In addition to the above limitations, 
Photometric Stereo technique can not operate in real- 
time since the light source has to be rotated around the 
camera axis to change the direction of incident 
illumination meanwhile multi images have to be 
acquired as the object remains constant. Using 
Photometrically | Extended Bundle . Adjustment 
technique, most inconveniences above are solved. The 
tests i 
some 
2. TH 
The s 
Bundl 
pairs 
laser 
Figure 
for ins 
(filam 
camel 
contai 
and le 
calibr: 
grabb 
etc. n 
each 
simult 
The s 
syster 
the ir 
1). Me 
stereo 
applic 
frame 
set co 
the so 
the re 
found 
formul 
real ce 
x = -f 
(Moffit 
to the 
tilts of 
as foll