de 
M X ps 2 KV 
have prompted interest in developing algorithms for 
translating single digital images into three dimensional 
information of the object surface. These algorithms di- 
rectly relate an image intensity value to the inclination of 
the corresponding surface patch relative to the direction 
of illumination. Such methods are called 'shape from 
shading’ or ’photoclinometry’ and have been pioneered 
by Rindfleisch /1966/ and Horn /1970/. While photocli- 
nometry has been developed for astro-geological re- 
search and most applications deal with the reconstruc- 
tion of planetary terrain profiles /e.g. Davis, Sonderblom 
1984/, SFS is a research direction within computer vision 
and focuses on the reconstruction of surfaces /Horn, 
Brooks 1989/. Both methods are referred to together in 
this paper and are abbreviated with SFS. As a conse- 
quence of being extremely sensitive to changes in incli- 
nation, SFS can detect small terrain undulations that are 
far below the sensitivity of photogrammetry. On the 
other hand, SFS relies on the correctness of various 
assumptions concerning the illumination and the light 
reflection of the object surface. Furthermore, in classical 
SFS, only surface slopes instead of heights can be deri- 
ved. À collection of papers on this topic and an excellent 
bibliography are contained in Horn, Brooks /1989/. 
Since the requirements for digital imagery, in order to 
be used for digital image matching or for SFS, are more 
or less complementary to each other, a combination of 
the two methods should yield reliable results also in 
image regions, where one of the two methods employed 
independently fails. Such a combination was already 
suggested by Barnard, Fischler /1982/. It is also in line 
with the ’cooperative methods paradigm’ of computer 
vision /McKeown 1991/, which basically states that the 
combined use of different methods for the same aim 
improves the results. In this context it is interesting to 
note that SFS also has its role in the human capability of 
depth perception. Following the work of Julesz /1971/ 
and Marr /1982/ it was commonly believed that humans 
rely only on image features, especially on zero crossings 
of the second derivative of the image intensity function, 
for binocular depth perception. Only recently it was 
shown that binocular SFS alone provides unambiguous 
depth clues as well /Mallot 1991/. 
In this paper a new global approach is presented and 
investigated integrating digital image matching and mul- 
ti image SFS in object space. In a least squares adjust- 
ment the unknowns (geometric and radiometric para- 
833 
meters of the object surface) are estimated from the 
pixel intensity values and control information. The per- 
spective projection is used for the transformation from 
object to image space. Chapter 2 describes a simple 
model for the generation of a digital image. In chapter 3 
a multi image object based least squares matching ap- 
proach developed over the last years is shortly reviewed. 
Chapter 4 contains an introduction to SFS. In chapter 5 
the integration of digital image matching and multi 
image SFS is presented. Experimental results using syn- 
thetic images are contained in chapter 6. In the last 
chapter conclusions and an outlook for further research 
are given. 
2. ASIMPLE MODEL FOR THE GENERATION OF 
A DIGITAL IMAGE 
In this chapter a model for the generation of a digital 
image taken with an optical sensor is shortly reviewed, 
since the resulting equations will be needed in the remai- 
ning part of the paper (see Horn /1986/ for more details). 
The image irradiance E; (x, y ) at point P’ (x,y) inthe 
image plane is formed by light reflected at a point 
P(X,Y,Z) on the object surface. For this imaging 
process the well known camera equation (1) holds (for 
the following derivations see also figure 1): 
Ei(x.y) = 5 CD ent e LOG Y) (1) 
x,y image coordinates 
X,Y, Z object coordinates 
E;(x,y) image irradiance 
d diameter of optical lens 
f focal length of optical lens 
a angle between optical axis and the 
ray through P and P 
degree of atmospheric transmission 
y unit vector in the viewing direction at 
POSX, 2) 
L(X,Y)  sceneradiance in the viewing direc- 
tionv 
In general, L depends on the illumination (number and 
size of light sources, direction and radiance of illumina- 
tion) and on the properties of surface reflection, which 
in turn depend on the surface material, its microstructu-