144 
From image sequence to virtual reality 
Jeröme BLANC Roger MOHR 
LIFIA - INRIA 
46, av. Félix Viallet 
38031 GRENOBLE CEDEX 1 
FRANCE 
phone : +33 76 57 43 28 far: +33 76 57 46 02 e-mail : Jerome.Blanc@imag.fr 
KEY WORDS : transfer, matching, reprojection, epipolar, trilinear. 
ABSTRACT : 
This paper presents a way to explore a 3D scene defined by 2D views : given some 2D views of the same 
scene, which we call reference views, we want the user to be able to move a virtual camera, so as to generate 
any other view of the scene, and carry out a virtual visit of the scene. 
We will show how we applied the trilinear relations stated by [Sha 94] to this so-called transfer problem. 
This approach avoids any kind of 3D reconstruction, thus allowing us to deal with real and complex images. 
We also successfully experimented on outdoor scenes, taken with a home video camera. 
The algorithm consists in two steps : it starts with a dense matching between the reference views ; each 
matched couple is then reprojected using the trilinear relations. 
This technique has numerous applications, among which virtual realities, or high rate video compression. 
The images obtained are more realistic than any 3D model we could have computed, while being geometrically 
correct. 
1 Introduction 
This paper presents a way to explore a 3D scene defined by 2D views : given some 2D views of the same (static) 
scene, we want the user to be able to move a virtual camera, so as to generate any other view of the scene, and 
carry out a virtual visit of the scene. This technique has numerous applications, among which virtual realities, 
or high rate video compression. 
The so-called transfer problem has an obvious solution : we can use some 2D views to build a 3D model of 
the scene, which we reproject afterwards on the plane of the virtual camera. This approach raises two problems : 
e if we want to compute a euclidean model, the cameras must be calibrated. Calibration is a delicate (and 
off-line) process ; one has to place a calibration grid in front of the cameras, then to compute the projection 
matrices. This can be an arduous process. Besides, this can’t be done without any physical access to the 
cameras, e.g. if only a video input-flow is available, which can be the case if we apply this technique to 
compression. 
e once the position of 3D points are computed, it’s still a partly unsolved problem to build the corresponding 
3D model (surfaces and curves). In fact, it’s getting nearly impossible for a “real” scene, especially where 
there are three-dimensional textures, like for instance wrought iron. Moreover, it may be useless if we 
don’t want to edit the structure afterwards but just display it under another point of view. 
IAPRS, Vol. 30, Part 5W1, ISPRS Intercommission Workshop "From Pixels to Sequences", Zurich, March 22-24 1995