right
ding.
teris-
puter
2000.
or 3d
num-
ram-
TOSS-
3641,
< and
ional
EEE,
s and
e es-
ition,
usler,
ering
ze on
City,
hape
SEMI-AUTOMATIC 3D RECONSTRUCTION OF OCCLUDED AND UNMARKED
SURFACES FROM WIDELY SEPARATED VIEWS
Sabry F. El-Hakim
Visual Information Technology (VIT) Group
Institute For Information Technology, National Research Council Canada (NRC)
Ottawa, Ontario, Canada K1A OR6
E-mail: Sabry.El-Hakim@nrc.ca
Commission V, Working Group V/2
KEY WORDS: Architecture, Automation, Three-dimensional, Reconstruction, Texture, Virtual Reality, Accuracy, Registration
ABSTRACT:
Three-dimensional modeling from images, when carried out entirely by a human, is time consuming and impractical for large-scale
projects. On the other hand, full automation may still be unachievable for many applications. In addition, 3D modeling from images
requires the extraction of features and needs them to appear in multiple images. However, in practical situations those features are
not always available, sometimes not even in a single image, due to occlusions or lack of texture. Taking closely separated images or
optimally designing view locations can preclude some occlusions. However, taking such images is often not practical and we are
usually left with images that do not properly cover every detail. This paper argues that widely separated views and a semi-automated
technique are the logical solutions to 3D construction of large and complex objects or environments. The proposed approach uses
both interactive and automatic techniques, each where it is best suited, to accurately and completely model man-made structures and
objects. It particularly focuses on automating the construction of unmarked surfaces such as columns, arches, steps and blocks from
minimum seed points. It also extracts the occluded or invisible corners and lines from existing ones. Many examples, such as Arc de
Triomphe in Paris and Florence's St. John baptistery, are completely modeled from a small number of images taken by tourists.
1. INTRODUCTION
A wide range of applications requires 3D reconstruction of real
world objects and scenes. In general, most applications specify
a number of requirements:
1. High geometric accuracy
2. Copturing all details
3. Photo-realism
In addition the following would ideally be desired in a system
that creates such models:
4. Full automation
Low cost
Portability
Flexibility in applications
Efficiency in model size
SoA
The order of importance of these requirements depends on the
application, but in many all are important. A single system that
satisfies all requirements is still in the future. In particular,
accurately covering all the details with a fully automated system
for a wide range of objects and scene remains elusive. For small
and medium sized objects, up to the size of human or a statue,
range-based techniques such as laser scanners can provide
accurate and complete details with high degree of automation
[Beraldin et al, 1999], but being a relatively new technology
that is not produced in large quantities, they remain costly.
They are also not portable enough for a single person to carry
around and use in a manner similar to a video or digital camera.
Image based approaches entail widely available hardware and
potentially the same system can be used for a wide range of
objects and scenes. They are also capable of producing realistic
looking models and those based on photogrammetry have high
accuracy. The issues that remain are coverage of details on
unmarked and sculpted surfaces and full automation. This paper
focuses on image-based methods aiming at increasing the
details level and automation for man-made objects.
Three-dimensional measurement from images naturally requires
that interest points or edges be visible in the image. This is
often not possible either because a region is hidden or occluded
behind an object or a surface, or because there is no mark, edge,
or visual feature to extract. This problem exists even with only
one object in the scene and when we can take images from
well-planned positions. In objects such as architectures and
monuments in their normal settings we are also faced with the
restrictions of limited locations from which the images can be
taken as well as the existence of other objects and illumination
variations and shadows. All this causes problems for automatic
modeling techniques and also generates incomplete models.
Our approach, which does not aim to be fully automatic nor
completely rely on human operator, is especially designed to
model occluded and unmarked surfaces for structure such as
classical architectures. The approach provides enough level of
automation to assist the operator without sacrificing accuracy.
Image registration and scene segmentation into separate regions
are done interactively. This is followed by automatics corner
detection and correspondence. Most points on blocks, windows,
doors, steps, cylinders or columns, arches, quadrics and planes
are measured automatically even when they are occluded or
unmarked. On average, 20% of the points are measured
interactively and the remaining 80% are added automatically.
The remainder of the paper is organized as follows. In section 2,
an overview of image-based 3D reconstruction techniques is
presented. This will lead to a deduction in the following section
that widely separated views and semi-automated techniques are
currently the logical solution to 3D construction of large and
complex objects or environments. A discussion on the effect of
occlusion and lack of textures is given in section 4 followed by
details of the proposed approach. Section 6 illustrates many
examples, such as Arc de Triomphe in Paris, Florence’s St.
John baptistery, San Giacomo dell’Orio church in Venice, and
other structure from around the world that were fully modeled
-]43-—
