BS o 
Neo €0 cC €) T CD UM 0 
T 
C$ 2A Ou Law OO OO O 
  
| 
  
have also been identified [Hartley, 2000, Kahl et al, 2001]. For 
metric construction, certain sequences will cause self- 
calibration to fail or not give a unique construction. These 
critical motions have been studied when camera intrinsic 
parameters do not vary [Sturm, 1997] and when they do vary 
from image to image [Kahl et al, 2000]. 
This paper focuses on the 3D construction, which is the least 
automated of all the steps, rather than correspondence, pose 
estimation or calibration. 
3. AUTOMATION AND WIDELY-SEPARATED VIEWS 
From the above overview of current techniques, the following 
points can be made: 
e In practical situations taking the sequences required for a 
fully automated techniques might not be feasible. 
e Large objects or complex scenes require a large number of 
images to carry matching and pose estimation automatically. 
e Since the modeling process still requires human interaction to 
define the topology, assign constraints, and post-process the 
results, large number of images makes this difficult. 
It is therefore important to develop an approach that requires 
only a small number of widely separated views and at the same 
time offers a high level of automation, and be able to deal with 
occluded and unmarked surfaces. 
4. ON OCCLUSION AND LACK OF TEXTURE 
Occlusions and lack of texture hinder image-based modeling 
since the methods require features that can be seen either by a 
computer or human. The scene in figure 1 has occlusions and 
most of the columns surfaces have no texture. Both fully 
automated and fully manual methods will have difficulty here. 
Yet, this is typical in much classical architecture. In our 
approach, with less than 30 manually measured points, the full 
scene [figure 2] with automatically added 300 points can be 
completed without further human intervention. 
   
    
Figure 1: Corner of a Figure 2: The constructed 
courtyard solid model 
   
5. DETAILS OF THE APPROACH 
The approach is designed mainly for man-made objects. A good 
example is classical architectures, which are designed within 
constraints of proportion and configurations. Classical buildings 
are divided into architectural elements. These elements are 
logically organized in space to produce a coherent work. There 
is a logical hierarchical relation among building parts and 
between parts and whole. The most common scheme divides 
the building into two sets of lines forming a rectangular grid 
[Tzonis and Lefaivre, 1986]. The distance between the grid 
lines are often equal or when they vary, they alter regularly. The 
grid lines are then turned into planes that partition the space and 
control the placement of the architectural elements. The 
automation of 3D reconstruction is better achieved when such 
understanding is taken into account. We will reconstruct the 
architecture elements from minimum number of points and put 
them together using the planes of a regular grid. Other schemes, 
such as a polar grid, also exist but the basic idea can be applied 
there too. Classical architecture can be reconstructed, knowing 
its components, even if only a fragment survives or seen in the 
images. For example, a columnar element consists of: 1) The 
capital, a horizontal member on top, 2) the column itself, a long 
vertical tapered cylinder, 3) a pedestal or a base on which the 
column rests. Each of those can be further divided into smaller 
elements. In addition to columns, other elements include pillars, 
pilasters, banisters, windows, doors, arches, and niches. Each 
can be reconstructed with a few seed points from which the rest 
of the element is built. 
imaging 
Selection 
Initial Point Extraction 
widely separated víews 
    
  
  
Interactive 
  
  
  
Segmentation «€-------—---— 
Constraints 
Seed Points Element Properties 
  
Figure 3. Simplified diagram of the general procedure that 
shows which is automatic. 
Our approach is photogrammetry-based. In order to increase the 
level of automation, the process takes advantage of properties 
like those mentioned above for man-made objects and 
structures. The approach provides an adequate amount of 
automation to assist an operator to provide high level of details 
with excellent geometric accuracy. Figure 3 summarizes the 
procedure and indicates which step is interactive and which is 
automatic (interactive operations are light gray). The figure also 
shows an option of taking a closely-spaces sequence of images, 
if conditions allow, and increase the level of automation. In the 
remainder of the paper, we will discuss only the option of 
widely separated views. Images are taken, all with the same 
camera set up, from positions where the object is suitably 
showing. There should be a reasonable distance, or baseline, 
between the images. Several features appearing in multiple 
images are interactively extracted, usually 12-15 per image. The 
user points to a corner and label it with a unique number and the 
system will accurately extract the corner point. Harris operator 
is used [Harris, 1998] for its simplicity and efficiency. Image 
registration and 3D coordinate computation are based on the 
photogrammetric bundle adjustment approach for its accuracy, 
flexibility, and effectiveness compared to other structure from 
motion techniques [Triggs et al, 2000]. Advances in bundle 
adjustment eliminated the need for control points or physically 
entering initial approximate coordinates. Many other aspects 
required for high accuracy such as camera calibration with full 
distortion corrections have long been solved problems in 
Photogrammetry and will not be discussed in this paper.