The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B5. Beijing 2008 
1085 
On the other hand, image-based methods (Remondino and El- 
Hakim, 2006) require a mathematical formulation to transform 
two-dimensional image measurements into 3D coordinates. 
Images contain all the useful information to derive geometry 
and texture for a 3D modeling application. Nevertheless, 
recovering a complete, detailed, accurate and realistic 3D 
textured model from images is still a difficult task, in particular 
for large and complex sites and if uncalibrated or widely 
separated images are used. Indeed (1) the wrong recovery of the 
camera parameters could lead to inaccurate and deformed 
results, while (2) wide baselines cause a challenge to matching 
techniques since they reduce geometric and radiometric 
(especially for marble) similarities between the images and 
cause occlusions. For many years photogrammetry dealt with 
the precise 3D reconstruction of objects from images. Although 
precise calibration and orientation procedures are required, 
suitable commercial packages are now available, mainly based 
on manual or semi-automated measurements (ImageModeler, 
iWitness, PhotoGenesis, PhotoModeler, ShapeCapture). After 
the tie point measurement and bundle adjustment phases, they 
allow 3D object point coordinates determination, as well as 
wireframe or textured 3D models generation. Fully automated 
approaches (Vergauwen and Van Gool, 2006; Goesele et al., 
2007) are getting quite common in the 3D modeling community. 
Nevertheless, they are not yet fully reliable for the daily and 
precise documentation of objects from any kind of image 
sequence and no commercial software is yet available. On the 
other hand, automated surface measurement and object 
reconstruction from calibrated and oriented images is making 
great progresses and the latest results are promising 
(Remondino et al., 2008). 
Calibration & Orientation 
GPS / Total station 
& E*te mai carne» 
parameters 
Absolute reference system 
Space- Airborne images 
Terrestrial images 
Active sensors 
Heritage ansèwwunt mapping 
Manual measurements, «tense 
matching, shape from X 
Registration, noase/owsHap 
reduction 
I 
L 
Sparse or Dense Point Cloud 
r 
f ~ ~~ t 
Geometric Modeling 
Appearance Modeling 
Integration., structuring, mesh 
generation & stropJ#»c«te>n, holes 
filing, segmentation, prsmawe 
fiEbng 
Texture & bump mapping, 
radwnetric corrections, 
blend mg, reflectance mapping, 
global 
I Detailed & photo-realistic textured 3D model 
Rendering 8k Visualization 
Figure 2. 3D modeling pipeline for Cultural Heritage objects 
and site. 
Nowadays, to achieve a good and realistic 3D model, that 
captures the required level of detail, the better way is still the 
combination of the different modeling techniques. In fact, as a 
single technique is not yet able to give satisfactory results in all 
situations, concerning high geometric accuracy, portability, 
automation, photo-realism and low costs as well as flexibility 
and efficiency, image and range data are generally combined 
(El-Hakim et al., 2004; El-Hakim et al., 2007). 
From our experience, and based on the current state of 
technology, in order to capture the information needed for the 
digital 3D documentation and conservation of large Cultural 
Heritage monuments, the resolution of the reconstruction in 
most parts must be about 5 mm with 1-2 mm on highly detailed 
parts, while for such details the accuracy should be about 1 mm. 
For an interactive visualisation of the results, the 3D model 
must viewable on a standard workstation with standard software, 
which limits the model resolution. A movie with photo-realistic 
colour and lighting is also required for dissemination and 
presentation of the detailed and full resolution models. 
3. PROBLEMS AND CHALLENGES 
IN 3D MODELING 
Modeling complex objects and heritage sites has challenges in 
every phase (Figure 3), from the data acquisition to the 
visualisation of the 3D results. In the following, we discuss the 
main challenges, in particular related to the image-based 
approach. 
3.1 Data acquisition 
The size, location and surface (geometry and material) of the 
object or site can create several problems. The dimensions and 
accessibility problems (due to location and obstructions) can 
cause delays, occlusions and can result in missing sections or 
enforce wide-baseline images and poor geometric 
configurations. The complexity of some parts creates self 
occlusions or holes in the coverage, in addition to the 
occlusions from plants, trees, restoration scaffolds or tourists. 
The absence of high platforms for a higher location of the data 
acquisition might cause missing parts, in particular for the roof. 
Other limitations can include restrictions on using some 
platforms like UAVs, airplanes and helicopters, rough and/or 
sloped terrain also with stones, rocks and holes, unfavourable 
weather conditions etc., problems that have been encountered to 
one or the other extent in this Acropolis project. The object 
material, in this case marble, has an important influence on the 
results, both for image matching and laser scanning as it will be 
discussed below. 
3.2 Data processing and point cloud generation 
The cameras must be accurately calibrated, preferably in a 
controlled lab environment, with a 3D testfield and a bundle 
adjustment with additional parameters to fully compensate 
systematic errors (Remondino and Fraser, 2006). 
As no commercial procedure is already available for automated 
markerless tie point extraction from terrestrial convergent 
images, the camera orientation phase is still highly interactive. 
For complex architectural objects, manual and semi-automated 
measurements are still much more reliable. For small areas or 
ornaments rich of details, dense matching or shape-from-X 
techniques can be instead selected to derive dense 3D point 
clouds. 
3.3 3D modeling 
Once a point cloud is available, a polygonal model (mesh) is 
usually generated to produce the best digital representation of 
the surveyed object or scene. This part is generally referred to 
as geometric modeling. Then photo-realism, defined as having