35. Istanbul 2004 
celli 'F., 2001 A 
, Proceedings of 
dam, Germany, 
Fuzzy Objective 
now A.J., 2000. 
Laser Scanning 
72, pp. 1-23 
'sakiri M., 2001. 
Laser Scanner, 
pe Measurement, 
3D INDUSTRIAL RECONSTRUCTION BY FITTING CSG MODELS TO A 
COMBINATION OF IMAGES AND POINT CLOUDS 
Tahir Rabbani, Frank van den Heuvel * 
Section of Photogrammetry and Remote Sensing, Faculty of Acrospace Engineering, Delft University of Technology, 
Kluyverweg 1, 2629 TS Delft, The Netherlands 
Email: t.rabbani@lr.tudelft.nl, F.A.vandenHeuvel@Ir.tudelft.nl 
Commission V, WG V/I 
KEY WORDS: Industrial Reconstruction, Point Cloud, Laser Scanning, Photogrammetry, Recognition, Registration, 
Modelling, Automation 
ABSTRACT: 
We present a method for 3D reconstruction of industrial sites using a combination of images and point clouds with a motivation of 
achieving higher levels of automation, precision, and reliability. Recent advances in 3D scanning technologies have made possible 
rapid and cost-effective acquisition of dense point clouds for 3D reconstruction. As the point clouds provide explicit 3D information, 
they have a much higher potential for the automation of reconstruction. However, due to the measurement principle employed by 
laser scanners and their limited point density, the information on sharp edges is not very reliable. It is precisely where images have 
superiority over point clouds. In addition images are required for visual interpretation, texture mapping, and modelling parts not 
visible in the point clouds. Moreover, image acquisition is more flexible, and the cost and time required for it is much lower than that 
of laser scanning, making their combined use essential for a cost-effective solution. These reasons led us to develop a modelling 
strategy that uses both images and point clouds in combination with a library of CAD primitives found in industrial scenarios 
represented as CSG (Constructive Solid Geometry) objects. The modelling pipeline in our algorithm starts from point clouds as the 
main data source for automation. First of all we segment the point cloud using surface smoothness and detect simple objects like 
planes and cylinders using Hough Transform. This is followed by fitting of CSG objects to a combination of segments. These fitted 
CAD models are used as registration targets for adding more scans to the project. Additionally, by fitting the projected edges to 
image gradients we register images to point clouds. Once we have a registered data set, manual measurements are added to images to 
model missing parts and to increase the reliability of modelling for portions where laser data is known to be noisy. The final phase is 
similar to bundle adjustment in traditional Photogrammetry as there we estimate pose and shape parameters of all CSG objects using 
all image measurements and points clouds simultancously. We name this final phase Integrated Adjustment as it integrates all 
available information to determine the unknown parameters. 
The results of applying this method to data from an industrial site are presented showing the complementary nature of point cloud 
and image data. An analysis of improvement in quality of 3D reconstruction shows the benefits of the adopted approach. 
1. INTRODUCTION 
As built CAD models of industrial sites are required for many 
purposes like maintenance, documentation, and training. 
Moreover, current research is focusing on applying Virtual and 
Augmented Reality for providing various services for training 
and operation in industrial environments. The implementation 
of these technologies requires accurate 3D models of industrial 
environments for various sub-processes like tracking and 
alignment of virtual and real objects. One such project on which 
the Section of Photogrammetry and Remote Sensing at TU 
Delft has been working since 2001 focuses on using Augmented 
Reality for providing various training services to industrial 
users (STAR, 2003). In contrast to Virtual Reality where 
everything has to be modelled explicitly, Augmented Reality is 
more flexible as it uses a mixture of real-time video and virtual 
objects and humans. As a result more realistic scenarios and 
services can be easily implemented without requiring explicit 
3D models for all the objects. At the same time the requirement 
for aligning the video frames to 3D objects becomes more 
critical. This necessitates more accurate 3D geometric models 
  
Corresponding author. 
for the objects present in the surrounding environment, which 
are used as targets during tracking and alignment. 
Traditional techniques for modelling industrial environments 
use point and line Photogrammetry as it is much faster and 
convenient compared to manual surveys. An improved approach 
suitable for  Photogrammetric modelling of industrial 
environments was presented by (Vosselman et al., 2003) which 
uses fitting. of image edge measurements to back-projected 
contours of the CSG object in the image. This eliminates the 
measurement to CAD model conversion stage, which is 
required for point and line Photogrammetry based approaches. 
Additionally, inclusion of various internal and external 
geometric and parametric constraints greatly reduces the 
degrees of freedom. Thus the number of the required manual 
measurements is also reduced. Still, this process requires a lot 
of manual work, which is the major cost in any modelling 
project. 
The prospects of implementing any automatic strategy for 
industrial environments using only images are very dim; therc 
are three main reasons for that. Firstly, there is no explicit 3D 
information in images; at least two images having good