International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol XXXV, Part B5. Istanbul 2004
Manual
Semiautomatic
Automatic
ome. 7 CE
2D Image
specific | Range data |
knowledge /
Segmentation |
Object Recognition |
Constraint detection
8
Integrated Adjustment
Fit final CSG models
cd
Figure 1. Flowchart of the modelling pipeline
intersection are needed along with the information about
corresponding points to calculate 3D coordinates. Secondly, the
clutter which is a universal feature of most industrial sites
combined with uncontrolled lighting makes the automatic
object detection much more difficult. Thirdly, the information
in images is contained mainly on the edges, as there the contrast
is usually much better, while in the absence of some
distinguishable marks the information about the 3D geometry of
the rest of the surface is at best minimal. This becomes a major
limitation as in industrial environments the curved objects are
universally present, and for these objects only edge-localized
information is not enough for automatic detection and fitting.
All these limitation are successfully resolved by laser scanning
techniques, where we get direct 3D information in the form of a
point cloud making the job of object detection and fitting much
easier. Furthermore, recent advances in 3D scanning
technologies have made possible high-speed acquisition of
dense and accurate point clouds at moderate costs (Laser
Scanner Survey, 2003).
The strengths of Laser Scanning do not mean that images lose
all their utility. Actually the fact that images provide accurate
information on edges becomes a source of strength if both
image and point cloud data are simultaneously used. Most of
the currently available laser scanners are using techniques based
on either triangulation or time of flight. In point clouds acquired
using either of them the data on edges is noisier compared to
that on the surface of the scanned object. This has to do with the
angle between the surface normal and the laser beam, which
changes very rapidly near the edges of the object, making
precise point cloud acquisition very difficult. Additionally, in
the case of time of flight multiple reflected pulses lead to
averaging of range measurements. This is especially true for the
step edges. These limitations of scanning technologies make
images a complementary source of information, especially on
the edges of the objects. Acquisition of images as a supporting
data source is not a problem, as cameras are still much faster
and cheaper than currently available laser scanners.
Additionally, images are much better for visual interpretation
and are required for producing texture-mapped models for
realistic visualization.
Based on the above observation of the complementary nature of :
images and point clouds the modelling strategy that we have
developed uses both data sources simultaneously and thus
exploits all available information to achieve a more accurate
estimation as well as higher levels of automation. The rest of thc
paper is organised as follows. In Section 2, we give a summary
of the modelling pipeline. Section 3 provides details of fitting
CSG objects to point clouds and to image edges. We present
fitting results on an industrial scenario in Section 4, along with
a discussion about the improvement in estimation accuracy
using two experiments of fitting on single objects. Finally, we
conclude in Section 5 and propose some directions for future
work.
2. MODELLING PIPELINE
Our modelling pipeline is shown in Fig. I. As it was noted in
the introduction we are using both images and point clouds as
data sources. We start from an initial approximate scan-to-scan
registration using Iterative Closest Point method (Besl and
McKay, 1992). The registration obtained from this pre-
processing stage is used until objects have been recognized and
fitted. Then this initial registration is refined in the final
Integrated Adjustment using object-based registration (Dijkman
and Heuvel, 2003). For image registration or exterior
orientation, image edge to back-projected CSG model contour
fitting is used, during which only image exterior orientation
parameters are adjusted and object parameters of the modelled
objects are kept fixed.
For next stages of segmentation and object recognition only
point cloud data is used, as in contrast to images it provides
explicit 3D information, and thus has better chances of
achieving automation. This is especially true for the
reconstruction of industrial sites as due to their man-made
origin presence of well-defined CAD primitives can be
expected. For example as reported by Nourse et al. (1980) 85%
of objects found in industrial scenes can be approximated by
planes, spheres, cones and cylinders. This percentage rises to
95% if toroidal surfaces are included in the set of available
primitives (Requicha and Voelcker, 1982; Petitjean, 2002).
Using point clouds we take a two-step approach, consisting of
segmentation followed by Hough transform based object
detection. In the first step we use a simple region growing based
segmentation using what we call Smoothness Constraint. lt is
based on the assumption that most of the surfaces in industrial
environments can be expected to be smooth with their surface
normals changing rapidly only on the object edges. First of all
we estimate the surface normal for each point in the point cloud
using plane fitting to the points within a small neighbourhood.
This is followed by the stage of region growing in which we
keep on adding points to one region until the angle between
normals exceeds a specified threshold. Actually, segmentation
and object recognition are two related problems, because if we
know the type and location of objects, segmentation is reduced
to selecting points having a low distance from the object
surface; and similarly if we have a perfect segmentation, the
object recognition is just a matter of surface fitting and finding
the surface which gives minimum error of fit. Most of the
segmentation approaches to date haven’t been able to achieve a
high success rate (Hoover et al. 1996; Min et al., 2000). The
segmentation approach we use leads usually to under-
segmented results, with multiple objects being assigned to one
segment. The following object recognition stage detects the
planes and cylinders present in the segments using a Hough
Transform. As presence of multiple objects and outliers is not a
problem for the Hough transform we are able to recover from
the errors of the preceding stage of segmentation. The object
Internat
detectio
many ii
70% of
Next is
been m
which n
betweer
against
segmeni
Catalog
automat
The ne»
of prec
resulted
belong
measure
assigned
fitting t
image n
in the fc
3.1 Fit
The pro
We hav
that can
estimate
which 1
distance
Q defin
surface
paramet
N point
To solw
method
its partiz
The cal
bounded
possible
compari
we refer
we use
modelli:
derivati
As note
step-size
approxir
solution
For min
the CSC
1996; Pi
CSG pa
given by
