». Istanbul 2004
1on-linear least
of minimizing
rtial derivatives
FON Qv
er
in of the partial
en by Ermes et
ly using finite
rg-Marquardt in
on. The details
4
nd point clouds
n, and by their
:'uracy. Edges of
> noisy data are
fitting bounded
formation about
g the full edge
n the case of a
are not scanned
'onnections with
't convenient to
ire visible. As a
r to be poorly
ie measurements
us help improve
the information
do some fitting
ill be fitted three
ing only image
both. The point
rax scanner. We
oint. The images
imera having a
| focal length of
measurements 1s
nent is shown in
, and images are
e back-projected
ges in red, while
te. A cylinder is
n, 3 for the axis,
able 1 we see the
ameters by doing
ation of both. For
, two and three
mages were used.
oud the length of
ence of points on
formation in the
ise singular value
ken out of the
its value remains
sults in standard
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol XXXV, Part B5. Istanbul 2004
s a
Fig. 4: Box fitting experiment (a) Point Cloud (b-d) Images with
back-projected model in yellow, point measurements in red, and
sub-sampled point cloud in white
Para Images Point Both
meter 1 2 3 Cloud
1 X 2.106 1.179 0.685 3.079 0.649
2 X 2.243 1.129 0.261 0.550 0.161
3 Z 1.669 0.760 0.338 389.84 0.314
4 q0 7.8e-1 2.6e-1 3.3e-2 | 3.97e-2 |. 3-0e-2
= ql 1.4e-3 5.0e-4 1.3e-4 | 2.40e-4 1.0e-4
6 q2 7.8e-4 3.0e-4 6.0e-5 | 5.20E-4 | 5.0e-5
7 q3 4.3e-3 1.6e-3 3.4e-A | 3.40E-4 | 2.0e-4
8 | Xsize | 2.855 1.023 | 0.689 2.890 0.661
9 | Ysize | 9.836 2.696 | 0.627 co 0.532
10 | Zsize | 2.309 1.161 | 0.349 oo 0.318
Table 2: Standard deviation for Box fitting experiment
As the z-axis 1s aligned with the length of the cylinder there is a
very high correlation between both of them. As a result the
estimation of z-position is also very weak compared to the
estimation of x and y position. But if we combine the point
cloud with measurements from the images (Table 1, column
"Both") the situation improves dramatically as the edges in the
images provide enough information about the length and the
resulting standard deviations are much lower, indicating much
better estimation precision.
Cylinder axis can be specified using two parameters, but as we
are using 3 without enforcing the constraint there is an over-
parameterisation. Although the standard deviation of axis
parameters look quite good, but due to over parameterisation
their correlation is very high. For example the correlation
between t] and t2 is 0.52, which indicates that the low values of
standard deviations are due to some numerical effects.
As expected as we use more images the standard deviation of
parameter estimation goes down. It also shows that even a
single image in combination with a good scan can lead to
significant improvement in the estimation of those parameters
which are not well-determined from the point cloud.
4.2 Box Fitting
The second example is that of a box, with only two of its faces
fully scanned. Additionally, three images are taken from
different positions (Fig.4). The box has 10 parameters, 3 for the
position, 4 for the rotation, and 3 for the sizes. Again, similar to
the example of cylinder discussed above, we have an over-
parameterisation for rotation, as we use 4 instead of required 3
parameters, and cannot enforce the constraint. Again we find a
very high correlation between different q parameters that lowers
the confidence in the otherwise low standard deviation values.
For example the correlation between q0 and ql 1s 0.511.
In the absence of points on all faces of the box, it is not possible
to reliably determine the size parameters of the box. That's what
we see in the standard deviation resulting from fitting using
only point clouds (Table 2), where the standard deviation for y
and z sizes is o» meaning that they could not be estimated. The
value of standard deviation for x size is low only because of the
coordinate system chosen for the box, which has its origin in the
left corner. This fixes the position of right side and thus the x
size is also determined. Due to high correlation between z-
position and z-size, its estimation 1s also bad.
Once again, we see from the last column of the Table 2 that the
inclusion of image measurements leads to a much better
estimation of size and position parameters.
Both of these examples prove our thesis, that although point
clouds contain direct 3D information, which is very useful for
automatic object recognition, the final adjustment must use a
combination of both data sources to account for missing or
noisy information in point clouds.
4.3 Modelling of an industrial site
We applied the presented methodology for making 3D model of
an industrial site shown in Fig. 5. Seven scans were made using
a Cyrax laser scanner. Each scan consisted of one million points
with a standard deviation of 5mm. Additionally about 60
images were taken from different positions. Following the
modelling pipeline discussed in Section 2 we started with
approximate registration using ICP. The approximately
registered scan was segmented using Smoothness constraint
based region growing. Cylinders and planes were automatically
detected using the Hough transform, and then used to refine the
scan-to-scan registration. For images the orientation was
approximated using vanishing points. This was followed by
scan-to-image registration using a few image measurements and
keeping all object parameters fixed, while estimating only
exterior orientation parameters of the images.
The process of combining automatically detected cylinders and
planes to full CSG objects as well as the process of adding
measurements to images was done manually. Once we have
image measurements as well as segmented points clouds, we
proceed with the Integrated adjustment using both data sources
simultaneously. This integrated adjustment minimizes the sum
of square of the distances of point cloud from the model surface
and sum of square of the image measurement distance from the
back projected edges of the model, while estimating the pose
and shape parameters of the CSG object as well as the
registration parameters of the individual scans and exterior
orientation of the images. This process is an extension of the
idea of bundle adjustment in traditional Photogrammetry but 1s
