International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B3, 2012 
XXII ISPRS Congress, 25 August — 01 September 2012, Melbourne, Australia 
  
id 
(b) Projection based on incorrect model parameters 
Figure 4: Precise image orientation and model parameters are 
both required for correct model projection 
In this paper, the image orientation parameters of the aerial 
photographs are known and fixed, while floating models are 
used for reconstructing geometric 3D building models. These 
reconstructed geometric models are later used for refining the 
image orientation parameters of the close-range photographs 
taken by personal mobile computing devices. So the 
reconstructed model parameters remain fixed while fitting 
floating model to the close-range photographs. 
3. LEAST-SQUARES MODEL-IMAGE FITTING 
The principle of model-image fitting algorithm is to adjust 
either model parameters or the image orientation parameters, so 
the model projection fit the building images. Since the floating 
model can be taken as a wire-frame model, the edge pixels are 
selected as fitting targets. The optimal fit is achieved by 
minimizing the sum of the perpendicular distances from the 
edge pixels to the corresponding projected line of the wire- 
frame model. Figure 5 depicts the optimal fitting procedure. 
The selected primitive model is projected onto the image and fit 
the extracted edge pixels 
Edge : 
JE 20 
Model Base 
Projection 
Figure 5: Optimal model-image fitting 
  
  
Optimal Fitting 
  
  
  
  
  
  
  
  
  
Either for geometric modeling or image orientation, an 
approximate fitting is required before applying the LSMIF 
algorithm. An interactive program is developed for model 
selection, approximate fitting, and visualization. To obtain as 
close as to the right fitting, this program provides a user 
interface that allows the operator to resize, rotate, and move a 
model to fit the corresponding building images approximately. 
Benefited from the approximate fitting, the LSMIF iteratively 
pulls the model to the optimal fit instead of blindly searching 
for the solution. To avoid the disturbance of irrelevant edge 
pixels, only those edge pixels distributed within the specified 
buffer zones will be used in the calculation of the fitting 
algorithm. Figure 6 depicts the extracted edge pixels Tj; and 
the buffer determined by a projected edge v;;v;; of the model. 
The suffix i represents the index of edge line, j represents the 
index of overlapped image, and K represents the index of the 
edge pixel. Filtering edge pixels with buffer is reasonable, 
because the discrepancies between the projected edges and the 
corresponding edge pixels should be small, as either the model 
parameters or the image orientation parameters are 
approximately known. 
*. Extracted Pixels 
» Yi) 
  
   
  
5 : > Tx jk» V ji) 
> ijk? * oe Se 
© . 
e e e 
$. Dr 3 
"e, . 
4 Vi (xo Va) 
— x 
  
Figure 6: Extracted edge pixels and buffer 
The optimal fitting condition we are looking for is the projected 
model edge line exactly falls on the building edges in the 
images. In Eq.(1), the distance dj, represents a discrepancy 
between an edge pixel Tj; and its corresponding edge line v;,,, 
which is expected to be zero. Therefore, the objective of the 
fitting function is to minimize the squares sum of dj. Suppose 
a projected edge line is composed of the projected vertices 
vi(Xij, Yır) and v;2(X;2, iz), and there is an edge pixel Ty (x, 
Vi) located inside the buffer. The distance d;, from the point 
Tj to the edge viv; can be formulated as the following 
equation: 
5 IO —Vn)X x + (Ay TX a * OX - Jas) (I) 
J Xn) +On —yp) 
where i = the index of the edge line 
J = the index of the overlapped image 
k = the index of the edge pixel 
d, 
ik 
  
  
The photo coordinates v;;(x;;, V;1) and v;2(X;2, V;2) are functions 
of the unknown model parameters, comparatively the exterior- 
orientation parameters of photos are known. Therefore, dj will 
be a function of the model parameters. Taking a box model for 
instance, dj; Will be a function of w, /, h, a, dX, dY, and dZ, 
with the hypothesis that a normal building rarely has a tilt angle 
(f) or swing angle (s). The least-squares solution for the 
unknown parameters can be expressed as: 
Sdyl = X[Fg(w,l,h, a, dX, dY,dZ)] — min. 0) 
Eq.(2) is a nonlinear function with regard to the unknowns, $0 
that the Newton's method is applied to solve for the unknowns. 
The nonlinear function is differentiated with respect to the 
unknowns and becomes a linear function with regard to the 
increments of the unknowns as follows: 
oF, oF, oF, oF, 
din —Fino 1 d x J [s i Act (3) 
ow J, a ), oh ), a jJ, 
oF, oF, oF, 
B | AdX+| — | MY+| == | AZ 
odX ), adY J, àdZ ), 
in which, Fj, is the approximation of the function 5 
calculated with given approximations of the unknow 
  
  
  
  
  
   
  
Intern 
parameters. 
squares sol 
and the @ 
Repeating 
solved iter 
model reco 
they are mx 
the increme 
X di 
di ri 
The lineari 
V-AX-L, v 
vector of th 
Vis the ve 
be express 
solved by 
normally : 
inadequate 
features or 
the comput 
Photo-reali 
informatior 
proposed a 
on least-sq; 
bottleneck. 
mapping it 
are used to 
the built-in 
orientation 
taken. Thei 
iteratively. 
results will 
The author 
the Nation: 
2221-E-00 
Braun, C., 
Cremers, / 
for Photog 
Graphics,