In: Paparoditis N., Pierrot-Deseilligny M.. Mallet C... Tournaire O. (Eds), IAPRS. Vol. XXXVIII. Part ЗА - Saint-Mandé, France. September 1-3. 2010 
Figure 3: From top to bottom: part of reference image, over 
segmentation using polygons (contours in red), view-centered 
model (texture and normals) reconstructed from 3 poses (in red). 
www.O—360.com, 2010. 
www.ptgrey.com, 2010. 
Zitnick, C. and Kang, S., 2007. Stereo for image-based rendering 
using image over-segmentation. 1JCV 75(1), pp. 49-65. 
APPENDIX A: POINT AND PLANE THRESHOLDING 
Let p be a 3D point. The covariance matrix C p of p is pro 
vided by ray intersection from p projections in images J\f(i) = 
{i — k i i + k} using Levenberg-Marquardt. In this pa 
per, ray intersection and covariance C p result from the angle er 
ror in (Lhuillier, 2008b). The Mahalanobis distance D p between 
points p and p' is D p (p') = y/(p - p') T C P 1 (p - p')- 
We define points p = c* + zu and p' = c, + z'u using camera 
location cray direction u and depths 2, z'. If 2 is large enough, 
u is a good approximation of the main axis of C p : we have C p ss 
cr p uu 7 and u r C p 1 u a p 2 where o p is the largest singular 
value of C p . In this context, we obtain D p (p') ss — ■■ ^. 
If x has the Gaussian distribution with mean p and covariance 
C p , -D p (x) has the A’ 2 distribution with 3 d.o.f. We decide that 
points p and p' are the same (up to error) if both D p (p') and 
Dp, (p) are less than the 90% quantile of this distribution: w ; e 
decide that p and p' are the same point if D{p. p') < 2.5 w'here 
D(P- p') = max{D p (p'), D p , (p)} « } • 
Let tt be the plane n r x-M = 0. The point-to-plane Mahalanobis 
distance is D p (n) = min x e»r D p (x) = fn nr P c + p - (Schindler 
and Bischof, 2003). Thus C p ^ a p uu 7 and p' G 77 imply 
D 2 p (n) 
(n 7 p'+d+n 1 (p—p'))~ (z — z') 2 
<т2(п ' r U ) ! 
D 2 P (p). 
Last, we obtain the point-to-plane thresholding and distance used 
in Section 2.5. We decide that p is in plane n if D{p. p') < 
2.5 where p' 6 77. The robust distance between p and tt is 
min{D(p. p'), 2.5} ~ min{ 
min{<Tp ,(T / } ’ 
2.5}, z' = 
n 1 c v +d 
u 
APPENDIX B: CONSTRAINED BUNDLE ADJUSTMENT 
In Section 2.3, we would like to apply CBA (constrained bun 
dle adjustment) summarized in (Triggs et al., 2000) to remove 
the drift. This method minimizes the re-projection error func 
tion x h-> /(x) subject to the drift removal constraint c(x) = 0, 
where x concatenates poses and 3D points. Here w'e have c(x) = 
xi — xf where xi and xf concatenate 3D locations of images 
j\f (?) and their duplicates of images A r (n + k), respectively. All 
3D parameters of sequence {0 • ■ • n + 2A-} are in x except the 3D 
locations of A r {j) and N (?i + k). During CBA, xf is fixed and 
xi evolves towards xf. 
How'ever. there is a difficulty with this scheme. CBA iteration 
(Triggs et ah, 2000) improves x by adding step A which mini 
mizes quadratic Taylor expansion of / subject to 0 % c(x + A) 
and linear Taylor expansion c(x + A) ^ c(x) + CA. We use no 
tations x 7 = (xf xf ). A r = (Af’ A-;T), C = (Ci C2) 
and obtain Ci = I. C2 = 0. Thus, we have Ai = —c(x) at the 
first CBA iteration. On the one hand, A] = — c(x) is the drift 
and may be very large. On the other hand. A should be small 
enough for quadratic Taylor approximation of /. 
The “reduced problem” in (Triggs et ah, 2000) is used: BA itera 
tion minimizes the quadratic Taylor expansion of A2 g(A-2) 
where g(A- 2 ) = /(A(A- 2 )) and A(A 2 ) r = (-c(x) r A-f). 
Step A2 meets T/2(A)A 2 = -g 2 , where (A, g 2 .// 2 (A)) are 
damping parameter, gradient and damped hessian of g. Update 
x <— x + A(A 2 ) holds if^(A 2 ) < min{l.l/o,.9(0)}, where 
/0 is the value of /(x) before CBA. It can be shown that this 
inequality is true if c(x) is small enough and A is large enough. 
Here we reset c. by c n at the n th iteration of CBA to have a small 
enough c(x). Let xf be the value of xi before CBA. We use 
c„(x) = xi — ((1 — 7n)x ( i 1 + 7nxf), w'here 7« increases pro 
gressively from 0 (no constraint at CBA start) to 1 (full constraint 
at CBA end). One CBA iteration is summarized as follows. First, 
estimate A 2 (7 fl ) for the current value of (A,x) (a single linear 
system 7/ 2 (A)X = Y is solved for all 7 n e [0,1]). Second, try 
to increase 7 „. such that g( A 2 (7 n )) < min{l.l/ 0 , ^(0)}. If the 
iteration succeeds, apply x <— x + A(A 2 ). Furthermore, apply 
A +— A/10 if 7n = 7n-i. If the iteration fails, apply A <— 100A. 
If 7rj > 7n-i or 7„ = 1, choose 7„+i = at the (n + l) th 
iteration to obtain A(A- 2 ) T = (o' A^) and to decrease / as 
soon (or much) as possible.