CV Chapter 7 3D Reconstruction

1. For checkerboard pattern and 2. with DLT?

1. Perform canny edge detection

2. Fit straight lines to detect linked edges

3. Intersect lines to obtain corners

4. Obtain points with localization accuracy < 1/10 pixel

Solve x_i x P*X_i = [0,-X_i^T,y_iX_i^T;X_i^T,0,-x_iX_i^T;-y_iX_i^T,x_iX_i^T,0] = 0 via SVD

1. Only two linearly independent equations

2. P has 11 DoF because 12^th parameter scale is arbitrary

3. 5½ correspondences are needed for minimal solution

4. For coplanar points with Prod^T*X=0 we get degenerate solution

5. Calibration points in more than one plane needed

1. Scale image points x_i with RMS distance sqrt(2) to image origin

2. Scale 3D points X_i with RMS distance sqrt(3) to world origin

3. More accurate algorithms than DLT available

4. Correct radial distortion

1. Vectors x, R*x’ and t are coplanar

2. Calibration matrices K and K’ are unknowns with x=K*~x and x’=K*~x’

3. ~x^T*E*~x’=0 → x^T*F*x’=0 with fundamental matrix F =K^-T*E*K’^-1

4. Estimate F with eight-point algorithm

1. Find interest points and compute correspondences

2. Approximatively correspondences enough for F

3. Use RANSAC for robust estimation

1. Compute point on third image with known correspondences

2. Compute lines l₃₁=F^T₁₂*x₁ and l₃₂=F^T₂₃*x₂

3. when does it fail?

1. Rewrite (u,v,1)*F*(u’;v’;1)=0 → (u’u,u’v,u’,uv’vv’,v’,u,v,1)*F=0

2. Fill with eight correspondences

3. Solve using SVD

Frobenius norm

1. Generalized Euclidean norm for matrices

2. |A|_F = sqrt(Sum_i=1^mSum_i=1ⁿ|a_ij|²) = sqrt(Sum_i=1^min(m,n)sigma_i²)

3. sigma_i singular values of A

4. A_p = U_pD_pV_p^T for sigma_p+1 to sigma_N set to zero

5. Then A_p is best rank-p approximation of A

1. No epipoles through which all epipolar lines pass

2. Enforce rank-2 constraint using SVD setting d₃₃ to zero

3. Still poor numerical conditioning → rescale

1. Center image data at origin and scale to mean squared distance of 2 pixels

2. Obtain F from eight-point algorithm

3. Enforce the ran-2 constraint and reconstruct to F’

4. Fundamental matrix is T^T*F*T’

With structured light

1. Principle based on optical triangulation

2. But replacing one camera by projector

1. Xbox Kinect

2. Laser scanning

3. Poor man’s scanner

1. IR projector

2. IR camera for depth

3. Regular camera for color

1. Project single stripe of laser light

2. Scan across surface of object

3. Very precise

1. Desk lamp as light source

2. Stick to simulate stripe

3. Camera to scan

1. Single stripe

2. Multi-stripe triangulation

3. Phase-shift scanning

4. Gray code

5. High-speed 3D scanner

Project multiple stripes instead of one as in laser scanning

1. Assume surface continuity

2. Colored stripes

3. Time coded stripes with unique illumination code

1. Project continuous (3 sinusoid) patterns shifted by 120°

2. For each pixel compute relative phase from 3 intensities with dPhi = arctan(sqrt(3)*I_r-I_b / 2*I_g-I_r-I_b)

3. Recover absolute phase using 2^nd camera

Dynamically moving objects, causing one moving point to correspond to different 3D points

Use geometric error model

Given m images of n 3D points x_ij = P_iX_j

Absolute scale is impossible to recover, because PX = P/k * kX = P*Q^-1*Q*X with factor k

1. Euclidean

2. Similarity

3. Affine

4. Projective

1. Angles, lengths preserved

2. 6 DoF

3. Q_E = [R,t;0^T,1]

1. Angles, length ratios preserved

2. 7 DoF

3. Q_S = [sR,t;0^T,1]

1. Parallelisms, volume ratios preserved

2. 12 DoF

3. Q_A = [A,t;0^T,1]

1. Intersection, tangency preserved

2. 15 DoF

3. Q_P = [A,t;v^T,v]

1. With no constraints on camera calibration we obtain Q_P

2. Upgrade transformation with additional information

1. Special case of perspective projection with infinite focal length

2. Projection matrix [1,0,0,0;0,1,0,0;0,0,0,1]

3. Projection can be orthographic or parallel

4. Tolerable if scene points are far away