CV Chapter 7 3D Reconstruction

1. Image axes are perpendicular a₁*a₂=0

2. Scale is 1 [a₁|²=[a₂|²=1

3. Yields system of 3m equations

1. ^a_i1*^a_i2=0 → a_i1^TCC^Ta_i2=0

2. |^a_i1|=1 → a_i1^TCC^Ta_i1=1

3. |^a_i2|=1 → a_i2^TCC^Ta_i2=1

1. Problem is to estimate C

2. Solve L and recover by Cholesky decomposition L=CC^T

3. Update M=MC and S=C^-1S

1. Typically assumption that all points are visible is not true

2. Decompose into dense sub-blocks // NP-complete

3. Fuse results

4. Incremental bilinear refinement

1. Perform factorization on sub-block

2. Solve for new 3D point visible by >=2 known cameras

3. Solve for new camera that sees >=3 known 3D points

1. Problem is to estimate all P_i and X_j

2. 2mn knowns and 11m+3n-15DoF unknowns // For 2 views we need 7 points

1. If fundamental matrix F is computable

2. Then two projection matrices can be estimated

1. If depth z is known then factorize D to estimate M and S

2. If M and S are known then solve for depth z

3. Use iterative method alternating between above two steps

1. Initialize motion from two images with F and structure

2. For each additional view perform calibration, triangulation and bundle adjustment

Determine projective matrix via additional points

Refine structure re and computing points

1. Refine structure and motion

2. Non-linear method

3. Minimize mean-square reprojection error E(P,X) = Sum_i=1^mSum_i=jⁿD(x_ij,P_iX_j)²

4. Seeks maximum likelihood assuming Gaussian noise

Generally used as final step of any multi-view reconstruction algorithm with good initialization

1. Do not solve, can already be useful answering lines intersecting with planes

2. Euclidean upgrade with new knowledge of calibration or markers

3. Self-calibration

1. Determine intrinsic parameters from uncalibrated images

2. Constraint that parameters of one camera remain fixed

1. Baseline

2. Apply RANSAC // Incorrect matches or moving objects

3. Estimation dependent on point location // Far points stable

1. Small yields large depth error

2. Large yields difficult search problem

3. Track feautres between frames until sufficient

1. Use calibrated cameras wherever possible

2. Perform SfM with two cameras

3. Any constraint on setup can be useful

1. Square pixel, zero skew, fixed focal length

2. Fixed baseline in stereo SfM setup

3. Constrained camera motion on a ground plane

Might need adapting algorithm

1. Difficult for large motion or field-of-view and depth variation

2. Limited camera calibration

3. Good feature trackers required

1. boujou

2. PFTrack

3. MatchMover putting virtual objects in videos

4. SynthEyes

5. Icarus

6. Voodoo camera tracker

7. Large-scale SfM by Flickr