CV Chapter 7 3D Reconstruction

Questions about the lecture 'Computer Vision' of the RWTH Aachen Chapter 7 3D Reconstruction

108

0.0 (0)

R. B.

Lernen

R. B.

Über diese Lernkartei

This flashcard set covers advanced university-level concepts in 3D reconstruction within computer science, focusing on techniques like structured light, affine transformations, and projective geometry. It delves into key aspects such as camera calibration, point matching, and the characteristics of different reconstruction methods. The flashcards are particularly useful for students and researchers in computer vision and 3D modeling, providing insights into solving complex reconstruction problems and understanding the underlying mathematical principles.

Karten

108

Lernende

Sprache

Englisch

Kategorie

Informatik

Stufe

Universität

Erstellt / Aktualisiert

04.02.2017 / 23.02.2017

Ähnliche Karteien

How does it work?

[euclidean upgrade.affine.SfM.3D resconstruction, 3]

1. Problem is to estimate C

2. Solve L and recover by Cholesky decomposition L=CC^T

3. Update M=MC and S=C^-1S

How to deal with missing data?

[affine.SfM.3D resconstruction, 4]

1. Typically assumption that all points are visible is not true

2. Decompose into dense sub-blocks // NP-complete

3. Fuse results

4. Incremental bilinear refinement

How does it work?

[incr bilin refinement.missing data.affine.SfM.3D resconstruction, 3]

1. Perform factorization on sub-block

2. Solve for new 3D point visible by >=2 known cameras

3. Solve for new camera that sees >=3 known 3D points

What is the problem which has to be solved?

[projective.SfM.3D resconstruction, 2]

1. Problem is to estimate all P_i and X_j

2. 2mn knowns and 11m+3n-15DoF unknowns // For 2 views we need 7 points

What holds for the two camera case?

[projective.SfM.3D resconstruction, 2]

1. If fundamental matrix F is computable

2. Then two projection matrices can be estimated

How does it works?

[proj factorization.projective.SfM.3D resconstruction, 3]

1. If depth z is known then factorize D to estimate M and S

2. If M and S are known then solve for depth z

3. Use iterative method alternating between above two steps

How does it works?

[seq struture.projective.SfM.3D resconstruction, 2]

1. Initialize motion from two images with F and structure

2. For each additional view perform calibration, triangulation and bundle adjustment

How does it works?

[calibration.seq struture.projective.SfM.3D resconstruction]

Determine projective matrix via additional points

How does it works?

[triangulation.seq struture.projective.SfM.3D resconstruction]

Refine structure re and computing points

How does it works?

[bundle adjustement.seq struture.projective.SfM.3D resconstruction, 4]

1. Refine structure and motion

2. Non-linear method

3. Minimize mean-square reprojection error E(P,X) = Sum_i=1^mSum_i=jⁿD(x_ij,P_iX_j)²

4. Seeks maximum likelihood assuming Gaussian noise

What is the characteristic?

[bundle adjustement.seq struture.projective.SfM.3D resconstruction]

Generally used as final step of any multi-view reconstruction algorithm with good initialization

How to solve projective ambiguity?

[projective.SfM.3D resconstruction, 3]

1. Do not solve, can already be useful answering lines intersecting with planes

2. Euclidean upgrade with new knowledge of calibration or markers

3. Self-calibration

How does it work?

[self-calibration.projective.SfM.3D resconstruction, 2]

1. Determine intrinsic parameters from uncalibrated images

2. Constraint that parameters of one camera remain fixed

What are the practical considerations?

[projective.SfM.3D resconstruction, 3]

1. Baseline

2. Apply RANSAC // Incorrect matches or moving objects

3. Estimation dependent on point location // Far points stable

What are the characteristics?

[baseline.prac considerations.projective.SfM.3D resconstruction, 3]

1. Small yields large depth error

2. Large yields difficult search problem

3. Track feautres between frames until sufficient

What are the guidelines?

[projective.SfM.3D resconstruction, 3]

1. Use calibrated cameras wherever possible

2. Perform SfM with two cameras

3. Any constraint on setup can be useful

What are possible ones?

[constraints.guidelines.projective.SfM.3D resconstruction, 3]

1. Square pixel, zero skew, fixed focal length

2. Fixed baseline in stereo SfM setup

3. Constrained camera motion on a ground plane

What holds?

[constraints.guidelines.projective.SfM.3D resconstruction]

Might need adapting algorithm

What are limitations?

[SfM.3D reconstruction, 3]

1. Difficult for large motion or field-of-view and depth variation

2. Limited camera calibration

3. Good feature trackers required

Name commercial SW packages?

[SfM.3D reconstruction, 7]

1. boujou

2. PFTrack

3. MatchMover putting virtual objects in videos

4. SynthEyes

5. Icarus

6. Voodoo camera tracker

7. Large-scale SfM by Flickr

What are cues for 3D recovery? [5]

Tastatur-Befehle:

= drehen,

= vor-/rückwärts,

= scrollen

1. Shading

2. Texture

3. Focus

4. Perspective

5. Motion

Which kind of geometry is necessary for stereo vision?

Tastatur-Befehle:

= drehen,

= vor-/rückwärts,

= scrollen

Multi-view geometry

What is the generic problem formulation for stereo vision?

Tastatur-Befehle:

= drehen,

= vor-/rückwärts,

= scrollen

“Given several images of an object or scene, compute 3D shape”

What is the narrower problem formulation for stereo vision?

Tastatur-Befehle:

= drehen,

= vor-/rückwärts,

= scrollen

“Given calibrated binocular stereo pair, fuse it to depth image”

What is the definition of triangulation in epipolar geometry?

Tastatur-Befehle:

= drehen,

= vor-/rückwärts,

= scrollen

Reconstruction as intersection of two rays

What is required for triangulation in epipolar geometry? [2]

Tastatur-Befehle:

= drehen,

= vor-/rückwärts,

= scrollen

1. Point correspondence

2. Camera pose or calibration

What are the two different set of parameters for camera calibration in triangulation in epipolar geometry? [2]

Tastatur-Befehle:

= drehen,

= vor-/rückwärts,

= scrollen

1. Extrinsic and 2. intrinsic

What does the extrinsic parameters describe for camera calibration in triangulation in epipolar geometry?

Tastatur-Befehle:

= drehen,

= vor-/rückwärts,

= scrollen

Camera ↔ reference frame

What does the intrinsic parameters describe for camera calibration in triangulation in epipolar geometry?

Tastatur-Befehle:

= drehen,

= vor-/rückwärts,

= scrollen

Image ↔ pixel coordinates // relative

What are the extrinsic parameters for camera calibration in triangulation in epipolar geometry? [2]

Tastatur-Befehle:

= drehen,

= vor-/rückwärts,

= scrollen

1. Rotation matrix

2. Translation vector

What are the intrinsic parameters for camera calibration in triangulation in epipolar geometry? [6]

1. Focal length

2. Pixel sizes [mm]

3. Image center point

4. Radial distortion

5. Pixel magnification factors

6. Skew (non-rectangular pixels)

Name the list of attributes necessary if we have parallel optical axes and known camera parameters in a simple stereo system? [6]

1. World point p and image point left p_l and right p_r

2. x_l and x_r

3. Focal length f

4. Optical center left O_l and right O_r

5. Baseline T

6. Depth Z of p

How does triangulation works if we have parallel optical axes and known camera parameters in a simple stereo system? [4]

1. (p_l, p, p_r) and (O_l, p, O_r)

2. (T-(x_r-x_l))/(Z-f) = T/Z

3. Z = f*T/(x_r-x_l) with (x_r-x_l) disparity

4. Update (x’,y’) with disparity map (x+D(x,y), y)

How do we proceed if optical axes are not parallel in a simple stereo system? [3]

1. Image scanlines are epipolar lines

2. Re-project image on plane parallel to optical centers // Stereo image rectification

3. Two homographies (3x3 transforms) are necessary

What is the definition of a baseline T in epipolar geometry?

Line joining camera centers

What is the definition of a epipole e in epipolar geometry?

Intersection of baseline and image plane

What is the definition of a epipolar plane p in epipolar geometry?

Plane containing baseline and world point

What is the definition of a epipolar line l in epipolar geometry?

Intersection of p and image plane

What is the stereo correspondence constraint and its problem? [2]

1. Reduce to 1D search along conjugate epipolar lines

2. But still multiple match hypotheses satisfy constraint

How is the rotational 3x3 R matrix defined for epipolar geometry? [3]

1. 3D rotation as series of rotations around three axes

2. Axis column and row filled with 1 and 0

3. Other entries are cos, *sin, °sin, cos with *=- for x,z and °=- for y

1 / 108

Lernkarten

What is the effect?

[camera.affine.SfM.3D resconstruction, 3]

1. Affine transformation of 3D space

2. Orthographic projection

3. Affine transformation of image

What is the problem which has to be solved?

[affine.SfM.3D resconstruction, 2]

1. Problem is to estimate all A_i, b_i and X_j for linear mapping plus translation x = AX + b

2. 2mn knowns and 8m+3n-12DoF unknowns // For 2 views we need 4 points

How does centering works?

[affine.SfM.3D resconstruction, 2]

1. Subtract centroid of image points

2. ^x_ij=x_ij – 1/n Sum_k=1ⁿ x_ik = A_i*^X_j

How is the matrix decomposed?

[measurement matrix.affine.SfM.3D resconstruction]

D = U*W*V^T= U₃*W₃*V₃^T = M*S

What are the characteristics?

[measurement matrix.affine.SfM.3D resconstruction, 4]

1. With measurement matrix D

2. Set to rank 3 due to singular values of 0 except of first 3

3. M = U₃*W₃^1/2 and S = W₃^!/2*V₃^T

4. Could be achieved by any 3x3 matrix C → euclidean upgrade

How to solve affine ambiguity?

[affine.SfM.3D resconstruction]

Via euclidean upgrade

What is the definition?

[orthographic assumption.euclidean upgrade.affine.SfM.3D resconstruction, 3]

1. Image axes are perpendicular a₁*a₂=0

2. Scale is 1 [a₁|²=[a₂|²=1

3. Yields system of 3m equations

What are the equations?

[orthographic assumption.euclidean upgrade.affine.SfM.3D resconstruction, 3]

1. ^a_i1*^a_i2=0 → a_i1^TCC^Ta_i2=0

2. |^a_i1|=1 → a_i1^TCC^Ta_i1=1

3. |^a_i2|=1 → a_i2^TCC^Ta_i2=1

How does it work?

[euclidean upgrade.affine.SfM.3D resconstruction, 3]

1. Problem is to estimate C

2. Solve L and recover by Cholesky decomposition L=CC^T

3. Update M=MC and S=C^-1S

How to deal with missing data?

[affine.SfM.3D resconstruction, 4]

1. Typically assumption that all points are visible is not true

2. Decompose into dense sub-blocks // NP-complete

3. Fuse results

4. Incremental bilinear refinement

How does it work?

[incr bilin refinement.missing data.affine.SfM.3D resconstruction, 3]

1. Perform factorization on sub-block

2. Solve for new 3D point visible by >=2 known cameras

3. Solve for new camera that sees >=3 known 3D points

What is the problem which has to be solved?

[projective.SfM.3D resconstruction, 2]

1. Problem is to estimate all P_i and X_j

2. 2mn knowns and 11m+3n-15DoF unknowns // For 2 views we need 7 points

What holds for the two camera case?

[projective.SfM.3D resconstruction, 2]

1. If fundamental matrix F is computable

2. Then two projection matrices can be estimated

How does it works?

[proj factorization.projective.SfM.3D resconstruction, 3]

1. If depth z is known then factorize D to estimate M and S

2. If M and S are known then solve for depth z

3. Use iterative method alternating between above two steps

How does it works?

[seq struture.projective.SfM.3D resconstruction, 2]

1. Initialize motion from two images with F and structure

2. For each additional view perform calibration, triangulation and bundle adjustment

How does it works?

[calibration.seq struture.projective.SfM.3D resconstruction]

Determine projective matrix via additional points

How does it works?

[triangulation.seq struture.projective.SfM.3D resconstruction]

Refine structure re and computing points

How does it works?

[bundle adjustement.seq struture.projective.SfM.3D resconstruction, 4]

1. Refine structure and motion

2. Non-linear method

3. Minimize mean-square reprojection error E(P,X) = Sum_i=1^mSum_i=jⁿD(x_ij,P_iX_j)²

4. Seeks maximum likelihood assuming Gaussian noise

What is the characteristic?

[bundle adjustement.seq struture.projective.SfM.3D resconstruction]

Generally used as final step of any multi-view reconstruction algorithm with good initialization

How to solve projective ambiguity?

[projective.SfM.3D resconstruction, 3]

1. Do not solve, can already be useful answering lines intersecting with planes

2. Euclidean upgrade with new knowledge of calibration or markers

3. Self-calibration

How does it work?

[self-calibration.projective.SfM.3D resconstruction, 2]

1. Determine intrinsic parameters from uncalibrated images

2. Constraint that parameters of one camera remain fixed

What are the practical considerations?

[projective.SfM.3D resconstruction, 3]

1. Baseline

2. Apply RANSAC // Incorrect matches or moving objects

3. Estimation dependent on point location // Far points stable

What are the characteristics?

[baseline.prac considerations.projective.SfM.3D resconstruction, 3]

1. Small yields large depth error

2. Large yields difficult search problem

3. Track feautres between frames until sufficient

What are the guidelines?

[projective.SfM.3D resconstruction, 3]

1. Use calibrated cameras wherever possible

2. Perform SfM with two cameras

3. Any constraint on setup can be useful

What are possible ones?

[constraints.guidelines.projective.SfM.3D resconstruction, 3]

1. Square pixel, zero skew, fixed focal length

2. Fixed baseline in stereo SfM setup

3. Constrained camera motion on a ground plane

What holds?

[constraints.guidelines.projective.SfM.3D resconstruction]

Might need adapting algorithm

What are limitations?

[SfM.3D reconstruction, 3]

1. Difficult for large motion or field-of-view and depth variation

2. Limited camera calibration

3. Good feature trackers required

Name commercial SW packages?

[SfM.3D reconstruction, 7]

1. boujou

2. PFTrack

3. MatchMover putting virtual objects in videos

4. SynthEyes

5. Icarus

6. Voodoo camera tracker

7. Large-scale SfM by Flickr

Lernen

CV Chapter 7 3D Reconstruction

Lernkarteien erstellen oder kopieren

Lernkarten

Bitte melde dich an

Google

Apple ID

Microsoft

Switch edu-ID

Edulog

Google

Apple ID

Microsoft

Switch edu-ID

Edulog