Mini project 4 · CMU 16-720
Two-view 3D reconstruction.
Recovering the 3D shape of an object from two different images of it, going from raw correspondences through epipolar geometry to a metric point cloud.
The pipeline
The goal is a 3D reconstruction of an object from two different images. After finding corresponding points, I estimated the fundamental matrix with the eight-point algorithm, then combined it with the calibrated camera intrinsics to compute the essential matrix, and triangulated the 2D correspondences into a 3D metric reconstruction.
The distinction between those two matrices is the crux. The fundamental matrix relates pixel coordinates between views and knows nothing about the cameras; folding in the intrinsics turns it into the essential matrix, which encodes actual relative rotation and translation, and only then can triangulation produce real geometry rather than something correct up to an unknown projective transformation.
I then added automatic point matching using the epipolar constraint, which reduces the search for a match from the whole second image to a single line. In the figure below, points selected on the left image lie on the corresponding epipolar lines on the right.
