Photometric Stereo · Harsh Dhruva

Course 16-720 Computer Vision

Level Graduate

Model Lambertian · orthographic

Focus Shape from shading

Rendering the n-dot-l lighting

Photometric stereo recovers the shape of an object from its appearance under a set of lighting directions. This implementation assumes a Lambertian object imaged with an orthographic camera. To build up the forward model first: for a fully reflective Lambertian sphere centered at the origin, with an orthographic camera placed away from it and three different incoming lighting directions, I simulate the sphere's appearance under the n-dot-l model.

n-dot-l lighting rendering setup — The n-dot-l rendering setup: a Lambertian sphere and an orthographic camera.

Sphere rendered under lighting direction 1 — Lighting direction 1.

Sphere rendered under lighting direction 2 — Lighting direction 2.

Sphere rendered under lighting direction 3 — Lighting direction 3.

Calibrated photometric stereo

Given seven images of a face lit from different directions, with the ground-truth lighting directions, I invert the image formation model. From the images I recover the surface albedo, which indicates how much light hitting the surface is reflected rather than absorbed, and the surface normals, which encode the depth. Integrating the normals with a special case of the Frankot-Chellappa algorithm to enforce integrability then gives the depth, and the reconstructed shape of the face.

Recovered albedo and surface normals — Recovered albedo (left) and false-color surface normals (right).

Depth reconstruction view 1 — Reconstructed shape, view 1.

Uncalibrated, and the bas-relief ambiguity

With no given lighting directions, I estimate the shape directly from the face images. The lighting directions are estimated by decomposing the matrix with SVD, which introduces a linear ambiguity known as the bas-relief ambiguity. Bas-relief is the old sculpting technique of flattened figures protruding from a wall that, viewed from the right angle, look full-depth.

After enforcing integrability with Frankot-Chellappa, the reconstruction looks very similar to the calibrated one, but is only resolved up to a family of transformations: for any parameters greater than zero there is a set of integrable pseudonormals producing the same appearance. This is not a flaw in the implementation but a real property of the problem. Without knowing the lights, the images genuinely cannot distinguish between a family of shapes.

Uncalibrated depth reconstruction — Uncalibrated reconstruction, recovered without the lighting directions.

Examples of varying those parameters are shown below, some of which look identical to the original when viewed from a particular angle.

Bas-relief ambiguity variation 1 — Varying the bas-relief parameters, example 1.

Bas-relief ambiguity variation 2 — Example 2.

Photometric Stereo Shape from Shading Surface Normals Frankot-Chellappa Bas-Relief Ambiguity SVD