Visual Hulls from Uncalibrated Snapshots


Latest versions of code now live at https://github.com/keith2000/VisualHullMesh and https://github.com/keith2000/VisualHullFromMirrors.

Download Matlab, C++ Visual Hull Code (This code is for visual hulls but not the double mirror method.)

Download MirrorDemo2012.zip (This Matlab code is a demo for the double mirror method.)


This page illustrates our method of forming visual hulls from uncalibrated snapshots using two planar mirrors. For more details please read our paper.

The only equipment required is a consumer grade digital camera and two bathroom-style mirrors. The method does not require any accurate positioning of apparatus, nor does it require calibration objects or markers. The silhouette poses are computed from the silhouette outlines in the image.

The object to be modelled is placed so that five views can be seen. One of the views is straight onto the object; two of the views are reflections; and two of the views are reflections of reflections.

The silhouette outlines provide enough constraints to compute the pose associated with each silhouette and the camera parameters.

VRML model of 10-view visual hull of a toy locust (146 kilobytes) formed from the two images below.

You'll need a VRML plugin for your browser to view the 3D model. I use Cortona VRML Client 4.0 which is available for free from http://www.parallelgraphics.com/products/downloads/.

toy locust

toy locust

Visual hull model computed from the two images above:

10-view visual hull


Matlab code is available for computing the self-calibration and the visual hulls. You will need to be running Windows (as there are some MEX file DLLs), and the Optimization Toolbox is needed. Run the script MirrorDemo2.m to form a visual hull model from segmented images.

Take two or more snapshots of your scene without changing the zoom between shots, and without moving the object or the mirrors. Segment the images, and then change the Matlab code (MirrorDemo1.m or MirrorDemo2.m) to reference your segmented images. Ensure that your silhouettes do not overlap one another. If they do, you should move the object away from the mirrors. You may need to use larger mirrors to do this.

Download MirrorDemo2012.zip (837 kilobytes) (updated many years later! works without toolboxes: this is the one you want [2012-05-29])

Download MirrorDemo2.zip (786 kilobytes) (updated, faster Matlab code)

Download DoubleMirror.zip (712 kilobytes) (original Matlab code)

[Note: (2009-07-01) This code does not seem to work with the more recent versions of Matlab. It does work on Windows R2006a with the Optimization Toolbox and the Image Processing Toolbox.]

If you use this code, please send me an email (keith2000@gmail.com) to let me know how well the method worked.

Update (3 October 2005): I have modified the code so that you no longer need two or more images of the object. You can now build visual hulls from single snapshots. Using more than one snapshot, however, will give much better results (provided that the viewpoints differ sufficiently).

Update (11 September 2009): Several people have asked me for visual hull source code that works without the mirror setup. I have put together a simple package that computes the visual hull mesh from silhouette images and associated camera parameters. Download Matlab, C++ Visual Hull Mesh Code.


VRML model of a horse (197 kilobytes) created using the double mirror method.


Reconstructing the 3D shape of a lion...

lion

lion

Resultant 10-view visual hull:

lion

VRML model of 10-view visual hull of a toy lion (483 kilobytes).


George took some photos of a toy moose using a backlight.

Three input images, with the camera fixed between shots:

moose

moose

moose

Resultant 15-view visual hull:

moose

VRML model of 15-view visual hull of a toy moose (211 kilobytes).

Matlab code for doing this type of self-calibration (the camera stays fixed, but the object moves between shots) is available. Note that you will need to use some sort of tripod to keep the camera fixed. Run the script StaticCameraMovingObjectDemo1.m. You can replace my segemented images with your own. Download StaticCameraMovingObjectDemo1.zip (465 kilobytes) (19 May 2006, requires a Windows version of Matlab with the Optimization Toolbox)

I also have code for the case in which neither the camera nor the object is fixed. This means that you don't need to use a tripod, and you can move the object into different poses to get a good range of views. I will post the Matlab code here at a later stage. The method is described in a report titled "Creating Visual Hull Models Using Only Two Mirrors and an Uncalibrated Perspective Camera."


Another example using Mathew's coffee cup...

Three input images, with only the camera moving between shots:

cup

cup

Epipoles computed from silhouette outlines:

cup

cup

Resultant 15-view visual hull:

cup


Mathew Price has used the foreground to create a 3D textured model of a toy cheetah.

One of the input images:

cheetah

VRML model cheetah with texture (638 kilobytes).