Publications

Keith Forbes. Calibration, Recognition, and Shape from Silhouettes of Stones. [36.7MB PDF file, print resolution] PhD thesis, University of Cape Town, June 2007.

Abstract: Multi-view shape-from-silhouette systems are increasingly used for analysing stones. This thesis presents methods to estimate stone shape and to recognise individual stones from their silhouettes. Calibration of two image capture setups is investigated. First, a setup consisting of two mirrors and a camera is introduced. Pose and camera internal parameters are inferred from silhouettes alone. Second, the configuration and calibration of a high throughput multi-camera setup is covered. Multiple silhouette sets of a stone are merged into a single set by inferring relative poses between sets. This is achieved by adjusting pose parameters to maximise geometrical consistency specified by the epipolar tangency constraint. Shape properties (such as volume, flatness, and elongation) are inferred more accurately from the merged silhouette sets than from the original silhouette sets. Merging is used to recognise individual stones from pairs of silhouette sets captured on different occasions. Merged sets with sufficient geometrical consistency are classified as matches (produced by the same stone), whereas inconsistent sets are classified as mismatches. Batch matching is determining the one-to-one correspondence between two unordered batches of silhouette sets of the same batch of stones. A probabilistic framework is used to combine recognition by merging (which is slow, but accurate) with the efficiency of computing shape distribution-based dissimilarity values. Two unordered batches of 1200 six-view silhouette sets of uncut gemstones are correctly matched in approximately 68~seconds (using a \unit[3.2]{GHz} Pentium 4 machine). An experiment that compares silhouette-based shape estimates with mechanical sieving demonstrates an application using the developed methods. A batch of 494 garnets is sieved 15 times. After each sieving, silhouette sets are captured for sub-batches in each bin. Batch matching is used to determine the 15 sieve bins per stone. Better estimates of repeatability, and better understanding of the variability of the sieving process is obtained than if only histograms (the natural output of sieving) were considered. Silhouette-based sieve emulation is found to be more repeatable than mechanical sieving.

1st half of thesis (3.0MB PDF file, screen resolution)
2nd half of thesis (4.7MB PDF file, screen resolution)
psdraft version (730kB PDF file, no figures)

Keith Forbes, Fred Nicolls, Gerhard de Jager and Anthon Voigt. Shape-from-Silhouette with Two Mirrors and an Uncalibrated Camera. In Proceedings of the 9th European Conference on Computer Vision (ECCV), May 2006.

Abstract: Two planar mirrors are positioned to show five views of an object, and snapshots are captured from different viewpoints. We present closed form solutions for calculating the focal length, principal point, mirror and camera poses directly from the silhouette outlines of the object and its reflections. In the noisy case, these equations are used to form initial parameter estimates that are refined using iterative minimisation. The self-calibration allows the visual cones from each silhouette to be specified in a common reference frame so that the visual hull can be constructed. The proposed setup provides a simple method for creating 3D multimedia content that does not rely on specialised equipment. Experimental results demonstrate the reconstruction of a toy horse and a locust from real images. Synthetic images are used to quantify the sensitivity of the self-calibration to quantisation noise. In terms of the silhouette calibration ratio, degradation in silhouette quality has a greater effect on silhouette set consistency than computed calibration parameters.

Conference Poster (A1 size, 3.6MB PDF file)

Keith Forbes. Creating Visual Hull Models Using Only Two Mirrors and an Uncalibrated Perspective Camera. Technical report, 2005.

Abstract: We present closed form solutions for the focal length and pose of a camera and two mirrors that are computed directly from the silhouette outlines that appear in a single image. In the noisy case, we show how these equations can be used to form an initial parameter estimate that can be refined with a nonlinear iterative minimisation. This allows five-view visual hulls to be constructed from a single image. We show how these five-view visual hulls can be used to form initial estimates of the similarity transforms that relate multiple five-view silhouette sets of a rigid object in different poses. A nonlinear iterative minimisation is then used to refine the solution so that all silhouettes are specified in a common reference frame, and visual hulls can be constructed from an arbitrary number of silhouettes. Experimental results demonstrating the reconstruction of a toy horse are presented.

Keith Forbes, Anthon Voigt and Ndimi Bodika. Visual Hulls from Single Uncalibrated Snapshots Using Two Planar Mirrors. In Proceedings of the Fifteenth Annual Symposium of the Pattern Recognition Association of South Africa (PRASA 2004), November 2004.

Abstract: Two mirrors are used to create five views of an object: a view onto the object, two reflections and two reflections of reflections. The five views are captured in a single snapshot. Epipolar geometry of the object's five silhouettes is determined directly from the image without knowing the poses of the camera or the mirrors. The epipolar geometry provides constraints on the pose of each silhouette, allowing the pose of each silhouette to be computed in a common reference frame using only the silhouette outlines. Once the pose associated with each silhouette has been computed, a five-view visual hull of the object can be computed from the five silhouettes. By capturing several images of a rigid object in different poses, sets of five silhouettes can be combined into a single silhouette set in which the pose of each silhouette is known in a common reference frame. This allows visual hulls of an arbitrary number of views to be computed if more than one image is used. The method is applied to an ornamental cat, and experimental results are shown.

Keith Forbes, Anthon Voigt and Ndimi Bodika. Using Silhouette Consistency Constraints to Build 3D Models. In Proceedings of the Fourteenth Annual Symposium of the Pattern Recognition Association of South Africa (PRASA 2003), November 2003.

Abstract: The visual hull is the largest object that is consistent with a set of silhouette views of an actual object. The visual hull can be built from a set of silhouettes in which the viewpoint corresponding to each silhouette is known. We show how several sets corresponding to the same rigid object, each containing a small number of silhouettes, can be merged into a single large set. In order to do this, the relative pose between the sets must be computed so that each viewpoint can be specified in a common reference frame. We show how the poses can be computed by enforcing silhouette consistency constraints between the sets. The single merged silhouette set can then be used to build a visual hull model that is a closer approximation to the actual object than visual hulls built from any of the original sets.

Keith Forbes, Anthon Voigt and Ndimi Bodika. An Inexpensive, Automatic and Accurate Camera Calibration Method. In Proceedings of the Thirteenth Annual Symposium of the Pattern Recognition Association of South Africa (PRASA 2002), November 2002.

Abstract: A calibration procedure for accurately determining the pose and internal parameters of several cameras is described. Multiple simultaneously-captured sets of images of a calibration object in different poses are used by the calibration procedure. Coded target patterns, which serve as control points, are distributed over the surface of the calibration object. The observed positions of these targets within the images can be automatically determined by means of code band patterns. The positions of the targets across the multiple images are then used to infer the camera parameters, as well as the 3D geometrical structure of the targets on the calibration object (thus avoiding the expense of a calibration object with accurately known 3D structure). Results for a three-camera system show RMS (root-mean-square) deviations of less than five microns of the inferred positions of 54 control points, distributed on the surface of a 50mm cube, from their expected positions on a flat surface. The RMS difference between the positions of 1423 observed control points and the positions predicted by a 330 parameter model of the camera system and calibration object was 0.09 pixels.

Keith Forbes. Volume Estimation of Fruit from Digital Profile Images. Master's thesis, University of Cape Town, March 2000.

Abstract: This dissertation investigates the feasibility of using the same digital profile images of fruit that are used in commercial packing houses for colour sorting and blemish detection purposes to estimate the volumes of the corresponding individual pieces of fruit. Data sets of actual fruit volumes and digital images of the fruit that simulate both single and multiple camera set-ups are obtained. Shape features are extracted from the digital profile images using various digital image processing techniques. These shape features are used in conjunction with the actual volumes of the fruit as targets, to train and test both linear and artificial neural network methods of volume estimation. The results, in terms of error and of the associated confidence of the volume estimation methods, are presented.

Keith Forbes and George Tattersfield. Estimating Fruit Volume from Digital Images. In Proceedings of the 5th IEEE Africon Conference in Africa, Electrotechnological Services for Africa, Cape Town, South Africa, October 1999.

Keith Forbes and George Tattersfield. Volumetric Determination of Apples Using Machine Vision Techniques. Elektron, Journal of the South African Institute of Electrical Engineers, 16(3):14-17, March 1999.

Keith Forbes and George Tattersfield. An investigation into the volumetric determination of apples using machine vision techniques. Proceedings of the Ninth Annual Symposium of the Pattern Recognition Association of South Africa (PRASA 1998), November 1998.

George Tattersfield and Keith Forbes. Classification of Tobacco Leaves by Colour and Plant Position. Proceedings of the Ninth Annual Symposium of the Pattern Recognition Association of South Africa (PRASA 1998), November 1998.

Keith Forbes. The Computer Processing of Digital Images of Tobacco Leaves to Classify Their Plant Position by Shape Analysis. Undergraduate thesis, University of Cape Town, 1997.

Comments? Queries?