Fusion of Depth Maps with Multiple Scales

This website provides material for the depth map fusion algorithm described in the following paper:

Fusion of Depth Maps with Multiple Scales [Paper, 9MB] [YouTube Video] [MP4 Video, 64MB]
Simon Fuhrmann and Michael Goesele
In: ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH Asia 2011), Hong Kong, China, 2011.

If you want to get in contact, write to simon.fuhrmann@gris.informatik.tu- (replace tu- with tu-darmstadt.de)

Teaser Image
Reconstruction pipeline. Input photographs (top left) depicting objects at different levels of detail. Multi-view stereo yields depth maps (bottom left), which inherit these multi-scale properties. Our system is able to fuse such depth maps and produce an adaptive mesh (right) with coarse regions as well as fine scale details (insets).

Paper Abstract

Multi-view stereo systems can produce depth maps with large variations in viewing parameters, yielding vastly different sampling rates of the observed surface. We present a new method for surface reconstruction by integrating a set of registered depth maps with drastically varying sampling rate. The method is based on the construction of a hierarchical signed distance field represented in an incomplete primal octree by incrementally adding triangulated depth maps. Due to the adaptive data structure, our algorithm is able to handle depth maps with varying scale and to consistently represent coarse, low-resolution regions as well as small details contained in high-resolution depth maps. A final surface mesh is extracted from the distance field by construction of a tetrahedral complex from the scattered signed distance values and applying the Marching Tetrahedra algorithm on the partition. The output is an adaptive triangle mesh that seamlessly connects coarse and highly detailed regions while avoiding filling areas without suitable input data.

Software

The following software implements the technique described in the paper. The software is experimental, not well tested and very CPU and memory hungry. Make sure you have sufficient memory and change parameters of the tool appropriately if you run out of memory: For example use a smaller sampling rate (-r parameter), use smaller depth maps as input or stock up your RAM. Note that this software depends on MVE, the Multi-View Environment.

In order to compile the software, first build MVE, copy the dmfusion directory into the apps/ directory and compile it using make.

Running the software involves first building an octree using the dmfoctree app, extracting a tetrahedral mesh from the octree using dmfextract and triangulating the tetrahedral mesh using dmfsurface.

Download the code here: dmfusion [313 KB] (last update 2014-05-05).

Here are a few shell commands that should you get started.

git clone https://github.com/simonfuhrmann/mve.git
wget http://www.gris.informatik.tu-darmstadt.de/projects/multiscale-depthmap-fusion/data/dmfusion-20131122.tar.gz
tar xf dmfusion-20131122.tar.gz
make -C mve
MVE_ROOT=`pwd`/mve make -C dmfusion