Depth Super Resolution by Rigid Body Self-Similarity in 3D_CVPR2013.pdf

Depth Super Resolution by Rigid Body Self-Similarity in 3D Michael Horna?c?ek1,*, Christoph Rhemann2, Margrit Gelautz1, and Carsten Rother2 1Vienna University of Technology 2Microsoft Research Cambridge Abstract We tackle the problem of jointly increasing the spatial resolution and apparent measurement accuracy of an input low-resolution, noisy, and perhaps heavily quantized depth map. In stark contrast to earlier work, we make no use of ancillary data like a color image at the target resolu- tion, multiple aligned depth maps, or a database of high- resolution depth exemplars. Instead, we proceed by identi- fying and merging patch correspondences within the input depth map itself, exploiting patchwise scene self-similarity across depth such as repetition of geometric primitives or object symmetry. While the notion of ‘single-image’ super resolution has successfully been applied in the context of color and intensity images, we are to our knowledge the first to present a tailored analogue for depth images. Rather than reason in terms of patches of 2D pixels as others have before us, our key contribution is to proceed by reason- ing in terms of patches of 3D points, with matched patch pairs related by a respective 6 DoF rigid body motion in 3D. In support of obtaining a dense correspondence field in reasonable time, we introduce a new 3D variant of Patch- Match. A third contribution is a simple, yet effective patch upscaling and merging technique, which predicts sharp ob- ject boundaries at the target resolution. We show that our results are highly competitive with those of alternative tech- niques leveraging even a color image at the target resolu- tion or a database of high-resolution depth exemplars. 1. Introduction With the advent of inexpensive 3D cameras like the Microsoft Kinect, depth measurements are becoming in- creasingly available for low-cost applications. Acquisitions made by such consumer 3D cameras, however, remain af- flicted by less than ideal attribu