Patch Based Synthesis for Single Depth Image Super-Resolution

Results


The results below are shown with buttons to allow easy comparison of our proposed technique vs. other approaches. Different preprocessing was used depending on the sensor that captured the low-resolution input. [Q. Yang et al.] use an intensity image as additional input while ours, [SCSR] and [Freeman and Liu] use only the low-resolution depth image. [SCSR] and [Freeman and Liu], both designed initially for intensity images, made their implementations available online, and we applied these directly, using default settings. For a subset of the scenes we also display the rendered 3D point cloud as in our supplementary video. None of these results display our post processing denoising step.

To produce the quantitative comparison in Fig 5 in the paper we need to resize the super-resolved outputs so they have the same width and height as the ground truth. Down sampling by a constant factor and upsampling again can result in images that are a different size from the original. In the paper we resolved this using MATLAB's built in imresize function and specified the output to be the same size as the ground truth. One alternative is to crop the super-resolved image so it's the same size as the ground truth. Cropped results for the Middlebury and laser scenes are presented here.

References
[Our Result] O. Mac Aodha, N. Campbell, A. Nair and G.J. Brostow. Patch Based Synthesis for Single Depth Image Super-Resolution. In ECCV 2012.

[Q. Yang et al.] Q. Yang, R. Yang, J. Davis, and D. Nister. Spatial-depth super resolution for range images. In CVPR, 2007.

[SCSR] J. Yang, J. Wright, T. S. Huang, and Y. Ma. Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 2010.

[Freeman and Liu] W. T. Freeman and C. Liu. Markov random fields for super-resolution and texture synthesis. In A. Blake, P. Kohli, and C. Rother, editors, Advances in Markov Random Fields for Vision and Image Processing, chapter 10. MIT Press, 2011.

[Brown Range Database] J. Huang, A. Lee and D. Mumford. Statistics of range images. In CVPR, 2000.

To switch between images please use the buttons on the right.
The methods that are available for comparison are using the following color code:
Our Result
Freeman and Liu
Nearest Neighbour
Bicubic
SCSR
Q. Yang et al.
Bilateral and NN
Input 3D
Our Result - Brown: Using the Brown Range Database instead of synthetic data
Upsampling First: Patching is performed at high resolution, like in image approaches
Ground Truth
Please note that the images are initialized to Our Result: Patch Based Synthesis for Single Depth Image Super-Resolution

Results

Scene 01

Output Image

Output
Magnification x4
Input Image

Input image

 













Scene 02

Output Image

Output
Magnification x4
Input Image

Input image

 










Scene 03

Output Image

Output
Magnification x2
Input Image

Input image

 











Scene 04

Output Image

Output
Magnification x4
Input Image

Input image

 










Scene 05

Output Image

Output
Magnification x4
Input Image

Input image

 








Scene 06

Output Image

Output
Magnification x10
Input Image

Input image

 










Scene 07

Output Image

Output
Magnification x10
Input Image

Input image

 










Scene 08

Output Image

Output
Magnification x4
Input Image

Input image

 









Scene 09

Output Image

Output
Magnification x4
Input Image

Input image

 










Scene 10

Output Image

Output
Magnification x4
Input Image

Input image

 












Scene 11

Output Image

Output
Magnification x4
Input Image

Input image

 












Scene 12

Output Image

Output
Magnification x4
Input Image

Input image

 













Scene 13

Output Image

Output
Magnification x4
Input Image

Input image

 











Scene 14

Output Image

Output
Magnification x4
Input Image

Input image

 











Scene ID Sensor Type Preprocessing Credits
1 PMD CamCube 2 Bilateral Filter [5,3,0.1]
2 Swiss Ranger 3000 ToF Bilateral Filter [5,1.5,0.01] [23] Schuon et al. - Lidarboost
3 Structured Light [21] Scharstein et al. - Middlebury Stereo Dataset
4 PMD CamCube 2 Bilateral Filter [5,1.5,0.01]
5 Kinect Gaussian Blur [3 0.5]
6 Canesta EP DevKit Bilateral Filter [5,3,0.1] [31] Q. Yang et al.
7 Canesta EP DevKit Bilateral Filter [5,3,0.1] [31] Q. Yang et al.
8 PMD CamCube 2 Bilateral Filter [5,1.5,0.01]
9 PMD CamCube 2 Bilateral Filter [5,1.5,0.01]
10 PMD CamCube 2 Bilateral Filter [5,3,0.1]
11 PMD CamCube 2 Bilateral Filter [5,3,0.1]
12 Laser Scanner
13 Structured Light [21] Scharstein et al. - Middlebury Stereo Dataset
14 Structured Light [21] Scharstein et al. - Middlebury Stereo Dataset