This paper describes an application framework to perform high quality upsampling and completion on noisy depth maps. Our framework targets a complementary system setup which consists of a depth camera coupled with an RGB camera. Inspired by a recent work that uses a nonlocal structure regularization, we regularize depth maps in order to maintain fine details and structures. We extend this regularization by combining the additional high-resolution RGB input when upsampling a low-resolution depth map together with a weighting scheme that favors structure details. Our technique is also able to repair large holes in a depth map with consideration of structures and discontinuities by utilizing edge information from the RGB input. Quantitative and qualitative results show that our method outperforms existing approaches for depth map upsampling and completion. We describe the complete process for this system, including device calibration, scene warping for input alignment, and even how our framework can be extended for video depth-map completion with consideration of temporal coherence.

- acknowledgement
This research was supported by the National Strategic R&D Program for Industrial Technology, Korea (No. 10031903) and the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIP) (No. 2010-0028680). Michael S. Brown was supported by A*STAR Science and Engineering Research Council, Public Sector Research Funding Grant (No. 1121202020).