自适应融合 RGB图像特征的稀疏深度修复

深度修复的目的是从稀疏深度图像中恢复出稠密的深度图像。现有方法通常是以稀疏深度图像及其对应的 RGB图像为输入,通过 1个卷积神经网络恢复出密集深度图像。然而,普通的卷积层在处理稀疏且不规则的深度信息时有较大的局限性,同时,RGB图像特征和深度图像特征属于不同的模态。针对这些问题,文章提出了自适应稀疏不变模块,根据输入像素的有效性来处理稀疏深度,并提出了结合注意力机制的多尺度特征融合模块,在关注有效特征的同时,抑制不必要的特征,进一步提高深度修复性能。文章在 NYUv2数据集上进行了一系列实验,实验结果表明了所提出算法和模块的有效性。

Adaptive Fusion of RGB Image Features for Sparse Depth Completion

The purpose of depth completion is to restore dense depth images from sparse depth images. Existing methods usually take sparse depth images and their corresponding RGB images as input and restore dense depth images through a convolutional neural network. However, ordinary convolutional layers have large limitations in dealing with sparse and ir-regular depth information, while RGB image features and depth image features belong to different modalities. To address these problems, an adaptive sparse invariant module to handle sparse depths according to the validity of the input pixels is proposed. The multi-scale features fusion incorporating attention mechanism is also proposed to further improve the depth completion performance by suppressing unnecessary features while focusing on effective features. A series of experiments are conducted on the NYUv2 dataset, and the experimental results demonstrate the effectiveness of the proposed algo-rithm and module.