基于深度神经网络的像素级别可见光图像配准

现有图像配准算法中，借助图像采集设备参数的方法存在硬件内参难以获得或精度不够的问题，采用匹配图像特征计算图像单应性的方法存在对场景深度信息利用不全的问题。针对这一现象，提出了结合可见光图像与其深度信息来生成更具有真实性的配准图像对数据，用以训练得到一个可以进行像素级别图像配准的深度神经网络PIR-Net。建立了一个大规模、多视角、超仿真的图像配准数据集:多视角配准(MVR)数据集，该数据集包含7 240对含有深度信息的待配准图像及其像素级别的坐标对准真值；基于编码器-解码器的深度神经网络结构，训练得到一个能以全分辨率形式对2幅输入图像之间的坐标变化矩阵进行重建的PIR-Net。通过实验验证了PIR-Net能够在未知相机内参的情况下实现不同视角的可见光图像配准，并比传统算法具有更高的配准精度。在MVR数据集上，PIR-Net的配准误差仅为通用的特征匹配对准算法(SIFT+RANSAC)的18%，同时减少了30%的时间消耗。 

Pixel-wise visible image registration based on deep neural network

Current image registration algorithms relying on the internal parameters of sensing devices for image alignment face the difficulty of acquiring precise device parameters and reaching high mapping precision; while the ones using matched image features to calculate image homography matric for registration have the problem of insufficient utilization of scene depth information. Based on this observation, we propose a method which can generate more authentic image registration data from monocular images and their depth-maps, and use the data to train a pixel-wise image registration network, the PIR-Net, for fast, accurate and practical image registration. We construct a large-scale, multi-view, realistic image registration database with pixel-wise depth information that imitates real-world situations, the multi-view image registration (MVR) dataset. The MVR dataset contains 7 240 pairs of RGB images and their corresponding registraton labels. With the dataset, we train an encoder-decoder structure based, fully convolutional image registration network, the PIR-Net, extensive experiments on the MVR dataset demonstrate that the PIR-Net can predict pixel-wise image alignment matrix for multi-view RGB images without accessing the camera internal parameters, and that the PIR-Net out-performs traditional image registration methods. On the MVR dataset, the registration error of PIR-Net is only 18% of the general feature matching method (SIFT+RANSAC), and its time cost is 30% less.