基于交互融合的高精度图像融合算法的研究 |
| |
作者姓名: | 王吉哲 李勃 马晨瑛 殷奇缘 周鹏 |
| |
作者单位: | 1.南京航空航天大学电子信息工程学院 南京 210016;2.南京航空航天大学无人机研究院 南京 210016 |
| |
基金项目: | 中国高校产学研创新基金(2021ZYA04004) |
| |
摘 要: | 针对可见光和SAR图像融合,提出了一种基于跨模态差分感知和注意力机制的交互融合(TDPAM Fusion)算法,能有效保留可见光图像中的纹理结构和SAR图像的细节信息。首先,采用跨模态差分感知融合(Cross-Modal Differential Perception Fusion,CMDAF)模块提取图像的互补信息,避免真值缺失并提高融合精度。其次,通过坐标注意力机制(Coordinate Attention,CA)提高特征提取的准确性和效率,增强语义信息的集成。最后通过交互融合算法(Interactive Fusion Module,IFM)将特征自适应融合。设计了相应的大型基准数据集,用于网络模型的训练和测试。实验结果表明:TDPAM Fusion融合算法可以获得包含清晰SAR信息的高质量可见光图像。此外,融合算法将互信息(Mutual Information,MI)、空间频率(Spatial Frequency,SF)、视觉保真度(Visual Fidelity,VIF)和相关系数(Correlation Coefficient,CC)等关键指标,分别提高了约6.41%、10.36%、14.25%和4.74%。
|
关 键 词: | 深度学习 图像融合 可见光图像 SAR图像 交互融合 |
收稿时间: | 2023/7/4 0:00:00 |
修稿时间: | 2023/8/16 0:00:00 |
Research on high-precision image fusion algorithm based on interactive fusion |
| |
Authors: | WANG Jizhe LI Bo MA Chenying YIN Qiyuan ZHOU Peng |
| |
Institution: | 1.College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China;2.Research Institute of Aerial Vehicles, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China |
| |
Abstract: | Aiming at the fusion of visible and SAR images, a novel interactive fusion algorithm based on Transmembrane Differential Perception and Attention Mechanism (TDPAM Fusion) is proposed, which can effectively preserve the texture structure of visible images and detail information of SAR images. Firstly, the Cross-Modal Differential Perception Fusion module is utilized to extract complementary information from images, which can avoid the missing of true values and improve the accuracy of fusion. Secondly, the coordinate attention mechanism is employed to enhance the accuracy and efficiency of feature extraction, and improve the integration of semantic information. Finally, a feature interaction fusion algorithm is used to adaptively fuse features from SAR and visible images. A corresponding large benchmark dataset is designed for model training and testing. Experimental results demonstrate that the fusion algorithm can obtain high-quality visible images with clear SAR information. In addition, the algorithm can improve key indicators such as mutual information, spatial frequency, visual fidelity, and correlation coefficient by approximately 6.41%, 10.36%, 14.25%, and 4.74%, respectively. |
| |
Keywords: | Deep learning Image fusion Visible image SAR image Interactive fusion |
|
| 点击此处可从《》浏览原始摘要信息 |
| 点击此处可从《》下载免费的PDF全文 |
|