Knowledge Commons of Institute of Automation,CAS
TR-MISR: Multiimage super-resolution based on feature fusion with transformers | |
An T(安泰)1,2![]() ![]() ![]() ![]() ![]() ![]() | |
发表期刊 | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
![]() |
2022-01 | |
卷号 | 15页码:1373-1388 |
产权排序 | 1 |
摘要 | Multiimage super-resolution (MISR), as one of the most promising directions in remote sensing, has become a needy technique in the satellite market. A sequence of images collected by satellites often has plenty of views and a long time span, so integrating multiple low-resolution views into a high-resolution image with details emerges as a challenging problem. However, most MISR methods based on deep learning cannot make full use of multiple images. Their fusion modules are incapable of adapting to an image sequence with weak temporal correlations well. To cope with these problems, we propose a novel end-to-end framework called TR-MISR. It consists of three parts: An encoder based on residual blocks, a transformer-based fusion module, and a decoder based on subpixel convolution. Specifically, by rearranging multiple feature maps into vectors, the fusion module can assign dynamic attention to the same area of different satellite images simultaneously. In addition, TR-MISR adopts an additional learnable embedding vector that fuses these vectors to restore the details to the greatest extent.TR-MISR has successfully applied the transformer to MISR tasks for the first time, notably reducing the difficulty of training the transformer by ignoring the spatial relations of image patches. Extensive experiments performed on the PROBA-V Kelvin dataset demonstrate the superiority of the proposed model that provides an effective method for transformers in other low-level vision tasks. |
关键词 | Deep learning end-to-end networks feature extraction and fusion multiimage super-resolution (MISR) remote sensing transformers |
收录类别 | SCI |
语种 | 英语 |
是否为代表性论文 | 是 |
七大方向——子方向分类 | 图像视频处理与分析 |
国重实验室规划方向分类 | 多尺度信息处理 |
是否有论文关联数据集需要存交 | 否 |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/54532 |
专题 | 多模态人工智能系统全国重点实验室_先进时空数据分析与学习 |
通讯作者 | Huo CL(霍春雷) |
作者单位 | 1.National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences 2.School of Artificial Intelligence, University of Chinese Academy of Sciences |
第一作者单位 | 模式识别国家重点实验室 |
通讯作者单位 | 模式识别国家重点实验室 |
推荐引用方式 GB/T 7714 | An T,Zhang X,Huo CL,et al. TR-MISR: Multiimage super-resolution based on feature fusion with transformers[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2022,15:1373-1388. |
APA | An T,Zhang X,Huo CL,Xue B,Wang LF,&Pan CH.(2022).TR-MISR: Multiimage super-resolution based on feature fusion with transformers.IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,15,1373-1388. |
MLA | An T,et al."TR-MISR: Multiimage super-resolution based on feature fusion with transformers".IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 15(2022):1373-1388. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
TR-MISR.pdf(6058KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论