Multi-Evidence and Multi-Modal Fusion Network for Ground-Based Cloud Recognition

doi:10.3390/rs12030464

CASIA OpenIR > 复杂系统管理与控制国家重点实验室 > 影像分析与机器视觉

	Multi-Evidence and Multi-Modal Fusion Network for Ground-Based Cloud Recognition
	Liu, Shuang 1; Li, Mei 1; Zhang, Zhong 1; Xiao, Baihua2 ; Durrani, Tariq S.3
发表期刊	REMOTE SENSING
	2020-02-01
卷号	12 期号:3 页码:20
通讯作者	Zhang, Zhong(zhangz@tjnu.edu.cn)
摘要	In recent times, deep neural networks have drawn much attention in ground-based cloud recognition. Yet such kind of approaches simply center upon learning global features from visual information, which causes incomplete representations for ground-based clouds. In this paper, we propose a novel method named multi-evidence and multi-modal fusion network (MMFN) for ground-based cloud recognition, which could learn extended cloud information by fusing heterogeneous features in a unified framework. Namely, MMFN exploits multiple pieces of evidence, i.e., global and local visual features, from ground-based cloud images using the main network and the attentive network. In the attentive network, local visual features are extracted from attentive maps which are obtained by refining salient patterns from convolutional activation maps. Meanwhile, the multi-modal network in MMFN learns multi-modal features for ground-based cloud. To fully fuse the multi-modal and multi-evidence visual features, we design two fusion layers in MMFN to incorporate multi-modal features with global and local visual features, respectively. Furthermore, we release the first multi-modal ground-based cloud dataset named MGCD which not only contains the ground-based cloud images but also contains the multi-modal information corresponding to each cloud image. The MMFN is evaluated on MGCD and achieves a classification accuracy of 88.63% comparative to the state-of-the-art methods, which validates its effectiveness for ground-based cloud recognition.
关键词	ground-based cloud recognition convolution neural network feature fusion
DOI	10.3390/rs12030464
关键词[WOS]	LOCAL BINARY PATTERN ; FEATURE-EXTRACTION ; CLASSIFICATION ; FEATURES ; IMAGES ; SCALE
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61711530240] ; Natural Science Foundation of Tianjin[19JCZDJC31500] ; Fund of Tianjin Normal University[135202RC1703] ; Open Projects Program of National Laboratory of Pattern Recognition[202000002] ; Tianjin Higher Education Creative Team Funds Program ; National Natural Science Foundation of China[61711530240] ; Natural Science Foundation of Tianjin[19JCZDJC31500] ; Fund of Tianjin Normal University[135202RC1703] ; Open Projects Program of National Laboratory of Pattern Recognition[202000002] ; Tianjin Higher Education Creative Team Funds Program
项目资助者	National Natural Science Foundation of China ; Natural Science Foundation of Tianjin ; Fund of Tianjin Normal University ; Open Projects Program of National Laboratory of Pattern Recognition ; Tianjin Higher Education Creative Team Funds Program
WOS研究方向	Remote Sensing
WOS类目	Remote Sensing
WOS记录号	WOS:000515393800123
出版者	MDPI
引用统计	被引频次：20[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/38329
专题	复杂系统管理与控制国家重点实验室_影像分析与机器视觉
通讯作者	Zhang, Zhong
作者单位	1.Tianjin Normal Univ, Tianjin Key Lab Wireless Mobile Commun & Power Tr, Tianjin 300387, Peoples R China 2.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 3.Univ Strathclyde, Dept Elect & Elect Engn, Glasgow G1 1XQ, Lanark, Scotland
推荐引用方式 GB/T 7714	Liu, Shuang,Li, Mei,Zhang, Zhong,et al. Multi-Evidence and Multi-Modal Fusion Network for Ground-Based Cloud Recognition[J]. REMOTE SENSING,2020,12(3):20.
APA	Liu, Shuang,Li, Mei,Zhang, Zhong,Xiao, Baihua,&Durrani, Tariq S..(2020).Multi-Evidence and Multi-Modal Fusion Network for Ground-Based Cloud Recognition.REMOTE SENSING,12(3),20.
MLA	Liu, Shuang,et al."Multi-Evidence and Multi-Modal Fusion Network for Ground-Based Cloud Recognition".REMOTE SENSING 12.3(2020):20.