CASIA OpenIR  > 多模态人工智能系统全国重点实验室
HTCViT: an effective network for image classification and segmentation based on natural disaster datasets
Ma, Zhihao1,3; Li, Wei1,3; Zhang, Muyang1,3; Meng, Weiliang1,2,3; Xu, Shibiao4; Zhang, Xiaopeng1,2,3
Source PublicationVISUAL COMPUTER
ISSN0178-2789
2023-07-04
Pages13
Abstract

Classifying and segmenting natural disaster images are crucial for predicting and responding to disasters. However, current convolutional networks perform poorly in processing natural disaster images, and there are few proprietary networks for this task. To address the varying scales of the region of interest (ROI) in these images, we propose the Hierarchical TSAM-CB-ViT (HTCViT) network, which builds on the ViT network's attention mechanism to better process natural disaster images. Considering that ViT excels at extracting global context but struggles with local features, our method combines the strengths of ViT and convolution, and can capture overall contextual information within each patch using the Triple-Strip Attention Mechanism (TSAM) structure. Experiments validate that our HTCViT can improve the classification task with 3 - 4% and the segmentation task with 1 - 2% on natural disaster datasets compared to the vanilla ViT network.

KeywordNatural disaster image analysis Vision transformer Convolution Hierarchical
DOI10.1007/s00371-023-02954-3
Indexed BySCI
Language英语
Funding ProjectNational Natural Science Foundation of China[U21A20515] ; National Natural Science Foundation of China[61972459] ; National Natural Science Foundation of China[62172416] ; National Natural Science Foundation of China[62102414] ; National Natural Science Foundation of China[U2003109] ; National Natural Science Foundation of China[62071157] ; National Natural Science Foundation of China[62171321] ; National Natural Science Foundation of China[62162044] ; Open Research Projects of ZhejiangLab[2021KE0AB07] ; [TC210H00L/42]
Funding OrganizationNational Natural Science Foundation of China ; Open Research Projects of ZhejiangLab
WOS Research AreaComputer Science
WOS SubjectComputer Science, Software Engineering
WOS IDWOS:001025049000002
PublisherSPRINGER
Sub direction classification图像视频处理与分析
planning direction of the national heavy laboratory环境多维感知
Paper associated data
Citation statistics
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/53649
Collection多模态人工智能系统全国重点实验室
Corresponding AuthorMeng, Weiliang; Xu, Shibiao
Affiliation1.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China
2.Zhejiang Lab, Hangzhou, Peoples R China
3.Chinese Acad Sci, Inst Automat, State Key Lab Multimodal Artificial Intelligence S, Beijing, Peoples R China
4.Beijing Univ Posts & Telecommun, Sch Artificial Intelligence, Beijing, Peoples R China
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Ma, Zhihao,Li, Wei,Zhang, Muyang,et al. HTCViT: an effective network for image classification and segmentation based on natural disaster datasets[J]. VISUAL COMPUTER,2023:13.
APA Ma, Zhihao,Li, Wei,Zhang, Muyang,Meng, Weiliang,Xu, Shibiao,&Zhang, Xiaopeng.(2023).HTCViT: an effective network for image classification and segmentation based on natural disaster datasets.VISUAL COMPUTER,13.
MLA Ma, Zhihao,et al."HTCViT: an effective network for image classification and segmentation based on natural disaster datasets".VISUAL COMPUTER (2023):13.
Files in This Item: Download All
File Name/Size DocType Version Access License
s00371-023-02954-3.p(8610KB)期刊论文出版稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Ma, Zhihao]'s Articles
[Li, Wei]'s Articles
[Zhang, Muyang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Ma, Zhihao]'s Articles
[Li, Wei]'s Articles
[Zhang, Muyang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Ma, Zhihao]'s Articles
[Li, Wei]'s Articles
[Zhang, Muyang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: s00371-023-02954-3.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.