Institutional Repository of Chinese Acad Sci, Inst Automat, CAS Key Lab Mol Imaging, Beijing 100190, Peoples R China
MSMFN: An ultrasound based multi-step modality fusion network for identifying the histologic subtypes of metastatic cervical lymphadenopathy | |
Zheling, Meng1,2![]() ![]() | |
发表期刊 | IEEE Transactions on Medical Imaging
![]() |
2022-11 | |
页码 | 1-13 |
文章类型 | SCI |
摘要 | Identifying squamous cell carcinoma and adenocarcinoma subtypes of metastatic cervical lymphadenopathy (CLA) is critical for localizing the primary lesion and initiating timely therapy. B-mode ultrasound (BUS), color Doppler flow imaging (CDFI), ultrasound elastography (UE) and dynamic contrast-enhanced ultrasound provide effective tools for identification but synthesis of modality information is a challenge for clinicians. Therefore, based on deep learning, rationally fusing these modalities with clinical information to personalize the classification of metastatic CLA requires new explorations. In this paper, we propose Multi-step Modality Fusion Network (MSMFN) for multi-modal ultrasound fusion to identify histological subtypes of metastatic CLA. MSMFN can mine the unique features of each modality and fuse them in a hierarchical three-step process. Specifically, first, under the guidance of high-level BUS semantic feature maps, information in CDFI and UE is extracted by modality interaction, and the static imaging feature vector is obtained. Then, a self-supervised feature orthogonalization loss is introduced to help learn modality heterogeneity features while maintaining maximal task-consistent category distinguishability ofmodalities. Finally, six encoded clinical information are utilized to avoid prediction bias and improve prediction ability further. Our three-fold cross-validation experiments demonstrate that our method surpasses clinicians and other multi-modal fusion methods with an accuracy of 80.06%, a true-positive rate of 81.81%, and a true-negative rate of 80.00%. Our network provides a multi-modal ultrasound fusion framework that considers prior clinical knowledge and modality-specific characteristics. Our code will be available at: https://github.com/RichardSunnyMeng/MSMFN. |
收录类别 | SCI |
语种 | 英语 |
是否为代表性论文 | 是 |
七大方向——子方向分类 | 人工智能+医疗 |
国重实验室规划方向分类 | 视觉信息处理 |
是否有论文关联数据集需要存交 | 否 |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/51470 |
专题 | 中国科学院分子影像重点实验室 |
通讯作者 | Fang, Nie; Kun, Wang |
作者单位 | 1.中国科学院自动化研究所 2.中国科学院大学人工智能学院 3.兰州大学第二医院 |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Zheling, Meng,Yangyang, Zhu,Wenjing, Pang,et al. MSMFN: An ultrasound based multi-step modality fusion network for identifying the histologic subtypes of metastatic cervical lymphadenopathy[J]. IEEE Transactions on Medical Imaging,2022:1-13. |
APA | Zheling, Meng,Yangyang, Zhu,Wenjing, Pang,Jie, Tian,Fang, Nie,&Kun, Wang.(2022).MSMFN: An ultrasound based multi-step modality fusion network for identifying the histologic subtypes of metastatic cervical lymphadenopathy.IEEE Transactions on Medical Imaging,1-13. |
MLA | Zheling, Meng,et al."MSMFN: An ultrasound based multi-step modality fusion network for identifying the histologic subtypes of metastatic cervical lymphadenopathy".IEEE Transactions on Medical Imaging (2022):1-13. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
FINAL VERSION.pdf(3049KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论