中国科学院分子影像重点实验室知识库

超声影像是当前医学临床实践中应用极其广泛的成像类型，在多种脏器疾病的检查、诊断、预后治疗等方面都有长期的实践。相比于计算机断层扫描等影像类型而言，超声成像的过程对人体更加安全，没有有害辐射，且价格较低、成像实时。然而，超声影像中容易存在斑点噪声、成像伪影等缺陷，降低了超声的成像质量和图像对比度。另一方面，由于超声影像分辨率有限等导致的小尺寸目标成像限制了对其进行分析的精确性。超声视频影像中的运动形变和时空特征信息异质性也要求更有针对性的分析理解方式。这些因素阻碍了超声影像在临床实践上更广泛的应用，同时对基于超声影像的计算机定量分析研究的准确性、稳定性和可复现性提出了更高的要求。

上述挑战性问题对目前的超声影像定量分析算法的建模带来了挑战，凸显了在机器学习算法建模过程中，为降低成像质量干扰而设计针对性解决方案的重要性。为此本文以深度学习的网络特征增强为出发点，在多种超声影像基础算法分析和临床应用研究上对不同类型的影像干扰问题以及超声视频的时空特征异质性设计合理的网络结构，提高算法对超声影像的建模学习能力。本文的主要工作和创新点归纳如下：

1. 针对超声影像中低对比度、低信噪比造成深度学习网络特征的判别力不足，导致建模准确性下降的问题，本文在超声心动图的语义分割任务中提出了局部注意金字塔网络结构和区域语义一致性学习机制。通过在注意机制中引入注意范围局部性和注意点稀疏性的原则，算法特征学习过程中可以避免累积大量超声影像噪声干扰，同时在具有强相关性的局部上下文邻域中增强影像特征的语义判别能力。区域语义一致性使得全卷积分割网络实现了不同影像区域分割预测的二阶联合学习，消除了分割过程产生的孤立点、空洞等现象。模型分割结果的Dice分数超过0.951，基于分割结果评估的心脏射血分数与真实值之间的相关系数达到0.882。

2. 针对超声视频影像中分析目标过小导致的影像特征信息微弱的问题，以及视频中因组织运动等带来的分析目标出现形变的干扰，本文在超声视频的单目标跟踪任务中提出了级联孪生网络结构和one-shot可变形卷积模块。通过将深度学习目标定位过程转换为由粗略到精细的两步级联方式，大幅提升了现有超声跟踪精度。One-shot 可变形卷积模块实现了算法自适应地调整网络感受野范围，更多地以判别性能强的图像区域计算影像特征，降低影像模糊和形变对跟踪稳定性的影响。该跟踪算法在实验数据集上的跟踪误差低于0.69±0.67 mm。

3. 针对超声造影视频影像中时空特征存在信息异质性，难以在端到端的深度学习算法中分析建模的问题，本文在基于超声造影的生存分析预测任务中，提出了基于 2D 卷积神经网络和局部聚合描述子向量的深度学习生存分析算法。通过将造影视频的空间信息和时间信息进行独立串联分析，并对时间维度特征进行分期聚类建模，提高了深度学习网络对超声造影时空特征的学习能力。进一步地，本文将深度学习算法与Cox风险比例回归函数结合构建端到端的深度学习生存分析预测算法(CNN-Cox)，在肝细胞癌的两种常见治疗方式射频消融术和手术切除上进行预后无进展生存期预测分析，并基于分析结果构建肝细胞癌患者的个体化治疗方式辅助决策，以提高患者的术后2年预期无进展生存概率。实验证明该算法在为射频消融术和手术切除评估预后无进展生存期的C-index值超过0.72，所提出的辅助决策方法可提高患者的术后2年预期无进展生存概率12%以上。

本研究围绕超声影像的低信噪比与对比度、目标过小与运动形变，以及时空特征信息的有效学习，开展基于深度学习的特征增强算法研究，针对不同的超声影像质量问题提出了特定的网络设计模型，并在超声心动图的语义分割任务、超声视频的单目标跟踪任务，以及在超声造影上的生存分析任务上进行了有效的验证。基于已有的研究成果，未来工作将在深度学习算法与医学先验的结合、多任务联合学习、跟踪算法的多目标场景扩展和与视频特征的结合，以及对已有模型的前瞻性检验等方向作进一步的探索。

Ultrasonography is a widely used imaging modality in current medical clinical practice. It has long-term practice in the screening, diagnosis, and prognostic treatment of various organ diseases. Compared with other modalities such as computed tomography,  the ultrasonography has advantages on the radiation-free characteristic, economical affordability, and real-time imaging. However, ultrasonic images are prone to defects such as speckle noise and imaging artifacts, which reduce the visual quality and image contrast. One the other hand, the accurate analysis for the small-size targets is limited by the lower resolution of ultrasound images. The movement deformation and the heterogeneity of spatial and temporal feature characteristics in ultrasound imaging also requires more specific image analysis methods. These factors hinder the wider application of ultrasound imaging in clinical practice, and put forward higher requirements for the accuracy, stability and reproducibility of computer quantitative analysis based on ultrasound imaging.

The above-mentioned problems have brought challenges to the current ultrasound quantitative analysis, highlighting the importance of designing targeted solutions to reduce the hindrance from low imaging quality during the modeling process of machine learning. Therefore, making network feature enhancement of deep learning in mind, we designed reasonable algorithms for different kind of imaging quality problems and the heterogeneity of the spatio-temporal feature characteristics on various ultrasound image analysis tasks, and improved the models’ learning ability. The main work and innovations of this thesis are summarized as follows:

1. For the challenges that low contrast and signal-to-noise ratio in ultrasound images cause insufficient discriminability of deep learning network features and lower modeling accuracy, we proposes a local attention pyramid network structure and label coherence learning mechanism in the task of semantic segmentation on echocardiography. By introducing the locality of attention range and sparsity of attention sampling into the attention mechanism, the harmful ultrasound image noise can be avoided in the process of feature learning, and the semantic discriminability of image features can be enhanced within the strongly correlated local context neighborhood. The label coherence learning mechanism achieves the second-order joint learning for fully convolutional segmentation network, and eliminates the segmentation errors such as isolated points and holes. The dice score of segmentation results was 0.951 and the correlation between estimated and true ejection fraction was 0.882.

2. For the challenge of weak image feature signals caused by the small targets in the ultrasound videos, and the problems of the target deformation caused by the movement in the video, we proposes cascaded siamese network and one-shot deformable convolutoin for the task of tracking on single-target. By converting the target positioning process into a two-step cascade method from coarse to fine, the ultrasonic tracking accuracy is significantly improved. The one-shot deformable convolution module allows the model to adaptively adjust receptive field, and extracts imaging features from regions with strong discriminative characteristics, reducing the influence of blurry areas and target deformation on tracking robustness. The tracking error of this algorithm on the experiment dataset was 0.69±0.67 mm.

3. Due to the heterogeneity of spatio-temporal features in contrast-enhanced ultrasound videos (CEUS), it is difficult to achieve survival analysis using CEUS in the end-to-end deep learning algorithm. In this thesis, we propose a survival analysis algorithm (CNN-Cox) based on 2D convolutonal neural network and vector of locally aggregated descriptors. With the cascaded analysis of the spatial-temporal information of the CEUS video, and the phase-per-clustering of features on temporal dimension, the deep learning network's ability to learn the temporal and spatial features of CEUS is enhanced. Furthermore, we construct an end-to-end deep learning survival analysis algorithm incorpating neural network and the Cox proportional hazard regression loss. We analyze the progression-free survival status for radiofrequency ablation (RFA) and surgical resection (SR), two kind of treatment methods for hepatocellular carcinoma, and optimize the treatment selection between RFA and SR, improving the expected 2-year progression-free survival probability for patients. It was proved that the C-index of RFA and SR by CNN-Cox was higher than 0.72, and optimization of treatment selection improved the expected 2-year progression-free survival probability by more than 12%.

Our research focused on the challenge of the low signal-to-noise ratio and contrast of ultrasound images, the small-scale research target and motion deformation, and the effective learning of spatio-temporal feature information. Specific network design strategies were proposed for different research problems, and proved on the task of semantic segmentation, single-target tracking, and survival analysis based on CEUS. Based on these researches, multiple researches will be carried out in the future work, such as the combination between deep learning and medical prior knowledge, multi-task learning, combination of multi-target tracking and video features, and the prospective validation of the existing models.