毕业生知识库

随着技术的不断进步，医学影像分析逐渐成为了临床疾病无创诊断的主要手段。医学声学成像是医学影像中的一大分支，它的典型特点是利用声学原理，以超声波作为原始信号提供成像目标的生理信息。光声断层成像（Photoacoustic Tomography，PAT）和超声成像是医学声学成像中的两大成像模态，二者可以分别反映成像组织的光学性质和声学性质，在结构、功能与分子成像上均具有广泛应用。

由于声学成像存在相同的信号域，在图像重建与分析算法上存在一定的共通性，也存在一些共同的挑战性问题。在图像重建方面，超声换能器作为声学成像的信号接收器，由于空间或成本的限制，在很多场景下无法实现足够密集的采样或全视角覆盖，这就导致了超声信号的损失，进而影响重建图像的质量。因此，研究有限视角稀疏采样情形下可靠的重建算法将会极大拓宽声学成像的应用场景。在图像分析方面，由于声学成像具有操作简单、无辐射和实时成像的特点，非常适合采集病灶的多时间点图像，从而监测病灶的变化。然而，手持式声学成像设备的图像质量很容易受到操作者的影响，导致多次采集的图像之间的一致性较低。因此，如何对多时间点低一致性图像进行有效分析也成为了声学成像的共有问题。

医学影像信息的复杂性使得越来越多的人工智能模型采用以孪生网络为代表的多分支网络结构。近年来，多分支网络已经广泛应用于医学图像重建与分析任务当中，并取得了非常可喜的成绩。本文以声学成像的两大共性问题——有限视角稀疏采样重建和多时间点低一致性图像分析作为研究问题切入点，以多分支网络作为方法切入点，以PAT图像重建与超声图像分析作为研究内容开展相关研究，具体包括：

（1）提出了基于多分支特征投影网络的PAT图像重建算法。该算法依托多分支网络架构将光声反投影重建的物理模型融入重建算法中，通过人工智能学习信号到图像域的变换过程，能够有效减少有限视角稀疏采样信号在重建过程中的信息损失，提升投影过程的精度。除了网络结构的创新，该研究还针对多分支特征投影网络设计了非线性信号预处理方法和引导学习策略来让模型适应不同强度分布的信号和图像后处理网络。本研究设计了包含四个仿真数据集和两个在体数据集的大量实验来验证模型的性能。实验结果表明，在联合基本图像后处理网络的前提下，该算法在有限视角稀疏采样条件下的理论重建精度、抗噪鲁棒性、病理重建鲁棒性和实际重建精度优于传统重建算法以及基于图像后处理的深度学习重建算法。

（2）提出了基于级联孪生网络的乳腺癌新辅助化疗治疗响应预测模型——深度学习影像组学管道。该模型包含两个按照时间级联的独立孪生网络，每个网络分别在不同时间点给出治疗响应预测结果，能够在化疗进程中实现基于多时间点的多步疗效预测。本研究将标准孪生网络的绝对值差别特征表示改为特征拼接特征表示，从而将差别特征学习改为共有特征学习，有效缓解了低一致性超声图像对结果的影响。此外，本研究还提出了多步迁移学习策略，通过自然图像到超声图像，简单问题到复杂问题的迁移学习方式，能够降低基于早期化疗图像模型的训练难度。本算法在单中心前瞻性数据集上表现出了良好的预测性能，对治疗无响应患者的预测准确率达到了90%。消融实验证明了本研究提出的特征表示方法与训练策略是有效的。

（3）提出了基于双输入Transformer的乳腺癌新辅助化疗病理性完全缓解术前评估模型。本研究是上述研究的拓展，利用Transformer的全局自注意力机制克服孪生网络内多时间点信息难以交互的弊端，进而提升模型在多时间点低一致性超声图像上的分析能力。本研究提出了基于独立Tokens-to-Token的块编码模块、共享位置编码模块、时间编码模块和基于加权均匀池化的特征表示模块，每个模块的设计都充分结合了多时间点超声图像的特点。本研究回顾性地收集了图像一致性更低的多中心多时间点超声图像数据，因而需要更有效的分析算法才能获得良好的评估性能。实验结果表明，本模型在该数据集上的表现优于孪生网络与基于单一时间点图像的Transformer模型，说明了本模型的整体有效性。消融实验的结果证明了本研究所提出的每个模块均能为模型性能带来一定的提升。

With the continuous progress of technology, medical imaging analysis has gradually become the main means of non-invasive disease diagnosis. Medical acoustic imaging is based on the principle of acoustics, using ultrasound as the original signal to provide the physiological information of imaging target, which is a major branch of medical imaging. Photoacoustic tomography (PAT) and ultrasonography are two major imaging modalities in medical acoustic imaging which are capable of reflecting the optical and acoustic properties of the imaging tissue respectively. Both PAT and ultrasonography are widely used in structural, functional, and molecular imaging.

Since acoustic imaging has the same signal domain, there are certain commonalities in image reconstruction and analysis algorithms, as well as some common challenges. In image reconstruction, the ultrasonic transducer serves as the signal receiver for acoustic imaging. Due to space or cost constraints, the ultrasonic transducer cannot achieve dense enough sampling or full-view coverage in many scenes, which leads to the loss of ultrasound signals, and then affects the quality of reconstructed images. Therefore, the research of reliable reconstruction algorithms for limited-view and sparsely sampled data will greatly broaden the application scenario of acoustic imaging. In image analysis, because the acoustic imaging has the characteristics of simple operation, non-radiation, and real-time imaging, it is very suitable for multi-time-point image acquisition to monitor the changes of lesions. However, the image quality of handheld acoustic imaging devices is easily affected by the operator, resulting in low consistency between images acquired over multiple time points. Therefore, how to analyze multi-time-point low-consistency images effectively has become a common problem in acoustic imaging.

The complexity of medical imaging information makes more and more artificial intelligence models adopt the multi-branch network structure represented by Siamese network. In recent years, multi-branch networks have been widely used in medical image reconstruction and analysis tasks and have achieved very promising results. In this thesis, the two common problems of acoustic imaging - limited-view and sparsely sampled reconstruction and multi-time-point low-consistency image analysis are taken as the starting point; the multi-branch network is taken as the starting point of methodology, and PAT image reconstruction and ultrasound image analysis are taken as the research contents to carry out relevant research, specially including:

(1) A PAT image reconstruction algorithm based on multi-branch feature projection network was proposed. Relying on the multi-branch network, this algorithm integrates the physical model of photoacoustic back-projection reconstruction into the reconstruction algorithm. By learning the transformation from the signal domain to the image domain, it can effectively reduce the information loss of the limited-view and sparsely sampled signal in the reconstruction process and improve the accuracy of the projection process. In addition to the innovation of network structure, the nonlinear signal pre-processing method and guided learning strategy were designed for the multi-branch feature projection network, which can help the model to adapt the signals with different intensity distribution and image post-processing network. In this research, numerous experiments based on four simulation datasets and two in vivo datasets were designed to verify the performance of the proposed model. Experimental results show that when the model combined with the post-processing network, the proposed reconstruction algorithm was superior to the traditional reconstruction algorithm and the post-processing-based deep learning reconstruction algorithm in terms of the theoretical reconstruction accuracy, noise robustness, pathological reconstruction robustness, and actual reconstruction accuracy under the condition of limited-view and sparsely sampled data. 

(2) A cascaded Siamese network model named deep learning radiomics pipeline for predicting response to neoadjuvant chemotherapy in breast cancer was proposed. The model consists of two independent Siamese networks cascaded by time and each network gives the prediction result of the treatment response at different time points, which can achieve the multi-step prediction based on multiple time points in the course of chemotherapy. In this research, the absolute difference feature representation of the standard Siamese network was changed into the feature concatenation feature representation, and the absolute feature difference learning was changed into the common feature learning, which can effectively relieve the influence of low-consistency ultrasound images. Moreover, this research proposed a multi-step transfer learning strategy. Through the transfer learning from natural images to ultrasound images and from the simple problem to the complex problem, the training difficulty of the model based on the images from early stage of chemotherapy can be reduced. This algorithm showed good predictive performance on a single-center prospective dataset and achieved a predictive accuracy of 90% for patients with no response to treatment. Ablation experiments demonstrated that the feature representation method and training strategy proposed in this research were effective.

(3) A dual-input Transformer for preoperative assessment of pathological complete response to neoadjuvant chemotherapy in breast cancer was proposed. This research is an extension of the above research, using the global self-attention mechanism of the Transformer to overcome the disadvantage of difficult interaction of multi-time-point information in the Siamese network, thereby further improving the analysis ability on multi-time-point low-consistency ultrasound images. This research proposed an independent Tokens-to-Token patch encoding module, a shared position encoding module, a time encoding module and a feature representation module based on weighted average pooling. The design of each module was fully incorporated the characteristics of multi-time-point ultrasound images. This research retrospectively collected multi-center multi-time-point ultrasound image data with lower image consistency, thus requiring more efficient analysis algorithms to achieve good assessment performance. Experimental results show that the proposed model outperformed the Siamese network and the Transformer model based on single-time-point images, demonstrating the overall effectiveness of the proposed model. The results of ablation experiments demonstrated that each module proposed in this research can improve the performance of our model.