With the continuous progress of technology, medical imaging analysis has gradually become the main means of non-invasive disease diagnosis. Medical acoustic imaging is based on the principle of acoustics, using ultrasound as the original signal to provide the physiological information of imaging target, which is a major branch of medical imaging. Photoacoustic tomography (PAT) and ultrasonography are two major imaging modalities in medical acoustic imaging which are capable of reflecting the optical and acoustic properties of the imaging tissue respectively. Both PAT and ultrasonography are widely used in structural, functional, and molecular imaging.
Since acoustic imaging has the same signal domain, there are certain commonalities in image reconstruction and analysis algorithms, as well as some common challenges. In image reconstruction, the ultrasonic transducer serves as the signal receiver for acoustic imaging. Due to space or cost constraints, the ultrasonic transducer cannot achieve dense enough sampling or full-view coverage in many scenes, which leads to the loss of ultrasound signals, and then affects the quality of reconstructed images. Therefore, the research of reliable reconstruction algorithms for limited-view and sparsely sampled data will greatly broaden the application scenario of acoustic imaging. In image analysis, because the acoustic imaging has the characteristics of simple operation, non-radiation, and real-time imaging, it is very suitable for multi-time-point image acquisition to monitor the changes of lesions. However, the image quality of handheld acoustic imaging devices is easily affected by the operator, resulting in low consistency between images acquired over multiple time points. Therefore, how to analyze multi-time-point low-consistency images effectively has become a common problem in acoustic imaging.
The complexity of medical imaging information makes more and more artificial intelligence models adopt the multi-branch network structure represented by Siamese network. In recent years, multi-branch networks have been widely used in medical image reconstruction and analysis tasks and have achieved very promising results. In this thesis, the two common problems of acoustic imaging - limited-view and sparsely sampled reconstruction and multi-time-point low-consistency image analysis are taken as the starting point; the multi-branch network is taken as the starting point of methodology, and PAT image reconstruction and ultrasound image analysis are taken as the research contents to carry out relevant research, specially including:
(1) A PAT image reconstruction algorithm based on multi-branch feature projection network was proposed. Relying on the multi-branch network, this algorithm integrates the physical model of photoacoustic back-projection reconstruction into the reconstruction algorithm. By learning the transformation from the signal domain to the image domain, it can effectively reduce the information loss of the limited-view and sparsely sampled signal in the reconstruction process and improve the accuracy of the projection process. In addition to the innovation of network structure, the nonlinear signal pre-processing method and guided learning strategy were designed for the multi-branch feature projection network, which can help the model to adapt the signals with different intensity distribution and image post-processing network. In this research, numerous experiments based on four simulation datasets and two in vivo datasets were designed to verify the performance of the proposed model. Experimental results show that when the model combined with the post-processing network, the proposed reconstruction algorithm was superior to the traditional reconstruction algorithm and the post-processing-based deep learning reconstruction algorithm in terms of the theoretical reconstruction accuracy, noise robustness, pathological reconstruction robustness, and actual reconstruction accuracy under the condition of limited-view and sparsely sampled data.
(2) A cascaded Siamese network model named deep learning radiomics pipeline for predicting response to neoadjuvant chemotherapy in breast cancer was proposed. The model consists of two independent Siamese networks cascaded by time and each network gives the prediction result of the treatment response at different time points, which can achieve the multi-step prediction based on multiple time points in the course of chemotherapy. In this research, the absolute difference feature representation of the standard Siamese network was changed into the feature concatenation feature representation, and the absolute feature difference learning was changed into the common feature learning, which can effectively relieve the influence of low-consistency ultrasound images. Moreover, this research proposed a multi-step transfer learning strategy. Through the transfer learning from natural images to ultrasound images and from the simple problem to the complex problem, the training difficulty of the model based on the images from early stage of chemotherapy can be reduced. This algorithm showed good predictive performance on a single-center prospective dataset and achieved a predictive accuracy of 90% for patients with no response to treatment. Ablation experiments demonstrated that the feature representation method and training strategy proposed in this research were effective.
(3) A dual-input Transformer for preoperative assessment of pathological complete response to neoadjuvant chemotherapy in breast cancer was proposed. This research is an extension of the above research, using the global self-attention mechanism of the Transformer to overcome the disadvantage of difficult interaction of multi-time-point information in the Siamese network, thereby further improving the analysis ability on multi-time-point low-consistency ultrasound images. This research proposed an independent Tokens-to-Token patch encoding module, a shared position encoding module, a time encoding module and a feature representation module based on weighted average pooling. The design of each module was fully incorporated the characteristics of multi-time-point ultrasound images. This research retrospectively collected multi-center multi-time-point ultrasound image data with lower image consistency, thus requiring more efficient analysis algorithms to achieve good assessment performance. Experimental results show that the proposed model outperformed the Siamese network and the Transformer model based on single-time-point images, demonstrating the overall effectiveness of the proposed model. The results of ablation experiments demonstrated that each module proposed in this research can improve the performance of our model.
|Keyword||光声断层成像 超声成像 图像重建 图像分析 多分支网络 深度学习|
|童同. 基于多分支网络的医学声学图像重建与分析算法研究[D]. 中国科学院自动化研究所. 中国科学院自动化研究所,2022.|
|Files in This Item:|
|Recommend this item|
|Export to Endnote|
|Similar articles in Google Scholar|
|Similar articles in Baidu academic|
|Similar articles in Bing Scholar|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.