中国科学院分子影像重点实验室知识库

数字病理技术，即将计算机技术应用于病理学领域的新型信息技术，是一种现代数字系统与传统病理学有机结合的新型产物。它是一种通过全自动显微镜或光学放大系统扫描采集得到高分辨数字图像，再应用计算机算法对得到的图像自动进行高精度处理与分析，获得优质的可视化数据以应用于病理学的各个领域的现代化新型临床辅助手段。

研究表明，肿瘤微环境（tumor mircroenvironment,TME）是肿瘤发生发展过程中所处的内环境，由肿瘤细胞，多种免疫细胞和间质细胞及肿瘤微血管等组成。由多种免疫细胞和细胞因子共同组成的TME是肿瘤免疫治疗是否响应的关键所在，与肿瘤免疫治疗的响应效率具有重要关系。因此，如何准确地分析TME、判断细胞类型与分析其空间相互作用对指导肿瘤免疫治疗具有重要意义。

本工作旨在利用数字病理技术，实现对TME中特定细胞表达的特异性细胞生物标志物的空间分布预测，从而实现对细胞分布的定位。医学图像处理问题通常面临着数据稀缺，数据标注难度大等挑战性问题，而病理图像对数据标注的专业要求尤为突出。此外，病理图像具有局部形态相似度高等固有特点，因此，本文针对上述存在的问题，根据病理图片的固有属性和任务特点，提出两种基于病理苏木精-伊红（hematoxylin and eosin, H&E）染色图像的细胞生物标志物预测算法，从预测精确度，鲁棒性，临床意义，临床应用前景等多个方向着手验证并优化算法性能。本文的主要工作和创新点归纳如下：

1、提出了一种基于UNet编码块的细胞生物标志物表达预测算法。

在病理图像中，肿瘤细胞的分布满足其特有的组织致密形态结构，由于肿瘤生长过程中的肿瘤浸润淋巴细胞（tumor infiltrating lymphocytes, TILs）的成团特性，TME的病理图片通常具有局部关联性的空间分布特征。为了更好地适应病理数据的结构分布特征，多尺度地提取细胞的有效特征进行分析与预测，本工作首先基于H&E染色图片以及多光谱免疫荧光染色图片，构建了有标签病理图像细胞生物标志物分布数据集，本工作结合了U形网络的整体结构以及多分支多尺度卷积核融合的特征提取通路，通过增加新的卷积分支来增加网络的感受野，以更好地提取到不同尺度下的图像特征信息,在更好地保留了从局部到整体的结构化信息的同时,突出了网络对局部细节特征信息的描述，已达到信息充分利用的目的，从而挖掘病理数据中的结构、语义特征，精准预测病理图像中表征TILs的生物标志物CD3、CD20、表征肿瘤细胞的生物标志物PanCK以及表征细胞核的生物标志物DAPI的表达分布。

2、提出了一种基于对抗学习策略的半监督细胞生物标志物表达预测算法。

在医学图像处理问题中，由于医学数据获取难度较大，并且数据标注难度大，特别是病理图片的标注，往往需要依靠专业的病理专家进行图像数据标注，标注任务量较大。基于对抗学习策略的半监督细胞生物标志物预测算法利用了一种生成器-判别器模型，利用生成器来完成对病理图像的初始预测，并训练判别器对预测图像的置信程度进行判断，以此完成对无标签数据的自学习过程，从而使得该网络可以利用部分有标签数据以及无标签数据进行模型训练，在保证模型泛化性能的同时降低了数据标注的工作量。该算法能够精准预测病理图像中的CD3、CD20、PanCK以及DAPI等细胞生物标志物的表达分布。针对该算法，本工作设计了多种对比试验以测试网络结构对预测精度的影响；此外，我们还结合了免疫荧光染色数据，验证了该方法对TILs以及肿瘤细胞的量化分析能力；另外，我们利用该算法筛选病理切片中的肿瘤区域，并在其基础上检测APC、TP53、KRAS等常见的肿瘤基因突变，进一步证实了该方法的临床应用前景。

以上研究成果以第一作者发表SCI检索期刊Cancers（IF：6.126）论文一篇；EI检索会议SPIE Medical Imaging 2021论文一篇，并受邀进行了口头汇报。申请发明专利一项。

Digital pathology technology refers to the application of computer technology to the field of pathology. It is a technology that combines modern digital systems with traditional pathological knowledge. It is a method that use high-resolution digital image acquired through automatic microscope or optical magnification system scanning and then use computer algorithms to automatically perform high-precision processing and analysis on the acquired image, and obtain high-quality visualization data for application in various fields of pathology for clinical aids.

Studies have shown that the tumor microenvironment (TME) is the internal environment that tumor grows, which is composed of tumor cells, a variety of immune cells and mesenchymal cells, and tumor capillaries. Studies have shown that the efficiency of tumor immunotherapy has an important relationship with the distribution of multiple immune cell populations in the TME. Therefore, how to accurately analyze theTME, determine the cell type and analyze its spatial interaction is of great significance for guiding tumor immunotherapy.

The purpose of this work is to predict the expression and distribution of cell biomarkers in TME by using deep learning methods. However, medical image processing task is faced with some challenging problems, such as the scarcity of data, the difficulty of data annotation and so on. In addition, pathological images also have the inherent characteristics of high morphological similarity. Therefore, in view of the above problems, in order to better tackle medical image processing problems, especially the inherent attributes and task characteristics of pathological images, this work proposes two kinds of cellular biomarker expression prediction algorithms based on pathological hematoxylin-eosin (H&E) staining images, the performance of the algorithm is optimized considering the prediction accuracy, robustness, clinical significance, and clinical application. The main contents and innovation points of this paper are summarized as follows:

1. Cell biomarker predictive network based on multi-resolution feature extraction UNet encoder.

In clinical usage, the distribution of tumor infiltrating lymphocytes (TILs) and tumor cells in pathological sections not only meets the unique histomorphology, but also has the characteristics of local correlation due to the clustering characteristics of TILs in the process of tumor growth. In order to better cope with the characteristics of the structural distribution of pathological data and to extract effective features of cell biomarkers for analysis and prediction, we first established the cell biomarker expression and distribution dataset which consists of H&E staining images and multiplexed immunofluorescence staining images. This work combines the encoder- decoder structure and multi-scale convolution kernel fusion branches which can make full use of the structural and semantic features in pathological data, so as to realize the accurate prediction of the cell biomarkers expression and distribution.

2. Semi-supervised cell biomarker predictive network based on adversarial learning strategy.

In medical image processing tasks, considering the difficulty of medical image acquisition and annotation, we proposed a semi-supervised cell biomarker predictive network, which adopted an encoder-decoder structure. We use the generator to make initial predictions and then use the discriminator to generate confidence map, only pixels with a high confidence are used for loss optimization, hence to train the unlabeled images in a self-taught manner and meanwhile improve the generation performance and reduce the burden of annotation work. In order to prove the effectiveness and clinical application potential of our proposed method, we conducted several comparative experiments. In addition, we also combined immunofluorescence staining data to validate the ability of the method to quantitatively analyze TILs as well as tumor cells; moreover, we used the algorithm to select tumor regions in pathological sections and performed tumor gene mutation detection experiments of APC, TP53 and KRAS on its basis, further confirming the clinical application prospects of the method.

The above research results were published as first author in SCI indexed journal Cancers (IF:6.126); and EI indexed conference SPIE Medical Imaging 2021 (Oral). A patent for inventions was also applied for based on this work.