毕业生知识库

更好地理解大脑工作机制，需要破译大脑功能背后复杂的神经元“连接线路 图”。大脑微观连接的研究需要通过连续切片和电子显微镜成像技术获得大规模 突触水平神经回路图像数据。由于基于切片的数据获取过程不可避免地造成脑 组织原始三维信息丢失，因此研究将序列二维电镜图像对齐成三维体块的脑序 列切片电镜图像体重建算法，恢复神经结构的原始三维形态对脑微观连接图谱 绘制至关重要。

脑序列切片电子显微镜图像体重建算法需要解决两个关键问题：神经解剖 结构 𝑧 轴连续性恢复和解剖结构原始三维形态复原。针对这两个问题，本文充分 分析和利用脑切片图像内容中神经结构的生物学先验信息，从神经结构所在的 切片内二维空间（2D）、相邻切片间的三维空间（3D）、序列切片与参考体块对 应的二维到三维空间（2D-3D）三个角度出发，展开体重建算法的研究：本文首 先针对脑组织切片图像切片内纹理相似度高的特点，学习切片内二维神经结构 间的相对空间关系，提出基于自注意力机制的深度自监督图像配准网络，接着针 对脑组织切片间结构变化大的特点，建模切片间三维神经结构的形态变化关系， 提出基于神经形态模型的序列图像配准框架，从而实现序列切片图像中神经解 剖结构沿 𝑧 轴连续性的恢复；最后，引入了原位参考信息，挖掘二维切片到三维 参考体块中神经结构在不同模态数据间的对应关系，提出基于 Transformer 的切 片到体块图像配准网络，完成序列切片电镜图像中神经解剖结构原始三维形态 的复原。论文的主要工作和创新点归纳如下：

1. 体重建中局部图像间相对空间关系学习。针对脑序列切片电镜图像切片 内相似性高引起的体重建困难，本文提出基于自注意力的自监督深度学习配准 网络，学习切片内局部图像块与全局内容之间空间相对性关系。提出的方法利用 卷积网络结合自注意力整合局部和全局空间相关信息，提高局部图像特征的显 著性，实现切片图像间空间形变场的获取，同时结合上空间变换网络对源图像进 行变形，实现端到端的自监督深度学习训练。在合成形变数据集和真实形变数据 集上进行对比，提出的方法在相似度指标上实现了 5%-15% 的精度提升。重建体 块的 𝑧 轴剖面图表明提出的方法能够缓解对应关系错误带来的纹理模糊和边界不清的问题，可以提供平滑的变换，可靠的体重建结果证明了本文提出的方法在 恢复序列电镜图像三维形态上的有效性。

2. 体重建中相邻图像间神经结构变化关系建模。针对脑序列切片电镜图像 切片间结构差异大造成的体重建挑战，本文采用球形变模型来模拟局部神经元 三维形态结构，利用相邻图像中神经元结构形状的变化与配准精度之间的关系， 选择圆形结构提取对应关系，有效降低在复杂变化的神经组织图像中寻找准确 鲁棒对应关系的难度。在此基础上，本文提出了基于神经元形态学模型的序列切 片图像体重建框架，利用膜分割网络和神经形态学物理选择模型来筛选神经组 织图像中稳定的圆形结构区域，通过构建两两图像对间对应点距离的全局优化 函数，求得调整后的序列间对应关系。通过对脑切片中神经结构三维形态信息的 有效利用，提出的方法在多个数据集的实验比较中实现了更稳定的、更符合神经 解剖结构的脑序列切片电镜图像体重建。

3. 体重建中序列切片与原位参考空间对应关系挖掘。为了复原神经解剖结 构原始三维形态，本文引入脑组织样品显微计算机断层扫描成像获得的原位 3D 图像，引导序列切片电镜图像进行体重建。针对脑切片电镜 2D 图像与参考体块 3D 图像配准中存在的维度和模态差异的挑战，本文提出了基于 Transformer 的切 片到体块图像配准方法，利用解剖结构分割结果的点云作为配准方法的输入。本 文提出结合基于注意力机制的自特征增强模块和引导特征增强模块，找到不同 模态数据间正确的对应关系，解决不同成像模态带来的配准困难，提出基于交叉 注意的融合模块来消除维度鸿沟，自适应地关注对任务更有用的信息。在合成数 据集和真实数据集上与现有方法进行对比，提出的方法在定量评价指标上实现 了精度的提升, 切片与体块配准结果中解剖结构标志点间的距离减小到 3.89𝜇𝑚。 本方法使切片电镜图像在参考体块图像中找到准确的对应关系，确定序列切片 在原始三维空间中的位置，从而实现神经解剖结构原始三维形态的复原。  

A better understanding of the brain working mechanisms requires deciphering the complex neuronal ”connectivity maps” behind brain functions. The study of brain micro connection needs to obtain large-scale synaptic neural circuit image data obtained by serial slicing and electron microscope (EM) imaging technologies. Section-based brain imaging method losing its three-dimensional integrity requires the integration of stacks of 2D images of brain sections into volumetric form by serial section electron microscopy (ssEM) image volume reconstruction algorithm.

There are two critical problems needing to be solved in the volume reconstruction algorithm of ssEM images: The restoration of Z-axis continuity of neuroanatomical structures and recovering the original 3D morphology of neuroanatomical structures. To address these problems, this paper fully analyzes and utilizes the prior biological information of neural structures in the brain sections and develops the volume reconstruction algorithm from three perspectives: two-dimensional space (2D) within sections, three-dimensional space (3D) between adjacent sections, two-dimensional to three-dimensional space (2D-3D) corresponding to serial sections and reference volume. Firstly, considering the characteristics of high texture similarity in brain section images, this paper studies the relative spatial relationship between neural structures in the sections and proposes a deep self-supervised image registration network based on the self-attention mechanism. Then, in light of significant structural changes between brain section images, the morphological change relationship of neural structures between sections is modeled, and this paper proposes a serial image registration framework based on the neural morphological model. The first two studies restore the continuity of neuroanatomical structures along the Z-axis in serial section images. Finally, the in-situ reference information is introduced to dig the corresponding relationship between different modal data of neural structures in sections to reference volume. A transformer-based slice-to-volume image registration network is proposed to restore the original 3D morphology of neural anatomical structures in ssEM images. The main work and innovation of the paper are summarized as follows.

1. Relative space relation learning between image patches in volume reconstruction. For the difficulties of volume reconstruction caused by high intra-section similarity in ssEM images, this paper proposes a deep self-supervised image registration network based on self-attention mechanism to learn the spatial correlation between local and global images. The proposed method uses convolutional networks combined with spatial attention to integrate local and global spatially relevant information, which improves the saliency of local image features, and achieve the acquisition of spatial deformation fields between sliced images while combining the upper spatial transformation network to deform the source images to achieve end-to-end self-supervised learning training. Comparative experiments on the synthetic and real deformation datasets demonstrate the proposed method achieves an accuracy improvement of 5%-15% on similarity metrics. The Z-axis sideviews of the reconstructed volume show that the proposed method can alleviate the texture blurring and unclear boundaries caused by correspondence errors, and can provide smooth transformations. The reliable volume reconstruction results prove the effectiveness of the proposed method in the way of recovering the 3D morphology of the ssEM images.

2. Neural structure change relation modeling between adjacent images in volume reconstruction. Facing the challenge of large structural differences between slices in volume reconstruction, in this paper, the spherical variable model is used to simulate the 3D morphological structure of local neurons. Using the relationship between the change of neuron structure shape in adjacent images and registration accuracy, the circular structure is selected to extract the correspondences, which effectively reduces the difficulty of finding accurate and robust corresponding relationships in complex changing neural tissue images. On this basis, this paper proposes a ssEM image volume reconstruction framework based on a neural morphology model. The membrane segmentation network and neuromorphology physical selection model are used to screen the stable circular structure areas in the neural tissue images. By constructing the global optimization function of the distance between the correspondences between adjacent images, the adjusted corresponding relationship is obtained. By effectively utilizing the 3D morphological information of neural structures in brain slices, the proposed method achieve a more stable and more consistent ssEM image volume reconstruction in the experimental comparison of multiple datasets.

3. Slice and volume corresponding relation mining in volume reconstruction. In order to recover the original 3D morphology of neuroanatomical structures, this paper introduces in-situ 3D images obtained by Micro-Computed Tomography (Micro-CT) of brain tissue samples to guide the ssEM images volume reconstruction. To address the challenges of dimensional and modal differences in the registration of brain section EM 2D images and reference 3D volumetric images, this paper uses the point cloud of anatomical structure segmentation results as input. In this paper, self feature enhancement module and guided feature enhancement module based on attention mechanism are proposed to find the correct corresponding relationship between different modal data and solve the registration difficulties caused by different imaging modalities. Furthermore, a fusion module based on cross attention is proposed to eliminate the dimension gap and adaptively focus on more useful information for the task. Compared with existing methods on synthetic and real datasets, the proposed method achieves an accuracy improvement on quantitative metrics. The distance between anatomical landmarks in the results of slice-to-volume registration is reduced to 3.89 um. The proposed method enables the section EM images to find an accurate correspondence in the reference Micro-CT images to determine the position of the serial sections in the original 3D space, thus achieving the recovery of the original 3D morphology of the neuroanatomical structures.