毕业生知识库

神经科学研究对于理解大脑的复杂性及其功能至关重要。其中一个重要的研究手段是通过绘制微观脑图谱，在细胞水平上可视化大脑的结构和组织。这些脑图谱可以帮助我们理解大脑如何处理信息以及神经元之间是如何相互连接的。然而，大规模绘制微观脑图谱离不开序列切片电子显微镜成像技术获得的大量数据。这种序列切片过程会导致连续性的丢失，进而导致神经元的形态和连接难以恢复。因此，在微观脑图谱绘制过程中，需要进行连续性重建，以便能够恢复神经元之间的连接，并更好地理解大脑的结构和功能。

当前微观脑图谱的连续性重建面临匹配困难、过度匹配和误差累积、以及切片破损等多个关键问题。本文提出了三种解决方案，以解决这些问题。首先，本文利用卷积神经网络建模生物切片，以解决传统特征无法捕捉生物组织天然形变的问题，提升匹配效率。其次，本文提出了一种基于序列切片配准的空间连续性重建算法，以应对序列配准中的过度匹配和误差累积问题。最后，本文还提出了一种针对破损切片的空间连续性恢复流程。这三种方案都展示了较好的结果，并为微观脑图谱连续性重建的进一步推进做出了贡献。本文还综合运用以上三种解决方案，提出了一个流程完成了国际上首套带生物学标记的斑马鱼全脑体量微观脑图谱绘制工程的空间连续性重建。本文的主要工作和创新包括以下四点：

    生物结构形态变化不敏感的描述子提取

    本项研究旨在解决生物切片图像描述问题。传统特征难以度量序列切片中生物组织轴向上的形态变化，因此，我们利用聚焦离子束扫描电子显微镜图像的原位性特点，采用深度学习方法提取学习型图像描述子。这些描述子可以忽略部分切片间的生物组织形态变化，从而让配准算法更加关注神经组织位置变化。为解决切片图像形变场难以标注的问题，我们利用聚焦离子束扫描电子显微镜图像的天然对齐特性，构建切片图像描述子匹配的代理任务，并提出自监督描述子提取框架，通过自监督训练，有效避免了切片图像形变数据的标注困难。此外，我们采用困难样本挖掘的方法对训练样本进行采样，并以此构造神经网络的训练损失。这种方法可以使神经网络正确区分生物结构形态变化与不同细胞差异之间的区别，保证配准结果的连续性，同时避免了过度匹配的问题。实验结果表明，本文提出的特征提取方法在匹配精度方面优于现有方法，并在应用于序列切片电子显微镜图像配准时，使得配准算法更专注于神经组织位置变化而非固有的形态变化。

    基于光流网络的空间连续性重建算法

    本项研究旨在通过序列切片配准方法恢复微观脑图谱的空间连续性。为此，我们提出了一种新的序列切片配准方法，利用深度学习光流网络进行切片间的配准。该方法使用对生物结构形态变化不敏感的描述子度量切片相似性，使得光流网络更注重切片的位置偏差而非形态学差异，从而提高了切片配准的准确性。此外，我们提出了一种层次化配准框架，通过将长序列转换为多个短序列的叠加。在长序列中，我们采用抽取基准层并对其进行刚性配准的方法重建整体空间连续性。在短序列内部，我们使用光流网络建模切片中存在的复杂形变关系并进行切片的序列配准。为了消除短序列内部配准误差的累积问题，我们提出了结构回归方法：在序列配准完成后，利用光流网络估计累积配准误差，并使用切片间相似性作为误差分配的权重，以一次性消除短序列内部的累积误差。实验结果表明，该方法能够有效提高序列切片的空间连续性，实现更准确的配准并改善生物组织结构的重建。

    损伤切片空间连续性恢复算法

    本项工作旨在解决绘制微观脑图谱的采集序列切片过程中可能出现的切片破损问题。由于破损切片中神经结构在成像平面上的连续性会被破坏，这使得与序列中其他完整的切片进行配准变得困难。为解决该问题，我们提出了一种新的序列切片配准流程，旨在更准确地配准包含不连续形变的破损切片。该方法首先提取待配准切片的关键点和描述子，再通过相互最近邻匹配器进行匹配。随后，我们使用K-means和随机样本一致性检查对关键点进行聚类，并估计这些聚类的局部仿射矩阵。然后我们计算每一类的概率密度并以此生成全局放射变换。在聚类和概率密度计算中，我们采用路径距离替代欧氏距离来度量采样点之间的相关强度。实验结果表明，该流程可以有效解决破损切片的配准问题，恢复成像平面上神经结构的空间连续性，避免破损切片对轴向空间连续性重建的影响。该方法能够适用于处理包含不连续形变的切片，提高了大型微观脑图谱连续性重建工作的数据利用率。

    斑马鱼全脑微观图谱数据的空间连续性重建。

    本项工作利用上述三种方法对其进行集成性应用，成功实现斑马鱼全脑微观图谱数据的空间连续性重建。该套数据是中国科学院脑科学与智能技术卓越创新中心-人工智能创新研究院2035创新交叉攻关项目斑马鱼全脑微介观联接图谱绘制项目中的一部分，是国际上首套带标记的斑马鱼全脑微观图谱数据，同时也是国内完成的最大体量微观脑图谱绘制工程。利用本论文所提出的流程，目前已获得斑马鱼全脑微观三维图像库。该工作的成功实施不仅为其他大型微观脑图谱空间连续性重建项目的开展提供了有效的解决方案，也对于我们理解大脑的结构与功能，突破现有计算框架提供了策略和思路，充分展现了本篇论文工作的系统性。

The field of neuroscience plays a vital role in unraveling the intricacies of brain function and complexity. One of the key tools employed in this field is the visualization of the brain's cellular structure and organization through the creation of micro-scale brain atlases. These atlases serve as a valuable resource for understanding the neural circuitry and the processing of information within the brain. However, creating large-scale micro-brain atlas requires a substantial amount of data that is typically acquired through serial section electron microscopy imaging techniques. Unfortunately, the serial sectioning process can cause discontinuities, thereby hampering the ability to accurately reconstruct the morphology and connections of neurons. Therefore, to improve our understanding of the structure and function of the brain, it is necessary to undertake continuity reconstruction during the creation of micro-scale brain atlas, so as to restore the disrupted connections between neurons.

The continuity reconstruction of micro-brain atlas presents several challenges, including difficulties in matching, over-registration and error cumulation, and damage to sections. In this thesis, we propose three solutions to overcome these issues. Firstly, we propose the use of a convolutional neural network to model biological sections. This approach addresses the limitations of traditional features, which fail to capture the natural deformation of biological tissue, thereby enhancing the matching efficiency. Secondly, we introduce a spatial continuity reconstruction algorithm based on serial section registration to mitigate the problems of over-registration and error cumulation. Lastly, we propose a spatial continuity restoration process for damaged sections. These three solutions demonstrate excellent results and represent significant contributions to the advancement of continuity reconstruction in micro-scale brain atlas. In addition, a process that combines the above three solutions is proposed to complete the spatial continuity reconstruction of the world's first set of  zebrafish whole-brain with biological markers volume microscopic brain atlas. The main innovations of this thesis are the following four points:

    Descriptor Extraction Insensitive to Biological Structural Morphological Changes.

    The primary objective of this study is to tackle the issue of describing biological section images. Traditional features have limitations in capturing the axial direction morphological changes of biological tissues in serial sections. To overcome this challenge, we leverage the in-situ characteristics of focused ion-beam scanning electron microscopy images and utilize deep learning methods to extract learned image descriptors. These descriptors effectively disregard the morphological changes in biological tissues between sections, allowing registration algorithms to focus more on the changes in the position of neural tissues. To address the difficulty of labeling the deformation field of section images, we exploit the natural alignment characteristics of focused ion-beam scanning electron microscopy images to develop a pretext task for section image descriptor matching, and propose a self-supervised descriptor extraction framework. By means of self-supervised training, we effectively bypass the labeling of section image deformation data. Additionally, we employ hard sample mining to sample the training data and construct the neural network training loss. This approach enables the neural network to differentiate between the differences in biological structural morphological changes and various cell differences, guaranteeing the continuity of registration results while preventing the issue of excessive matching. Experimental results demonstrate that the proposed feature extraction method surpasses existing techniques in terms of matching accuracy. When applied to electron microscope image registration of serial sections, it allows registration algorithms to concentrate more on the changes in the position of neural tissues, rather than inherent morphological changes.

    Serial Registration Algorithm for Spatial Continuity Reconstruction.

    The purpose of this research is to restore the spatial continuity of micro-scale brain atlas through a serial registration algorithm. To achieve this, we introduce a novel serial registration method that employs a deep learning-based optical flow network to register sections. This method employs descriptors that are insensitive to biological structural morphological changes to measure section similarity, allowing the optical flow network to focus more on positional deviations rather than morphological differences, thereby enhancing the accuracy of section registration.Moreover, we propose a hierarchical registration framework that decomposes long serial into multiple shorter serials. Within the long serial, we extract a reference layer and perform rigid registration to reconstruct the overall spatial continuity. For the short serials, we model the complex deformation relationships in sections using the optical flow network and perform serial registration. To eliminate the accumulation of registration errors within short serials, we propose a structural regression method that estimates cumulative registration errors using the optical flow network and uses section similarity as weights for error compensation to prevent the cumulation of errors within the short serials.Experimental results demonstrate that this method effectively improves the spatial continuity of serial sections, resulting in more accurate registration and better reconstruction of biological tissue structure.

    Damaged section Spatial Continuity Restoration Algorithm.

    The aim of this study is to address the issue of damaged sections in the collection of serial sections in the creation of micro-scale brain atlas, which can disrupt the continuity of neural structures on the imaging plane and make accurate registration with other intact sections in the serial difficult. To solve this problem, we propose a novel damaged section registration process that can accurately register damaged sections containing non-continuous deformations. First, key points and descriptors are extracted from the section to be registered, and then they are matched using a nearest neighbor matcher. Next, we cluster the key points and estimate the local affine matrix of these clusters using K-means and random sample consensus checks. We then calculate the probability density of each cluster and use this to generate the global expectation transformation. In clustering and probability density calculation, we use path distance instead of Euclidean distance to measure the strength of correlations between sampled points. Experimental results demonstrate that this process can effectively solve the registration problem of damaged sections, restore the spatial continuity of neural structures on the imaging plane, and avoid the impact of damaged sections on axial spatial continuity reconstruction. This method can be applied to register sections containing non-continuous deformations and can improve the data utilization rate of large-scale micro-scale brain atlas continuity reconstruction work.

    The Spatial Continuity Reconstruction of the Zebrafish Whole-brain Micro-Scale Atlas Data.

    This work successfully achieved the spatial continuity reconstruction of the zebrafish whole-brain microscale map data by integrating the three methods mentioned above. This set of data is part of the Zebrafish Whole-brain Micro-meso Scale Connectome Altas Project of the CAS Center for Excellence in Brain Science and Intelligence Technology - AI Innovation Research Institute 2035 Innovation Cross-cutting Project. It is the first set of labeled zebrafish whole-brain microscale map data in the world, and also the largest volume of microscale brain map drawing project completed in China. Using the process proposed in this paper, a zebrafish whole-brain microscale three-dimensional image library has been obtained. The successful implementation of this work not only provides an effective solution for other large-scale microscale brain map spatial continuity reconstruction projects, but also provides strategies and ideas for us to understand the structure and function of the brain and break through the existing computing framework, fully demonstrating the systematic nature of this paper.