CASIA OpenIR  > 毕业生  > 博士学位论文
基于颅内电生理记录的面孔加工脑活动研究
李文路
2024-05
Pages131
Subtype博士
Abstract

面孔加工是人脑最重要的认知功能之一。人们可以通过面孔了解他人的身份、性别和年龄,也能够通过面孔分析他人的情感、判断他人的社会,因此面孔承载了丰富的个人信息和社会信息。为解析面孔所包含的丰富的信息,大脑对面孔信息的加工与处理经历了一系列复杂的阶段,例如面孔检测、面孔身份识别和面孔社会特征判断。面孔加工的重要性和复杂性使得大脑形成了专门处理面孔信息的神经回路和神经机制。

大量研究致力于探查面孔加工的神经机制,建立了各种面孔加工神经认知模型,这些模型包含了多个面孔加工阶段,阐明了每个加工阶段涉及的脑区以及阶段之间的信息传递模式。然而,目前仍然缺乏在单个面孔加工阶段以及不同阶段之间的交互作用的更细粒度(例如更高的时空分辨率)的脑活动研究。例如在面孔检测阶段,已有研究表明人脑颞叶腹侧皮层的局部神经活动会被面孔图像特异性地激活,而且人类面孔检测行为对清晰度线性变化的面孔图像呈现出一种非线性、阈值性的变化模式,然而目前仍然不清楚与这种行为变化模式相关的神经测量指标,以及这种有视觉刺激引起的局部神经活动是否与意识水平的面孔检测有关(即被试表现出对面孔的主观感知);在面孔识别阶段,有研究表明人类对面孔身份的识别与面孔检测的速度一样快,然而相反观点认为人类遵循从粗到细的加工模式,即先检测面孔再识别面孔,因此目前并不清楚人脑如何组织面孔识别与面孔检测这两个阶段的加工顺序,以及在同一个加工阶段内的不同时间的表征格式的差异;在面孔社会特征判断阶段,由于面孔同时包含了身份信息与社会特征信息,尚不清楚面孔身份信息对面孔社会特征信息的影响。

近年来,随着颅内电生理技术的发展,越来越多的工作将该技术应用到面孔加工的研究当中。相比于非侵入性的神经影像技术和头皮脑电记录,颅内电生理技术可以精准定位大脑内部区域,直接测量深部神经元集群的场电位或单神经元放电,具有更高的信噪比和时空分辨率,为研究面孔加工的认知过程提供了更详细和更准确的神经活动信息。本文研究基于人脑颅内电信号(皮层局部场电位和单神经元放电),对面孔加工过程中的四个关键问题进行探讨,为面孔加工神经认知模型提供电生理证据,促进对面孔加工的神经机制的理解。论文的研究内容如下:

(1)已有研究表明面孔清晰度对人类面孔检测行为的影响不是线性的,随着面孔图像清晰度的线性减小,人类的面孔检测行为呈现出一种非线性、阈值性的变化模式。本项研究旨在探寻与这种非线性行为变化模式相关的神经测量指标。在实验范式中,4名人类被试观看了图像清晰度线性减小的面孔图像和非面孔(房屋)图像,并在看到面孔图像时按键报告,同时采集人脑颞叶腹侧皮层的局部场电位信号;在数据分析中,分析对面孔图像有特异性响应的神经信号的变化模式,并计算信号幅度与人类面孔检测行为准确度的相关性。结果表明:1)对面孔图像有特异性响应的神经信号主要位于梭状回及其附近皮质;2)随着面孔清晰度的线性减小,面孔特异性神经活动幅度呈现出非线性、阈值性的变化模式,与面孔检测的行为变化模式高度相似;3)面孔特异性神经活动幅度与被试的面孔检测准确度显著相关。这些结果表明人脑颞叶腹侧皮层的面孔特异性神经活动是与面孔检测行为变化模式相关的神经测量指标。

(2)已有研究表明人类的面孔检测行为准确度表现出对视觉输入的清晰度的非线性依赖,行为表现的骤变体现了人脑在面孔检测过程中的视觉输入与意识感知的转变。上一章研究结果表明人脑颞叶腹侧皮层的神经信号也表现出对视觉输入的非线性依赖,且变化模式与人类的行为变化模式相似。然而,由于神经活动与行为都受到视觉输入的影响,两者的相似的变化模式并不能作为这些神经信号参与意识水平的面孔检测的充分证据。本项研究旨在检验这种局部面孔特异性神经活动是否与意识水平的面孔检测有关(即被试表现出对面孔的主观感知)。在实验范式中,4名人类被试观看了图像清晰度线性减小的面孔图像和非面孔(房屋)图像,并在看到面孔图像时按键报告,同时采集人脑颞叶腹侧皮层的局部场电位信号;在数据分析中,将引起人类面孔检测行为准确度骤变的视觉输入定义为意识检测的阈值水平范围,并在意识检测的阈值水平范围内,比较不同感知条件(主观报告看到面孔/没有看到面孔)的面孔特异性神经活动的响应差异。结果表明:在视觉输入是相同清晰度的面孔图像的情况下,相比于人类被试没有主观感知到面孔,当主观感知到面孔时,人脑的1)面孔特异性神经活动的幅度更高;2)面孔特异性神经活动表现出更一致的响应模式;3)沿着颞叶后部传向颞叶前部的面孔特异性神经活动的传递强度更高;4)面孔特异性神经活动曲线的峰值发生时间与被试的响应时间显著相关,且始终超前于被试的响应时间200毫秒左右。这些结果表明人脑颞叶腹侧皮层的局部面孔特异性神经活动与意识水平的面孔检测有关。

(3)面孔检测与面孔识别是两个不同的面孔加工阶段。由于人类在日常生活中获取了大量面孔相关的视觉经验,因此有研究认为人类是面孔加工的专家,人类对面孔的识别与检测可能是同时进行的;而与这种“面孔专家观点”相矛盾的观点认为,面孔加工遵循的是一种从粗到细的加工策略,即在进行更精细的面孔识别之前,需要先经过面孔检测阶段。本项研究旨在探查人脑中这两个阶段的加工顺序,以及在同一个加工阶段内的不同时间的表征格式的差异,并验证了这种加工顺序组织是否是一种计算优化。在实验范式中,8名人类被试观看了清晰的面孔图像和非面孔(房屋)图像,同时采集人脑颞叶腹侧皮层的场电位信号;在数据分析中,使用表征相似性分析对人脑和人工神经网络的面孔检测与面孔识别这两个阶段的加工顺序进行了探究。结果表明:1)人脑颞叶腹侧的面孔特异性神经活动在170~390毫秒期间表现出类别特异性表征,可能参与面孔检测功能;在280~390毫秒期间表现出身份特异性表征,可能参与面孔识别功能;2)面孔检测的神经表征在130~ 200毫秒期间表现出动态性,面孔识别的神经表征在整个试次期间保持稳定;3)人工神经网络表现出与人脑相同的面孔加工顺序。这些结果表明人脑使用“先检测,后识别”的面孔加工策略,早期时间段的面孔类别信息的神经表征会随时间而变化,而且这种加工策略可能是由于计算优化形成的。

(4)面孔同时包含了身份信息和社会特征信息。本项研究旨在探查面孔身份信息对面孔社会特征信息的影响。在实验范式中,12名人类被试观看了清晰的面孔图像,同时采集人脑内侧颞叶(杏仁核和海马)的单神经元放电信号;在数据分析中,比较具有不同视觉经验的卷积神经网络与人脑内侧颞叶神经元对面孔社会特征的表征相似性。结果表明:1)具有面孔身份识别视觉经验的卷积神经网络(预训练VGG-Face)表现出类脑的社会特征神经表征,包括社会特征耦合效应和社会特征混淆效应;2)具有相同网络架构而没有面孔身份识别视觉经验的卷积神经网络(VGG-16和未训练VGG-Face)的社会特征神经表征没有表现出任何类脑效应;3)在所有单个社会特征表征上,相比于VGG-16和未训练VGG-Face,预训练VGG-Face和人脑的神经表征的相似性更高。这些结果表明类脑的社会特征神经表征需要面孔身份识别视觉经验,揭示了面孔身份识别视觉经验对发展面孔社会特征判断的必要性。

总之,本文围绕面孔加工过程中的面孔检测、面孔身份识别和面孔社会特征判断三个阶段的四个研究问题,基于人脑颞叶的颅内电生理信号,结合传统的脑电信号分析方法、机器学习方法以及深度神经网络,探究在这三个面孔加工阶段以及不同阶段之间的交互作用的神经响应特性,为揭示面孔加工的神经机制提供电生理学方面的支持证据,同时为使用神经网络解决认知神经科学领域的问题提供新的思路。

Other Abstract

Face processing is one of the most crucial cognitive functions of the human brain. People can discern others' identities, gender, and age through faces, as well as analyze emotions and judge personalities, thereby faces carry rich personal and social information. To decode the wealth of information contained in faces, the brain undergoes a series of intricate stages in processing face information, such as face detection, face identity recognition, and assessment of face social traits. The significance and complexity of face processing have led the brain to develop specialized neural circuits and mechanisms for handling face information.

A large body of research has been dedicated to investigating the neural mechanisms underlying face processing, leading to the development of various neurocognitive models of face processing. These models encompass multiple stages of face processing, elucidating the brain areas involved in each processing stage and the patterns of information transmission between these stages. However, there is still a lack of finer-grained (such as higher temporal and spatial resolution) neural activity studies at individual stages and the interactions between different stages. In the face detection stage, previous studies have indicated a nonlinear, threshold-like change in human face detection behavior in response to face clarity, but the neural measurement indices associated with this behavioral change pattern remain unclear. In addition, previous studies have shown that local neural activity in the ventral temporal lobe cortex of the human brain is specifically activated by face images, but it is still uncertain whether this localized neural activity is related to conscious face detection (i.e., the subjective perception of faces). In the face recognition stage, some studies suggest that humans identify face identities as quickly as they detect faces, while the contradictory view proposes that humans detect faces first and then recognize them. Thus, it is currently unclear how the human brain organizes the sequence of face detection and recognition stages and whether there are differences in representation formats at different times within the same processing stage. In the face social trait judgment stage, due to faces containing both identity and social trait information simultaneously, it remains unclear how face identity information influences face social trait information.

In recent years, with the advancement of intracranial electrophysiological techniques, an increasing number of studies have applied this technology to the study of face processing. Compared to non-invasive neuroimaging techniques and scalp electroencephalogram (EEG) recordings, intracranial electrophysiological techniques can precisely locate intracranial regions, directly measure field potentials of deep neural clusters and single-unit discharges, and offer higher signal-to-noise ratio and spatiotemporal resolution, providing more detailed and accurate neural activity information for studying the cognitive processes involved in face processing. This paper, based on intracranial neural recordings (local field potentials and single-unit discharges), discusses four key issues in the process of face processing. It aims to provide electrophysiological evidence for neurocognitive models of face processing and advance understanding of the neural mechanisms underlying face processing.

(1) Previous research has shown that the effect of face clarity on human face detection behavior is not linear. As face image clarity linearly decreases, human face detection behavior exhibits a non-linear, threshold-like change pattern. In this study, to explore neural measurement indices associated with this behavioral change pattern, four human participants viewed face images and non-face (house) images with linearly decreasing clarity, and reported key presses when observing face images, local field potential signals from the ventral temporal lobe cortex of the human brain were collected. Then the change pattern of neural signals specifically responsive to face images was analyzed, and the correlation between signal amplitude and the accuracy of human face detection behavior was calculated. The results indicate that: 1) Neural signals specifically responsive to face images are mainly located in the fusiform gyrus and its adjacent cortex; 2) As clarity decreases linearly, the amplitude of face-specific neural activity exhibits a non-linear, threshold-like change pattern, highly similar to the behavioral change pattern of face detection; 3) The amplitude of face-specific neural activity is significantly correlated with the accuracy of face detection behavior. These results suggest that face-specific neural activity in the ventral temporal lobe cortex of the human brain is a neural measurement metric associated with the behavioral change pattern of face detection.

(2) Existing research has shown that the behavioral performance of human face detection exhibits nonlinear dependencies on the clarity of visual input, and the sudden changes in behavioral performance reflect the transformation of visual input and conscious perception of the human brain in the face detection process. We found that face-specific neural activity in the ventral temporal cortex of the human brain also shows nonlinear dependence on visual input, and the change pattern is similar to the behavioral change pattern. However, since both the behavioral performance and the face-specific neural activity are affected by visual input, similar change patterns are not sufficient to prove that this local face-specific neural activity is involved in conscious face detection. In this study, to examine whether this localized face-specific neural activity is related to conscious face detection (i.e., the subjective perception of faces), four human participants viewed face images and non-face (house) images with linearly decreasing clarity, and reported key presses when observing face images, local field potential signals from the ventral temporal lobe cortex of the human brain were collected. Then the visual input causing sudden changes in human behavior was defined as the threshold level range for conscious detection, and within this threshold level range, responses of face-specific neural activity under different perceptual conditions (subjective report of seeing faces vs. not seeing faces) were compared. The results indicate that even if the clarity of the visual input image remains unchanged, when human participants subjectively perceive faces, 1) the amplitude of face-specific neural activity is higher; 2) face-specific neural activity exhibits a more consistent response pattern; 3) the transmission strength of face-specific neural activity from the posterior to anterior temporal lobe cortex is higher; 4) the peak time of face-specific neural activity curve is significantly correlated with the response time of participants, and it consistently precedes the participants' response time by approximately 200 milliseconds. These results suggest that local face-specific neural activity in the ventral temporal lobe cortex of the human brain is related to conscious face detection.

(3) Face detection and face recognition are two distinct stages of face processing. Due to the abundant face-related visual experiences humans encounter in daily life, some studies propose that humans are experts in face processing, suggesting that face recognition and detection might occur simultaneously. Conversely, views conflicting with this "face-expertise" perspective suggest that face processing follows a coarse-to-fine processing strategy, implying that face detection precedes finer face recognition. In this study, to investigate the processing sequence of these two stages in the human brain, as well as potential differences in representation formats at different times within the same processing stage, eight human participants viewed clear face images and non-face (house) images, field potential signals from the ventral temporal lobe cortex of the human brain were collected. Then the representational similarity analysis was used to investigate the temporal dynamics of face detection and face recognition in the human brain and the artificial neural networks. The results indicate that: 1) Face-specific neural activity in the ventral temporal lobe cortex exhibits category-specific representation between 170 to 390 milliseconds, possibly involved in face detection, and identity-specific representation between 280 to 390 milliseconds, possibly involved in face recognition; 2) Neural representations of face detection show dynamics between 130 to 200 milliseconds, while neural representations of face recognition remain stable throughout the trial; 3) Artificial neural networks exhibit the same facial processing sequence as the human brain. These results suggest that the human brain employs a "detection-first, recognition-later" face processing strategy, and neural representations of face category information in early periods change over time, and this processing strategy may be formed by computational optimization.

(4) Faces contain both identity information and social trait information. In this study, to investigate the influence of face identity information on face social trait information, twelve human participants viewed clear face images, single-unit neuronal discharges from the medial temporal lobe (amygdala and hippocampus) of the human brain were collected. Then the similarity of neural representations of face social traits between convolutional neural networks with different visual experiences and the medial temporal lobe neurons was computed. The results indicate that: 1) Convolutional neural networks with face identity recognition training experience (pre-trained VGG-Face) exhibit brain-like neural representations of social traits, including social trait coupling effects and social trait confusion effects; 2) Convolutional neural networks with the same architecture but without face identity recognition visual experience (VGG-16 and untrained VGG-Face) do not show any brain-like effect; 3) On all individual social trait representations, the similarity of neural representations between pre-trained VGG-Face and the human medial temporal lobe is higher compared to VGG-16 and untrained VGG-Face. These results suggest that brain-like neural representations of social traits require visual experience in face identity recognition, revealing the necessity of face identity recognition experience for developing the judgment of face social traits.

In summary, this article addresses four research questions revolving around three stages of face processing: face detection, face identity recognition, and face social trait judgement. Utilizing intracranial electrophysiological signals from the human temporal lobe, alongside traditional EEG signal analysis methods, machine learning techniques, and deep neural networks, it investigates the neural response characteristics of individual face processing stage and the interaction across these different stages. This provides electrophysiological support evidence for revealing the neural mechanisms of face processing, and provides new ideas for using deep neural networks to solve problems in the field of cognitive neuroscience.

Keyword面孔检测 面孔身份识别 面孔社会特征判断 颅内电生理记录 人脑颞叶
Language中文
Document Type学位论文
Identifierhttp://ir.ia.ac.cn/handle/173211/56559
Collection毕业生_博士学位论文
Recommended Citation
GB/T 7714
李文路. 基于颅内电生理记录的面孔加工脑活动研究[D],2024.
Files in This Item:
File Name/Size DocType Version Access License
博士学位论文_李文路.pdf(5800KB)学位论文 限制开放CC BY-NC-SA
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[李文路]'s Articles
Baidu academic
Similar articles in Baidu academic
[李文路]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[李文路]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.