中国科学院自动化研究所机构知识库

Decoding human brain activities based on linguistic representations
has been actively studied in recent years. However,
most previous studies exclusively focus on word-level representations,
and little is learned about decoding whole sentences
from brain activation patterns. This work is our effort
to mend the gap. In this paper, we build decoders to associate
brain activities with sentence stimulus via distributed
representations, the currently dominant sentence representation
approach in natural language processing (NLP).We carry
out a systematic evaluation, covering both widely-used baselines
and state-of-the-art sentence representation models. We
demonstrate how well different types of sentence representations
decode the brain activation patterns and give empirical
explanations of the performance difference. Moreover, to
explore how sentences are neurally represented in the brain,
we further compare the sentence representation’s correspondence
to different brain areas associated with high-level cognitive
functions. We find the supervised structured representation
models most accurately probe the language atlas of human
brain. To the best of our knowledge, this work is the first
comprehensive evaluation of distributed sentence representations
for brain decoding. We hope this work can contribute
to decoding brain activities with NLP representation models,
and understanding how linguistic items are neurally represented.