CASIA OpenIR  > 类脑智能研究中心  > 神经计算及脑机交互
Reconstructing Perceived Images from Human Brain Activities with Bayesian Deep Multi-view Learning
Du Changde1,2; Du Changying3; Huang Lijie1; He Huiguang1,2,4
Source PublicationIEEE Transactions on Neural Networks and Learning Systems
2018
Issue0Pages:1-14
Abstract

Neural decoding, which aims to predict external visual stimuli information from evoked brain activities, plays an important role in understanding human visual system. Many existing methods are based on linear models, and most of them only focus on either the brain activity pattern classification or visual stimuli identification. Accurate reconstruction of the perceived images from the measured human brain activities still remains challenging. In this paper, we propose a novel deep generative multi-view model (DGMM) for the accurate visual image reconstruction from the human brain activities  measured by functional magnetic resonance imaging (fMRI). Specifically, we model the statistical relationships between two views (i.e., the visual stimuli and the evoked fMRI) by using two view-specific generators with a shared latent space. On the one hand, we adopt a deep neural network architecture for visual image generation, which mimics the stages of human visual processing. On the other hand, we design a sparse Bayesian linear model for fMRI activity generation, which can effectively capture voxel correlations, suppress data noise and avoid overfitting. Furthermore, we devise an efficient mean-field variational inference method to train the proposed model. The proposed method can accurately reconstruct visual images via Bayesian inference. In particular, we exploit a posterior regularization technique in the Bayesian inference to regularize the model posterior. The quantitative and qualitative evaluations conducted on multiple fMRI datasets demonstrate the proposed method can reconstruct visual images more accurately than the state-of-the-art.

KeywordDeep Neural Network Multi-view Learning Variational Bayesian Inference Neural Decoding Image Reconstruction
Language英语
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/23609
Collection类脑智能研究中心_神经计算及脑机交互
Corresponding AuthorHe Huiguang
Affiliation1.Research Center for Brain-Inspired Intelligence, Institute of Automation, Chinese Academy of Sciences (CAS), Beijing 100190, China
2.Center for Excellence in Brain Science and Intelligence Technology, CAS, Shanghai 200031, China
3.University of Chinese Academy of Sciences, Beijing 100049, China
4.Laboratory of Parallel Software and Computational Science, Institute of Software, CAS, Beijing 100190, China
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Du Changde,Du Changying,Huang Lijie,et al. Reconstructing Perceived Images from Human Brain Activities with Bayesian Deep Multi-view Learning[J]. IEEE Transactions on Neural Networks and Learning Systems,2018(0):1-14.
APA Du Changde,Du Changying,Huang Lijie,&He Huiguang.(2018).Reconstructing Perceived Images from Human Brain Activities with Bayesian Deep Multi-view Learning.IEEE Transactions on Neural Networks and Learning Systems(0),1-14.
MLA Du Changde,et al."Reconstructing Perceived Images from Human Brain Activities with Bayesian Deep Multi-view Learning".IEEE Transactions on Neural Networks and Learning Systems .0(2018):1-14.
Files in This Item: Download All
File Name/Size DocType Version Access License
TNNLS_2018_Reconstru(3843KB)期刊论文出版稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Du Changde]'s Articles
[Du Changying]'s Articles
[Huang Lijie]'s Articles
Baidu academic
Similar articles in Baidu academic
[Du Changde]'s Articles
[Du Changying]'s Articles
[Huang Lijie]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Du Changde]'s Articles
[Du Changying]'s Articles
[Huang Lijie]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: TNNLS_2018_Reconstructing Perceived Images From Human Brain Activities With Bayesian Deep Multiview Learning.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.