CASIA OpenIR  > 类脑智能研究中心  > 神经计算及脑机交互
Sharing deep generative representation for perceived image reconstruction from human brain activity
Du Changde1,3; Du Changying2; He Huiguang1,3,4
Conference NameInternational Joint Conference on Neural Networks
Conference DateMay 14, 2017 - May 19, 2017
Conference PlaceAnchorage, Alaska

Decoding human brain activities via functional magnetic resonance imaging (fMRI) has gained increasing attention in recent years. While encouraging results have been reported in brain states classification tasks, reconstructing the details of human visual experience still remains difficult. Two main challenges that hinder the development of effective models are the perplexing fMRI measurement noise and the high dimensionality of limited data instances. Existing methods generally suffer from one or both of these issues and yield dissatisfactory results. In this paper, we tackle this problem by casting the reconstruction of visual stimulus as the Bayesian inference of missing view in a multiview latent variable model. Sharing a common latent  representation, our joint generative model of external stimulus and brain response is not only ``deep" in extracting nonlinear features from visual images, but also powerful in capturing correlations among voxel activities of fMRI recordings. The nonlinearity and deep structure endow our model with strong representation ability, while the correlations of voxel activities are critical for suppressing noise and improving prediction.  We devise an efficient variational Bayesian method to infer the latent variables and the model parameters. To further improve the reconstruction accuracy, the latent representations of testing instances are enforced to be close to that of their neighbours from the training set  via posterior regularization. Experiments on three fMRI recording datasets demonstrate that our approach can more accurately reconstruct visual stimuli.

Document Type会议论文
Corresponding AuthorHe Huiguang
Affiliation1.Research Center for Brain-Inspired Intelligence, Institute of Automation, Chinese Academy of Sciences
2.Laboratory of Parallel Software and Computational Science, Institute of Software, Chinese Academy of Sciences
3.University of Chinese Academy of Sciences
4.Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Du Changde,Du Changying,He Huiguang. Sharing deep generative representation for perceived image reconstruction from human brain activity[C],2017.
Files in This Item: Download All
File Name/Size DocType Version Access License
IJCNN_2017_Sharing D(531KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Du Changde]'s Articles
[Du Changying]'s Articles
[He Huiguang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Du Changde]'s Articles
[Du Changying]'s Articles
[He Huiguang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Du Changde]'s Articles
[Du Changying]'s Articles
[He Huiguang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: IJCNN_2017_Sharing Deep Generative Representation for Perceived Image Reconstruction from Human Brain Activity.pdf
Format: Adobe PDF
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.