Knowledge Commons of Institute of Automation,CAS
Sharing deep generative representation for perceived image reconstruction from human brain activity | |
Du Changde1,3; Du Changying2; He Huiguang1,3,4 | |
2017 | |
会议名称 | International Joint Conference on Neural Networks |
会议日期 | May 14, 2017 - May 19, 2017 |
会议地点 | Anchorage, Alaska |
摘要 | Decoding human brain activities via functional magnetic resonance imaging (fMRI) has gained increasing attention in recent years. While encouraging results have been reported in brain states classification tasks, reconstructing the details of human visual experience still remains difficult. Two main challenges that hinder the development of effective models are the perplexing fMRI measurement noise and the high dimensionality of limited data instances. Existing methods generally suffer from one or both of these issues and yield dissatisfactory results. In this paper, we tackle this problem by casting the reconstruction of visual stimulus as the Bayesian inference of missing view in a multiview latent variable model. Sharing a common latent representation, our joint generative model of external stimulus and brain response is not only ``deep" in extracting nonlinear features from visual images, but also powerful in capturing correlations among voxel activities of fMRI recordings. The nonlinearity and deep structure endow our model with strong representation ability, while the correlations of voxel activities are critical for suppressing noise and improving prediction. We devise an efficient variational Bayesian method to infer the latent variables and the model parameters. To further improve the reconstruction accuracy, the latent representations of testing instances are enforced to be close to that of their neighbours from the training set via posterior regularization. Experiments on three fMRI recording datasets demonstrate that our approach can more accurately reconstruct visual stimuli. |
语种 | 英语 |
文献类型 | 会议论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/23607 |
专题 | 脑图谱与类脑智能实验室_神经计算与脑机交互 |
通讯作者 | He Huiguang |
作者单位 | 1.Research Center for Brain-Inspired Intelligence, Institute of Automation, Chinese Academy of Sciences 2.Laboratory of Parallel Software and Computational Science, Institute of Software, Chinese Academy of Sciences 3.University of Chinese Academy of Sciences 4.Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences |
推荐引用方式 GB/T 7714 | Du Changde,Du Changying,He Huiguang. Sharing deep generative representation for perceived image reconstruction from human brain activity[C],2017. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
IJCNN_2017_Sharing D(531KB) | 会议论文 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论