The purpose of my thesis is to investigate the neural specificity of face perception and neural mechanism of emotional processing based on fMRI and EEG, respectively (1) We propose fMRI-based method to study the neural specificity of face perception. The Chinese characters were used as comparison stimuli because they are extremely similar to faces in terms of expertise and being processed at individual level. In the present study, stronger activities elicited by face relative to characters were observed in right middle fusiform, whose locus was in agreement with that of FFA. Such distinction of neural representation between faces and Chinese characters demonstrated that FFA was specialized in the processing of face per se rather than the processing of visual expertise and categorization at individual level. (2) We used priming effect to study the neural mechanism of overlapping regions of neural representations of faces and characters in fusiform. We found that in these regions two populations of neurons selectively responded to faces and characters, respectively and they were interdigitated. Because of the limitation of resolution of fMRI, it is difficult to distinguish them. (3) We proposed an EEG-based experimental scheme to determine the human brain response under the stimulation of standard emotion pictures. We found that the emotional processing engaged a complex neural circuit including brain structures associated with various functions such as emotion perception, cognitive processing, attention mediating, episodic memory retrieval and the interaction of these functions. These brain structures participated in emotional processing with discrete spatial distribution and independent temporal course. We also found that responses to positive and negative condition were associated with distinct spatiotemporal activation pattern, respectively. (4) We proposed an ICA-LORETA-based analysis strategy to investigate the single-trial EEG records of responses to emotional pictures. Our method has two advantages: First it reduces the complexity and improves the spatial resolution of LORETA. Second, it overcomes some shortcomings of averaging EEG record. This method can not only identify the spatial regions activated by emotional stimuli, but also reveal the temporally activated course corresponding to each region.