Human pose estimation is a fundamental technique for natural human-computer interaction, which allows people to control the computers using their own cognition and perceptions and to interact with others naturally in the virtual world.It is also the technical foundation of intelligent human-computer interaction, because computers can catch semantic information of user motion and make a smart response by recognizing and analysing human poses. However, most of current human motion capture systems are marker-based or in need of wearable sensors, which makes them invasive and hard to popularize. For natural interaction, marker-less human pose capture is the trend of development.
Therefore, the paper is focused on marker-less 3D human pose estimation for interactive applications, including recovering human joint rotation angles and estimating 3D human pose from image sequence. On the basis of the researches, the paper implements a interactive control system for virtual character.
The main contributions of the paper are as follows:
1. A method of recovering human joint angles from human joint positions. Human joint position is a sparse description of human pose and contains limited information. In many applications, human joint positions can not fully represent human poses and other information is needed including human joint angles. To solve this problem, this paper propose a method of recovering human joint angles from human joint positions. The proposed method synthesizes training datasets of corresponding human joint positions and joint angles under different human poses, based on reconstructed human pose models from motion capture datasets. Then it constructs residual network to learn the hidden prior relation between joint positions and joint angles. The learned network can recover natural and valid poses from sparse human joint positions, which provides an approach to lifting the results of existing methods for joint position estimation and to fusing human pose datasets with different formats of annotations.
2. A method of human pose estimation from video sequences. This paper decompose the initial task into two parts, estimation human pose estimation from monocular images and estimating the change of human poses between two frames. Two models are constructed for the two sub-problems respectively and then combined into one running framework by applying some conditions. A loss of batch pose differences is introduced for the model based on monocular images. It improves the accuracy of the output poses. For estimating pose changes between image frames, the frame-to-frame difference and pose relevance are introduced into the model, which constrain the degree of pose changes so that the model is able to predict the pose changes between two consecutive frames accurately.
3. An instance of applying human pose estimation technique to virtual character interactive scenarios. A real-time interactive system for character motion control is built based on multi-view kinect human skeleton data and the methods proposed above. The motions of characters are driven by captured human poses. Users can control virtual characters with their own actions in real-time.