中国科学院自动化研究所机构知识库

This paper provides a dual-mode real-time lip-sync system for a bionic dinosaur robot. Different from traditional mono-modality control systems, our system is constructed with different controllers and classifiers in both time domain and frequency domain. Specially, a classifier in time domain is designed to extract the sound features including pitch and intensity. Meanwhile, a nonlinear mapping relationship between time-domain feature parameters and mouth open angles is particularly designed. In time domain, an efficient algorithm consisting of original and modified short-term average amplitude difference function (AMDF) is applied for frequency measurement. With the goal of predicting the curve of mouth open, we train the audio data of dinosaurs to get a frequency classifier by Support Vector Machine with racial basis function (RBF-SVM), which has a relatively high accuracy. Finally, extensive experiments validate the effectiveness of this proposed system on a real bionic dinosaur robot.