Rehabilitation begins and ends with assessment. Assessment accompanies stroke patients’ rehabilitation cycle and is significant for their hand rehabilitation. However, there are still many shortcomings and limitations in assessing hand motor function: lack of further mining of patient’s data and poor accuracy of hand motor function assessment; the assessment of hand motor function depends on the observation of the therapist and is highly subjective; lack of quantitative assessment of the patient’s hand motor function; there is a “ceiling effect” in the scale, and it is impossible to distinguish the difference in rehabilitation grade of patients with better hand function. In response to the above problems, this paper focuses on the intelligent quantitative assessment of the hand motor function of stroke patients. The main research contents and innovations of this paper are as follows:
1. A patient data augmentation method based on empirical mode decomposition and integrated long short-term memory neural network (EMD-ILSTM) is proposed and validated in clinical trials. This method decomposes the 1-dimensional motor signal into a multi-dimensional signal matrix by EMD, and the whole multidimensional signal matrix is trained and augmented by the LSTM network. After that, the classification test is conducted on the public dataset of hand movements called “Ninaweb.” The test results show that the classification accuracy could be improved by 5.2% using the enhanced data in the classification task. In addition, clinical tests are conducted at Beijing Bo’ai Hospital, affiliated with the China Research and Rehabilitation Center. Through dimensionality reduction processing of the collected patient hand movements data, it is verified that the intra-class distance between the augmented data and the original data after dimensionality reduction is smaller, and the inter-class distance is larger, which proves the effectiveness of data augmentation.
2. A semi-quantitative assessment method of hand motor function based on ensemble learning is proposed and validated in clinical trials. Firstly, a method of hand kinematics analysis based on LSTM is proposed to determine the clinical trial paradigm. Twenty hand movements in the public data set are divided into four classes by hierarchical clustering, and five cluster center movements are determined. Secondly, a data acquisition system based on Leap Motion is designed, and a clinical trial is carried out based on the system to collect the hand movement data of 50 stroke patients. Finally, the ensemble learning method based on ensemble subspace is used to classify patients. The effectiveness of the proposed method is verified by comparing the classification accuracy with other traditional classification methods (W-kNN, Decision Tree, SVM) and utilizing two non-parametric statistical indexes, Wilcoxon signed rank test
and Kappa test.
3. A quantitative assessment method of hand motor function based on the Gaussian mixture model is proposed and validated in clinical trials. Firstly, according to various hand function assessment scales and the clinical experience of multiple therapists, a hand function assessment paradigm and a multi-modality data acquisition system based on data gloves and sEMG bracelets is designed. Based on the system, 35 patient subjects and 35 healthy subjects’ hand movements and sEMG data are collected, and the data sets of stroke patients and healthy subjects are established. Then, 11 kinds of feature values are extracted manually according to the movement characteristics of the activities, and the feature combination is carried out. Finally, the qualitative assessment of hand motor function based on the Gaussian mixture model is completed, and the effectiveness of the assessment method is proved by the Spearman coefficient.
4. A multi-modality hand motion fine assessment framework based on HAGCN (Hand Motor Function assessment Graph Network) is proposed and validated in clinical trials. First of all, the HAGCN network structure is proposed, which not only analyzes the structure of the spatial structure and the attention mechanism of the joint nodes of the hands in the spatial domain but also designs the convolution calculation algorithm in time. Then, the sEMG data is processed by the LSTM network, and the features
of sEMG are extracted automatically. Then, this framework proposes a weighted fusion method to assess the hand motor function. Finally, with the help of the clinical trial data, the validity of the assessment framework is verified by using the Spearman's coefficient. This framework can not only complete fine assessment but also avoid the “ceiling effect” of some traditional scales.
Key Words: Quantitative Assessment of Hand Motor Function, Kinematic Analysis, Data Augmentation, Gaussian Mixture Model, Graph Convolutional Network