毕业生知识库

步态识别是生物特征识别领域重要任务之一，其应用十分广泛，特别是远距离非配合场景的智能监控系统。随着近年来深度学习的不断发展，步态识别的精度进一步提高，并取得了一定应用。目前学术上的步态识别通常仅有单人且背景简单，步态序列收集较为容易。而实际应用中，通常为多人且背景复杂，识别精度难以保证。针对这些问题，我们研究适合步态识别的多目标跟踪，用来收集多人条件下的步态序列；研究判别式步态时空特征学习，用来提升步态识别精度；研究步态软生物特征用来辅助步态识别，进一步提升精度。通过整合各个模块，实现高效鲁棒的步态分析相关应用。因此，本文可以认为是更实用的基于视频的步态分析与识别关键问题研究。具体工作如下：
• 为了在多人条件下高效地获得每个人的步态轮廓序列，我们提出多任务在线多目标跟踪算法。我们探索了检测和表观特征这两个影响多目标跟踪的因素，认为检测对于多目标跟踪的性能影响很大，表观特征能够帮助数据关联。使用多任务学习方式来共同提高这两方面因素：通过检测框质量评估模块，过滤低质量检测，进而提高跟踪精度；使用三元组损失函数学习更好的表观特征，比以往基于孪生网络的多目标跟踪方法有效率优势，仅需计算表观特征间的相似度而不必两两组对由孪生网络输出相似度；通过多任务学习的方式，将两个任务结合在一起，既保证了速度，又能获得不错的性能。我们的方法在先进视频信号监控会议多目标跟踪比赛中获得了第 3 名的成绩，证明了该方法的有效性。
• 为了更好地提取步态特征，我们提出基于判别式时空特征学习的跨视角步态识别算法。借鉴人脸识别中损失函数的经验，提出一种更符合跨视角步态识别定义的损失函数。该损失函数能学到更好的类内聚合特征，而且对于训练时的视角样本不均衡问题也有帮助。时空特征学习包括可学习身体部件分块方法，能够学到对步态识别更有益的部件特征，以及时序注意力机制，使网络更关注于对识别有帮助的视频帧。时空特征提取方法能够获得更精细的局部步态特征，较以往基于全局输入或者时序平均池化效果好。
• 为了辅助步态识别系统，我们研究了步态软生物特征，提出了一种融合步态识别和步态软生物特征的联合学习框架。对性别、年龄、身体形态等步态软生物特征进行联合研究，发现这些软生物特征对于场景、视角、行走状态不敏感。通过软生物特征与步态识别的融合，发现两个任务都有一定提升。而且软生物特征能够缩小步态识别的搜索范围，或者在非身份识别场合提供一些语义描述，具有实际使用意义。同时，我们还探讨了计算效率，即使在嵌入式设备 TX2 上也能高效运行，这进一步证明了该方法的实用性。
• 为了步态分析系统能够真正落地，我们整合各个环节，实现完整的步态分析实际应用：集群端嫌疑人检索系统以及嵌入式端家庭成员身份识别系统。对于嫌疑人检索，使用融合表观特征的在线多目标跟踪算法以及多帧步态识别算法；对于家庭成员身份识别，考虑到嵌入式设备计算能力弱的情况，使用仅基于交并比的在线多目标跟踪算法以及少数帧识别算法。相关应用受到了各地公安和智能家居厂商的好评。

Gait recognition is an important task in biometric recognition. It has many applications, especially in long-distance non-cooperative intelligence surveillance systems. With the development of deep learning, the accuracy of gait recognition has been improved and there has been some applications. Academic gait recognition often studies the cases with only one person in simple background. The gait silhouette sequences could be easily collected. However in real applications with multiple people in complex background, the accuracy could not be guaranteed. Considering these problems, we study gait-suited multiple object tracking to collect gait sequences when there are multiple people. We study discriminative spatial-temporal gait feature extraction to improve gait recognition accuracy. We study gait-based soft biometrics to assist gait recognition systems. By combing these steps, we build efficient and robust gait analysis applications. The thesis can thus be regarded as the research on key problems of more practical video-based gait analysis and recognition. Our detailed contributions are listed below:
• We propose a multi-task online multiple object tracking algorithm to capture gait sequences of each person efficiently. We study the effect of object detection and appearance features. We find that object detection plays an important role and appearance features could help data association. We use multi-task learning methods to improve the two factors at the same time. With the proposed quality-aware network, we can filter out detections with low quality and thus improve tracking performance. We learn better appearance features by triplet loss, which has efficiency advantages over Siamese networks in previous works. We only need to compute similarities between the appearance features rather than making pairs for Siamese networks to output the similarities. We achieve efficiency and accuracy by jointly learning these two tasks. Our method achieved 3rd on Advanced Video and Signal Based Surveillance Experienced Multiple Object Tracking Competition, which proves its effectiveness.
• We propose a discriminative spatial-temporal feature learning method to better extract gait features. Inspired by face recognition, a new loss function is proposed to better match the definition of cross-view gait recognition. The loss function can learn better intra-class compactness and it is also beneficial with unbalanced view data at training time. The spatial-temporal feature learning includes a learned body part partition method to extract informative local gait features, and temporal attention mechanism to help the network focus more on those informative frames. The spatial-temporal feature learning methods could extract fine-grained gait features, and have achieved better performance than global input or temporal mean average pooling.
• We study gait-based soft biometrics to assist gait recognition systems. We propose a joint framework to learn both gait recognition and gait-based soft biometrics at the same time. We study gander, age and body shape jointly and find that these soft biometrics are not sensitive to scenes, views or walking conditions. Gait recognition and gait-based soft biometrics both benefit from joint learning of the two tasks. Soft biometrics may also help to reduce the search space of gait recognition or to provide semantic descriptions for some non-identification cases. This shows the feasibility of soft biometrics for real cases. We also study the efficiency at the same time. Our method could be executed on embedded devices such as TX2, which further proves the practicability.
• We combine the steps of gait analysis to build practical systems, namely: a server-based suspect retrieval system and an embedded family member recognition system. For suspect retrieval, we use appearance-aided multiple object tracking and multiframe gait recognition. For family member recognition, we have to use intersectionover-union-based multiple object tracking and limited-frame recognition because of low
computation capacity. The applications are highly praised by the police and smart home manufacturers.