模式识别国家重点实验室知识库

Action recognition plays a fundamental role in computer vision and video analysis. Nevertheless, extracting effective spatial-temporal features remains a challenging task. This paper proposes three simple but effective weakly Semantic Guided Modules (SGMs) for both environment constrained and cross domain action recognition. The SGMs are composed of totally 3D convolution and element-wise gated operations, thus they are efficient and easy to be implemented. The semantic guidance is obtained in a weakly supervised manner, in which each video clip is labeled with only an action class instead of pixel level semantics. Benefitting from the semantic guidance, the network (named SGN, abbreviation for Semantic Guided Network) can focus on the salient parts of the video clips. Consequently, the redundant information can be reduced and the model is more robust to noises. Besides, benefitting from the intrinsic property of SGMs, SGN is totally end-to-end trainable. Quantities of experiments on both environment constrained (e.g., Penn, HMDB-51 and UCF-101) and cross domain (e.g., ODAR) action recognition datasets demonstrate its effectiveness. Specifically, SGN gets improvements of 3:7%, 2:1% and 5:2% for Penn, HMDB-51 and UCF-101 than the baseline ResNet3D, respectively, and SGN ranked the 3rd place in ODAR 2017 challenge.