中国科学院自动化研究所机构知识库

Crowd attribute recognition is a challenging task for crowd video understanding because a crowd video often contains multiple attributes from various types. Traditional deep learning based methods directly treat this recognition problem as a multiple binary classification problem, and represent video by vectorizing and fusing the separately learned spatial and temporal features in the fully connected layers. Therefore, the correlations between these attributes may not be well captured. In this paper, a bidirectional recurrent prediction model with a semantic aware attention mechanism is proposed to explore the spatio-temporal and semantic relations between attributes for more accurate recognition. The ConvLSTM is introduced for feature representation to capture the spatio-temporal structure of crowd videos and facilitate visual attention. A bidirectional recurrent attention module is proposed for sequential attribute prediction by associating each subcategory attributes to corresponding semantic related regions iteratively. Experiments and evaluations on the challenging WWW crowd video dataset not only show that our approach significantly outperforms state-ofthe-art methods, but also verify that our approach can effectively capture the spatio-temporal and semantic relations of the crowd attributes.