Knowledge Commons of Institute of Automation，CAS
| Temporal-adaptive sparse feature aggregation for video object detection | |
He, Fei1,2 ; Li, Qiaozhe1,2; Zhao, Xin1,2; Huang, Kaiqi1,2,3
| |
| Source Publication | PATTERN RECOGNITION
 |
| ISSN | 0031-3203 |
| 2022-07-01 | |
| Volume | 127Pages:10 |
| Corresponding Author | Zhao, Xin(xzhao@nlpr.ia.ac.cn) |
| Abstract | Video object detection is a challenging task due to the appearance deterioration in video frames. To enhance feature representation of the deteriorated frames, previous methods usually aggregate features from fixed-density and fixed-length adjacent frames. However, due to the redundancy of videos and irregular object movements over time, temporal information may not be efficiently exploited using the traditional inflexible strategy. Alternatively, we present a temporal-adaptive sparse feature aggregation framework, an accurate and efficient method for video object detection. Instead of adopting a fixed-density and fixed-length window fusion strategy, a temporal-adaptive sparse sampling strategy is proposed using a stride predictor to encode informative frames more efficiently. A collaborative feature aggregation framework, which consists of a pixel-adaptive aggregation module and an object-relational aggregation module, is proposed for feature enhancement. The pixel-adaptive aggregation module enhances pixel level features on the current frame using corresponding pixel-level features from other frames. Similarly, the object-relational aggregation module further enhances feature representation at proposal level. A graph is constructed to model the relations between different proposals so that the relation features and proposal features are adaptively fused for feature enhancement. Experiments demonstrate that our proposed framework significantly surpasses traditional dense aggregation methods, and comprehensive ablation studies verify the effectiveness of each proposed module in our framework. |
| Keyword | Video object detection Temporal-adaptive sparse sampling Pixel-adaptive aggregation Object-relational aggregation |
| DOI | 10.1016/j.patcog.2022.108587 |
| Indexed By | SCI |
| Language | 英语 |
| Funding Project | National Natural Science Foundation of China[61721004] ; National Natural Science Foundation of China[61876181] ; Projects of Chinese Academy of Science[QYZDB-SSW-JSC006] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDA27000000] ; Youth Innovation Promotion Association CAS |
| Funding Organization | National Natural Science Foundation of China ; Projects of Chinese Academy of Science ; Strategic Priority Research Program of Chinese Academy of Sciences ; Youth Innovation Promotion Association CAS |
| WOS Research Area | Computer Science ; Engineering |
| WOS Subject | Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic |
| WOS ID | WOS:000776971700003 |
| Publisher | ELSEVIER SCI LTD |
| Sub direction classification | 图像视频处理与分析 |
| Citation statistics | |
| Document Type | 期刊论文 |
| Identifier | http://ir.ia.ac.cn/handle/173211/48302 |
| Collection | 智能系统与工程 |
| Corresponding Author | Zhao, Xin |
| Affiliation | 1.Chinese Acad Sci, Inst Automat, CRISE, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China 3.CAS Ctr Excellence Brain Sci & Intelligence Techn, Shanghai, Peoples R China |
| First Author Affilication | Institute of Automation, Chinese Academy of Sciences |
| Corresponding Author Affilication | Institute of Automation, Chinese Academy of Sciences |
| Recommended Citation GB/T 7714 | He, Fei,Li, Qiaozhe,Zhao, Xin,et al. Temporal-adaptive sparse feature aggregation for video object detection[J]. PATTERN RECOGNITION,2022,127:10. |
| APA | He, Fei,Li, Qiaozhe,Zhao, Xin,&Huang, Kaiqi.(2022).Temporal-adaptive sparse feature aggregation for video object detection.PATTERN RECOGNITION,127,10. |
| MLA | He, Fei,et al."Temporal-adaptive sparse feature aggregation for video object detection".PATTERN RECOGNITION 127(2022):10. |
| Files in This Item: | There are no files associated with this item. | |||||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment