Knowledge Commons of Institute of Automation,CAS
Temporal-adaptive sparse feature aggregation for video object detection | |
He, Fei1,2; Li, Qiaozhe1,2; Zhao, Xin1,2; Huang, Kaiqi1,2,3 | |
发表期刊 | PATTERN RECOGNITION |
ISSN | 0031-3203 |
2022-07-01 | |
卷号 | 127页码:108587 |
摘要 | Video object detection is a challenging task due to the appearance deterioration in video frames. To enhance feature representation of the deteriorated frames, previous methods usually aggregate features from fixed-density and fixed-length adjacent frames. However, due to the redundancy of videos and irregular object movements over time, temporal information may not be efficiently exploited using the traditional inflexible strategy. Alternatively, we present a temporal-adaptive sparse feature aggregation framework, an accurate and efficient method for video object detection. Instead of adopting a fixed-density and fixed-length window fusion strategy, a temporal-adaptive sparse sampling strategy is proposed using a stride predictor to encode informative frames more efficiently. A collaborative feature aggregation framework, which consists of a pixel-adaptive aggregation module and an object-relational aggregation module, is proposed for feature enhancement. The pixel-adaptive aggregation module enhances pixel level features on the current frame using corresponding pixel-level features from other frames. Similarly, the object-relational aggregation module further enhances feature representation at proposal level. A graph is constructed to model the relations between different proposals so that the relation features and proposal features are adaptively fused for feature enhancement. Experiments demonstrate that our proposed framework significantly surpasses traditional dense aggregation methods, and comprehensive ablation studies verify the effectiveness of each proposed module in our framework. |
关键词 | Video object detection Temporal-adaptive sparse sampling Pixel-adaptive aggregation Object-relational aggregation |
DOI | 10.1016/j.patcog.2022.108587 |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Natural Science Foundation of China[61721004] ; National Natural Science Foundation of China[61876181] ; Projects of Chinese Academy of Science[QYZDB-SSW-JSC006] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDA27000000] ; Youth Innovation Promotion Association CAS |
项目资助者 | National Natural Science Foundation of China ; Projects of Chinese Academy of Science ; Strategic Priority Research Program of Chinese Academy of Sciences ; Youth Innovation Promotion Association CAS |
WOS研究方向 | Computer Science ; Engineering |
WOS类目 | Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic |
WOS记录号 | WOS:000776971700003 |
出版者 | ELSEVIER SCI LTD |
七大方向——子方向分类 | 图像视频处理与分析 |
国重实验室规划方向分类 | 视觉信息处理 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/48302 |
专题 | 复杂系统认知与决策实验室_智能系统与工程 |
通讯作者 | Zhao, Xin |
作者单位 | 1.Chinese Acad Sci, Inst Automat, CRISE, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China 3.CAS Ctr Excellence Brain Sci & Intelligence Techn, Shanghai, Peoples R China |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | He, Fei,Li, Qiaozhe,Zhao, Xin,et al. Temporal-adaptive sparse feature aggregation for video object detection[J]. PATTERN RECOGNITION,2022,127:108587. |
APA | He, Fei,Li, Qiaozhe,Zhao, Xin,&Huang, Kaiqi.(2022).Temporal-adaptive sparse feature aggregation for video object detection.PATTERN RECOGNITION,127,108587. |
MLA | He, Fei,et al."Temporal-adaptive sparse feature aggregation for video object detection".PATTERN RECOGNITION 127(2022):108587. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
1-s2.0-S003132032200(1549KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论