Knowledge Commons of Institute of Automation,CAS
Temporal Action Proposal Generation With Action Frequency Adaptive Network | |
Yepeng Tang2; Weining Wang1; Chunjie Zhang2; Jing Liu1; Yao Zhao2 | |
发表期刊 | IEEE Transactions on Multimedia |
ISSN | 1520-9210 |
2023-07 | |
卷号 | 26页码:2340 - 2353 |
通讯作者 | Zhang, Chunjie(cjzhang@bjtu.edu.cn) |
文章类型 | SCI |
摘要 | As the cornerstone of human-behavior analysis in video understanding, temporal action proposal generation aims to predict the starting and ending time of human action instances in untrimmed videos. Although large achievements in temporal action proposal generation have been achieved, most previous studies ignore the variability of action frequency in raw videos, leading to unsatisfying performances on high-action-frequency videos. In fact, there exists two main issues which should be well addressed: data imbalance between high and low action-frequency videos, and inferior detection of short actions in high-action-frequency videos. To address the above issues, we propose an effective framework by adapting to the variability of action frequency, namely Action Frequency Adaptive Network (AFAN), which can be flexibly built upon any temporal action proposal generation method. AFAN consists of two modules: Learning From Experts (LFE) and Fine-Grained Processing (FGP). The LFE first trains a series of action proposal generators on different subsets of imbalanced data as experts and then teaches a unified student model via knowledge distillation. To better detect short actions, FGP first finds out high-action-frequency videos and then performs fine-grained detection. Extensive experimental results on four benchmark datasets (ActivityNet-1.3, HACS, THUMOS14 and FineAction) demonstrate the effectiveness and generalizability of the proposed AFAN, especially for high-action-frequency videos. |
关键词 | Proposals Task analysis Data models Time-frequency analysis Representation learning Predictive models Information science Temporal action proposal generation expert learning fine-gained detection action frequency |
DOI | 10.1109/TMM.2023.3295090 |
关键词[WOS] | IMBALANCED DATA ; SMOTE |
收录类别 | SCI |
语种 | 英语 |
资助项目 | Institute of Automation, Chinese Academy of Sciences |
项目资助者 | Institute of Automation, Chinese Academy of Sciences |
WOS研究方向 | Computer Science ; Telecommunications |
WOS类目 | Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications |
WOS记录号 | WOS:001168330100010 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
七大方向——子方向分类 | 多模态智能 |
国重实验室规划方向分类 | 多模态协同认知 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/55386 |
专题 | 紫东太初大模型研究中心 |
作者单位 | 1.Institute of Automation, Chinese Academy of Sciences 2.h The Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing Jiaotong University |
推荐引用方式 GB/T 7714 | Yepeng Tang,Weining Wang,Chunjie Zhang,et al. Temporal Action Proposal Generation With Action Frequency Adaptive Network[J]. IEEE Transactions on Multimedia,2023,26:2340 - 2353. |
APA | Yepeng Tang,Weining Wang,Chunjie Zhang,Jing Liu,&Yao Zhao.(2023).Temporal Action Proposal Generation With Action Frequency Adaptive Network.IEEE Transactions on Multimedia,26,2340 - 2353. |
MLA | Yepeng Tang,et al."Temporal Action Proposal Generation With Action Frequency Adaptive Network".IEEE Transactions on Multimedia 26(2023):2340 - 2353. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
[TMM.2023] Temporal_(10095KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论