CASIA OpenIR  > 紫东太初大模型研究中心
Temporal Action Proposal Generation With Action Frequency Adaptive Network
Yepeng Tang2; Weining Wang1; Chunjie Zhang2; Jing Liu1; Yao Zhao2
Source PublicationIEEE Transactions on Multimedia
2023-07
Volume26Pages:2340 - 2353
SubtypeSCI
Abstract

As the cornerstone of human-behavior analysis in video understanding, temporal action proposal generation aims to predict the starting and ending time of human action instances in untrimmed videos. Although large achievements in temporal action proposal generation have been achieved, most previous studies ignore the variability of action frequency in raw videos, leading to unsatisfying performances on high-action-frequency videos. In fact, there exists two main issues which should be well addressed: data imbalance between high and low action-frequency videos, and inferior detection of short actions in high-action-frequency videos. To address the above issues, we propose an effective framework by adapting to the variability of action frequency, namely Action Frequency Adaptive Network (AFAN), which can be flexibly built upon any temporal action proposal generation method. AFAN consists of two modules: Learning From Experts (LFE) and Fine-Grained Processing (FGP). The LFE first trains a series of action proposal generators on different subsets of imbalanced data as experts and then teaches a unified student model via knowledge distillation. To better detect short actions, FGP first finds out high-action-frequency videos and then performs fine-grained detection. Extensive experimental results on four benchmark datasets (ActivityNet-1.3, HACS, THUMOS14 and FineAction) demonstrate the effectiveness and generalizability of the proposed AFAN, especially for high-action-frequency videos.

Language英语
Sub direction classification多模态智能
planning direction of the national heavy laboratory多模态协同认知
Paper associated data
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/55386
Collection紫东太初大模型研究中心
Affiliation1.Institute of Automation, Chinese Academy of Sciences
2.h The Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing Jiaotong University
Recommended Citation
GB/T 7714
Yepeng Tang,Weining Wang,Chunjie Zhang,et al. Temporal Action Proposal Generation With Action Frequency Adaptive Network[J]. IEEE Transactions on Multimedia,2023,26:2340 - 2353.
APA Yepeng Tang,Weining Wang,Chunjie Zhang,Jing Liu,&Yao Zhao.(2023).Temporal Action Proposal Generation With Action Frequency Adaptive Network.IEEE Transactions on Multimedia,26,2340 - 2353.
MLA Yepeng Tang,et al."Temporal Action Proposal Generation With Action Frequency Adaptive Network".IEEE Transactions on Multimedia 26(2023):2340 - 2353.
Files in This Item: Download All
File Name/Size DocType Version Access License
[TMM.2023] Temporal_(10095KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Yepeng Tang]'s Articles
[Weining Wang]'s Articles
[Chunjie Zhang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Yepeng Tang]'s Articles
[Weining Wang]'s Articles
[Chunjie Zhang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Yepeng Tang]'s Articles
[Weining Wang]'s Articles
[Chunjie Zhang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: [TMM.2023] Temporal_Action_Proposal_Generation_With_Action_Frequency_Adaptive_Network.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.