CASIA OpenIR  > 多模态人工智能系统全国重点实验室
Learning Proposal-Aware Re-Ranking for Weakly-Supervised Temporal Action Localization
Hu, Yufan1,2; Fu, Jie3; Chen, Mengyuan3; Gao, Junyu3; Dong, Jianfeng4; Fan, Bin1,2; Liu, Hongmin1,2
Source PublicationIEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
ISSN1051-8215
2024
Volume34Issue:1Pages:207-220
Corresponding AuthorLiu, Hongmin(hmliu_82@163.com)
AbstractWeakly-supervised temporal action localization (WTAL) aims to localize and classify action instances in untrimmed videos with only video-level labels available. Despite the remarkable success of existing methods, whose generated proposals are commonly far more than the ground-truth action instances, it still makes sense to improve the ranking accuracy of the generated proposals since users in real-world scenarios usually prioritize the action proposals with the highest confidence scores. The inaccuracy of the proposal ranking mainly comes from two aspects: For one thing, the traditional proposal generation manner entirely relies on snippet-level perception, resulting in a significant yet unnoticed gap with the target of proposal-level localization. For another, existing methods commonly employ a hand-crafted proposal generation manner, a post-process that does not participate in model optimization. To address the above issues, we propose an end-to-end trained two-stage method, termed as Learning Proposal-aware Re-ranking (LPR) for WTAL. In the first stage, we design a proposal-aware feature learning module to inject the proposal-aware contextual information into each snippet, and then the enhanced features are utilized for predicting initial proposals. Furthermore, to perform effective and efficient proposal re-ranking, in the second stage, we contrast the proposals attached with high confidence scores with our constructed multi-scale foreground/background prototypes for further optimization. Evaluated by both the vanilla and Top- $k$ mAP metrics, results of extensive experiments on two popular benchmarks demonstrate the effectiveness of our proposed method.
KeywordProposals Feature extraction Location awareness Videos Measurement Task analysis Optimization weakly-supervised temporal action localization Proposal-aware reranking
DOI10.1109/TCSVT.2023.3283430
WOS KeywordNETWORK ; VIDEO ; RETRIEVAL ; ATTENTION
Indexed BySCI
Language英语
Funding ProjectBeijing Natural Science Foundation
Funding OrganizationBeijing Natural Science Foundation
WOS Research AreaEngineering
WOS SubjectEngineering, Electrical & Electronic
WOS IDWOS:001138814400041
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Citation statistics
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/55513
Collection多模态人工智能系统全国重点实验室
Corresponding AuthorLiu, Hongmin
Affiliation1.Univ Sci & Technol Beijing, Key Lab Intelligent Bion Unmanned Syst, Minist Educ, Sch Intelligence Sci & Technol, Beijing 100083, Peoples R China
2.Univ Sci & Technol Beijing, Inst Artificial Intelligence, Beijing 100083, Peoples R China
3.Chinese Acad Sci, Inst Automat, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China
4.Zhejiang Gongshang Univ, Coll Comp & Informat Engn, Hangzhou 310018, Peoples R China
Recommended Citation
GB/T 7714
Hu, Yufan,Fu, Jie,Chen, Mengyuan,et al. Learning Proposal-Aware Re-Ranking for Weakly-Supervised Temporal Action Localization[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2024,34(1):207-220.
APA Hu, Yufan.,Fu, Jie.,Chen, Mengyuan.,Gao, Junyu.,Dong, Jianfeng.,...&Liu, Hongmin.(2024).Learning Proposal-Aware Re-Ranking for Weakly-Supervised Temporal Action Localization.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,34(1),207-220.
MLA Hu, Yufan,et al."Learning Proposal-Aware Re-Ranking for Weakly-Supervised Temporal Action Localization".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 34.1(2024):207-220.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Hu, Yufan]'s Articles
[Fu, Jie]'s Articles
[Chen, Mengyuan]'s Articles
Baidu academic
Similar articles in Baidu academic
[Hu, Yufan]'s Articles
[Fu, Jie]'s Articles
[Chen, Mengyuan]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Hu, Yufan]'s Articles
[Fu, Jie]'s Articles
[Chen, Mengyuan]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.