Knowledge Commons of Institute of Automation,CAS
Description-Enhanced Label Embedding Contrastive Learning for Text Classification | |
Zhang, Kun1; Wu, Le1; Lv, Guangyi2; Chen, Enhong3; Ruan, Shulan3; Liu, Jing4; Zhang, Zhiqiang5; Zhou, Jun5; Wang, Meng1 | |
发表期刊 | IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS |
ISSN | 2162-237X |
2023-06-16 | |
页码 | 14 |
通讯作者 | Wu, Le(lewu.ustc@gmail.com) |
摘要 | Text Classification is one of the fundamental tasks in natural language processing, which requires an agent to determine the most appropriate category for input sentences. Recently, deep neural networks have achieved impressive performance in this area, especially Pre-trained Language Models (PLMs). Usually, these methods concentrate on input sentences and corresponding semantic embedding generation. However, for another essential component: labels, most existing works either treat them as meaningless one-hot vectors or use vanilla embedding methods to learn label representations along with model training, underestimating the semantic information and guidance that these labels reveal. To alleviate this problem and better exploit label information, in this paper, we employ Self-Supervised Learning (SSL) in model learning process and design a novel self-supervised Relation of Relation (R2) classification task for label utilization from a one-hot manner perspective. Then, we propose a novel Relation of Relation Learning Network (R2-Net) for text classification, in which text classification and R2 classification are treated as optimization targets. Meanwhile, triplet loss is employed to enhance the analysis of differences and connections among labels. Moreover, considering that one-hot usage is still short of exploiting label information, we incorporate external knowledge from WordNet to obtain multi-aspect descriptions for label semantic learning and extend R2-Net to a novel Description-Enhanced Label Embedding network (DELE) from a label embedding perspective. ... |
关键词 | Contrastive learning (CL) label embedding representation learning text classification |
DOI | 10.1109/TNNLS.2023.3282020 |
关键词[WOS] | NETWORK |
收录类别 | SCI |
语种 | 英语 |
资助项目 | Young Scientists Fund of the National Natural Science Foundation of China[62006066] ; Young Scientists Fund of the National Natural Science Foundation of China[61727809] ; Young Scientists Fund of the National Natural Science Foundation of China[61922073] ; National Natural Science Foundation of China[62006066] ; National Natural Science Foundation of China[61727809] ; National Natural Science Foundation of China[61922073] ; joint Funds of the National Natural Science Foundation of China[61922073] ; Fundamental Research Funds for the Central Universities[61922073] ; National Natural Science Foundation of China ; Fundamental Research Funds for the Central Universities ; [72188101] ; [U22A2094] ; [JZ2021HGTB0075] |
项目资助者 | Young Scientists Fund of the National Natural Science Foundation of China ; National Natural Science Foundation of China ; joint Funds of the National Natural Science Foundation of China ; Fundamental Research Funds for the Central Universities ; National Natural Science Foundation of China ; Fundamental Research Funds for the Central Universities |
WOS研究方向 | Computer Science ; Engineering |
WOS类目 | Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic |
WOS记录号 | WOS:001019417800001 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/53592 |
专题 | 紫东太初大模型研究中心 |
通讯作者 | Wu, Le |
作者单位 | 1.Hefei Univ Technol, Sch Comp & Informat, Hefei 230029, Anhui, Peoples R China 2.Lenovo Res, AI Lab, Beijing 100094, Peoples R China 3.Univ Sci & Technol China, Sch Comp Sci & Technol, Hefei 230026, Peoples R China 4.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China 5.Ant Grp Co Ltd, Hangzhou 310007, Peoples R China |
推荐引用方式 GB/T 7714 | Zhang, Kun,Wu, Le,Lv, Guangyi,et al. Description-Enhanced Label Embedding Contrastive Learning for Text Classification[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2023:14. |
APA | Zhang, Kun.,Wu, Le.,Lv, Guangyi.,Chen, Enhong.,Ruan, Shulan.,...&Wang, Meng.(2023).Description-Enhanced Label Embedding Contrastive Learning for Text Classification.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,14. |
MLA | Zhang, Kun,et al."Description-Enhanced Label Embedding Contrastive Learning for Text Classification".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2023):14. |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论