CASIA OpenIR  > 学术期刊  > Machine Intelligence Research
The Life Cycle of Knowledge in Big Language Models: A Survey
Boxi Cao1,3; Hongyu Lin1; Xianpei Han1,2; Le Sun1,2
Source PublicationMachine Intelligence Research
ISSN2731-538X
2024
Volume21Issue:2Pages:217-238
Abstract

Knowledge plays a critical role in artificial intelligence. Recently, the extensive success of pre-trained language models (PLMs) has raised significant attention about how knowledge can be acquired, maintained, updated and used by language models. Despite the enormous amount of related studies, there is still a lack of a unified view of how knowledge circulates within language models throughout the learning, tuning, and application processes, which may prevent us from further understanding the connections between current progress or realizing existing limitations. In this survey, we revisit PLMs as knowledge-based systems by dividing the life circle of knowledge in PLMs into five critical periods, and investigating how knowledge circulates when it is built, maintained and used. To this end, we systematically review existing studies of each period of the knowledge life cycle, summarize the main challenges and current limitations, and discuss future directions.

KeywordPre-trained language model, knowledge acquisition, knowledge representation, knowledge probing, knowledge editing, knowledge application
DOI10.1007/s11633-023-1416-x
Sub direction classification其他
planning direction of the national heavy laboratory其他
Paper associated data
Chinese guidehttps://mp.weixin.qq.com/s/rKHHYu_CrXvFrcUSBqOf8w
Citation statistics
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/56035
Collection学术期刊_Machine Intelligence Research
Affiliation1.Chinese Information Processing Laboratory, Beijing 100190, China
2.State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China
3.University of Chinese Academy of Sciences, Beijing 101408, China
Recommended Citation
GB/T 7714
Boxi Cao,Hongyu Lin,Xianpei Han,et al. The Life Cycle of Knowledge in Big Language Models: A Survey[J]. Machine Intelligence Research,2024,21(2):217-238.
APA Boxi Cao,Hongyu Lin,Xianpei Han,&Le Sun.(2024).The Life Cycle of Knowledge in Big Language Models: A Survey.Machine Intelligence Research,21(2),217-238.
MLA Boxi Cao,et al."The Life Cycle of Knowledge in Big Language Models: A Survey".Machine Intelligence Research 21.2(2024):217-238.
Files in This Item: Download All
File Name/Size DocType Version Access License
MIR-2022-10-329.pdf(1430KB)期刊论文出版稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Boxi Cao]'s Articles
[Hongyu Lin]'s Articles
[Xianpei Han]'s Articles
Baidu academic
Similar articles in Baidu academic
[Boxi Cao]'s Articles
[Hongyu Lin]'s Articles
[Xianpei Han]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Boxi Cao]'s Articles
[Hongyu Lin]'s Articles
[Xianpei Han]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: MIR-2022-10-329.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.