Text Difficulty Study: Do Machines Behave the Same as Humans Regarding Text Difficulty?

doi:10.1007/s11633-023-1424-x

CASIA OpenIR > 学术期刊 > Machine Intelligence Research

	Text Difficulty Study: Do Machines Behave the Same as Humans Regarding Text Difficulty?
	Bowen Chen 1; Xiao Ding 1; Yi Zhao 1; Bo Fu 2; Tingmao Lin 2; Bing Qin 1; Ting Liu1
发表期刊	Machine Intelligence Research
ISSN	2731-538X
	2024
卷号	21 期号:2 页码:283-293
摘要	With the emergence of pre-trained models, current neural networks are able to give task performance that is comparable to humans. However, we know little about the fundamental working mechanism of pre-trained models in which we do not know how they approach such performance and how the task is solved by the model. For example, given a task, human learns from easy to hard, whereas the model learns randomly. Undeniably, difficulty-insensitive learning leads to great success in natural language processing (NLP), but little attention has been paid to the effect of text difficulty in NLP. We propose a human learning matching index (HLM Index) to investigate the effect of text difficulty. Experiment results show: 1) LSTM gives more human-like learning behavior than BERT. Additionally, UID-SuperLinear gives the best evaluation of text difficulty among four text difficulty criteria. Among nine tasks, some tasks′ performance is related to text difficulty, whereas others are not. 2) Model trained on easy data performs best in both easy and medium test data, whereas trained on hard data only performs well on hard test data. 3) Train the model from easy to hard, leading to quicker convergence.
关键词	Cognition inspired natural language processing, psycholinguistics, explainability, text difficulty, curriculum learning
DOI	10.1007/s11633-023-1424-x
引用统计
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/56039
专题	学术期刊_Machine Intelligence Research
作者单位	1.Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology, Harbin 150001, China 2.Fundamental Technology Center, China Construction Bank Financial Technology Co., Ltd., Beijing 100033, China
推荐引用方式 GB/T 7714	Bowen Chen,Xiao Ding,Yi Zhao,et al. Text Difficulty Study: Do Machines Behave the Same as Humans Regarding Text Difficulty?[J]. Machine Intelligence Research,2024,21(2):283-293.
APA	Bowen Chen.,Xiao Ding.,Yi Zhao.,Bo Fu.,Tingmao Lin.,...&Ting Liu.(2024).Text Difficulty Study: Do Machines Behave the Same as Humans Regarding Text Difficulty?.Machine Intelligence Research,21(2),283-293.
MLA	Bowen Chen,et al."Text Difficulty Study: Do Machines Behave the Same as Humans Regarding Text Difficulty?".Machine Intelligence Research 21.2(2024):283-293.

条目包含的文件		下载所有文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
MIR-2022-11-338.pdf（1796KB）	期刊论文	出版稿	开放获取	CC BY-NC-SA	浏览下载