Knowledge Commons of Institute of Automation,CAS
Synchronous Bidirectional Inference for Neural Sequence Generation | |
Zhang, Jiajun1,2; Zhou, Long1,2; Zhao, Yang1,2; Zong, Chengqing1,2,3 | |
发表期刊 | Artificial Intelligence |
2020-01 | |
期号 | 281 (2020) 103234页码:pp.1-19 |
摘要 | In sequence to sequence generation tasks (e.g. machine translation and abstractive summarization), inference is generally performed in a left-to-right manner to produce the result token by token. The neural approaches, such as LSTM and self-attention networks, are now able to make full use of all the predicted history hypotheses from left side during inference, but cannot meanwhile access any future (right side) information and usually generate unbalanced outputs (e.g. left parts are much more accurate than right ones in Chinese-English translation). In this work, we propose a synchronous bidirectional inference model to generate outputs using both left-to-right and right-to-left decoding simultaneously and interactively. First, we introduce a novel beam search algorithm that facilitates synchronous bidirectional decoding. Then, we present the core approach which enables left-to-right and right-to-left decoding to interact with each other, so as to utilize both the history and future predictions simultaneously during inference. We apply the proposed model to both LSTM and self-attention networks. Furthermore, we propose a novel fine-tuning based parameter optimization algorithm in addition to the simple two-pass strategy. The extensive experiments on machine translation and abstractive summarization demonstrate that our synchronous bidirectional inference model can achieve remarkable improvements over the strong baselines. |
关键词 | Sequence to sequence learning, Bidirectional inference, Beam search, Machine translation, Summarization |
收录类别 | SCI |
语种 | 英语 |
七大方向——子方向分类 | 自然语言处理 |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/39590 |
专题 | 多模态人工智能系统全国重点实验室_自然语言处理 |
作者单位 | 1.National Laboratory of Pattern Recognition, CASIA, Beijing, China 2.University of Chinese Academy of Sciences, Beijing, China 3.CAS Center for Excellence in Brain Science and Intelligence Technology, Beijing, China |
第一作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Zhang, Jiajun,Zhou, Long,Zhao, Yang,et al. Synchronous Bidirectional Inference for Neural Sequence Generation[J]. Artificial Intelligence,2020(281 (2020) 103234):pp.1-19. |
APA | Zhang, Jiajun,Zhou, Long,Zhao, Yang,&Zong, Chengqing.(2020).Synchronous Bidirectional Inference for Neural Sequence Generation.Artificial Intelligence(281 (2020) 103234),pp.1-19. |
MLA | Zhang, Jiajun,et al."Synchronous Bidirectional Inference for Neural Sequence Generation".Artificial Intelligence .281 (2020) 103234(2020):pp.1-19. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Synchronous bidirect(1794KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论