毕业生知识库

Recently, deep learning techniques represented by neural networks have made breakthrough progress in the field of natural language processing (NLP). Essentially, neural network is a complex nonlinear function, where its parameters and representations have no clear physical senses. This black-box characteristic not only hides the decision-making process of neural networks, but also prevent an control effective control of their outputs, which greatly limits their utilization in practice. Under this context, researches on interpretability and explanation methods emerge. Among them, explanation methods based on the latent variable model (LVM) have received growing attention because LVM has a solid theoretical foundation and is able to effectively model unobservable latent explanations in the human decision-making process.

LVM is a statistical model based on observable variables and latent variables. In the context of interpretability, the observable variables are the task data, and the latent variables are the explanations that control the generation of them. In LVMs, as the carrier of explanations, the explanation generative distribution describes the probability of explanations given the input. Focusing on the explanation generative distribution, LVM-based explanation methods can be divided into two levels: application and theory. The goal of application-level explanation methods is to learn explanation generative model for specific tasks and models, and the challenge lies in designing a highly understandable latent variable form with respect to the target task. On the other side, the goal of theoretical level explanation methods is to directly study theoretical properties of the explanation generative distribution, and the challenge lies in designing a reasonable and easy-to-analyze LVM structure and then analyzing properties of the unknown explanation generative distribution. 

This paper aims to study the explanation methods at the above two levels for different tasks and scenarios of NLP. Firstly, to address the application-level challenge, this study focuses on two basic NLP tasks: natural language inference (NLI) and sentiment analysis, studying highly understandable latent variable forms and corresponding explanation methods:

Phrase alignment post-hoc explanation method for natural language inference: Word and phrase alignment between sentence pairs forms the decision-making basis in NLI tasks. However, existing works either use mismatched explanation forms such as word-level attribution, or utilize co-attention mechanism with poor interpretability to model the alignment between sentences. To address this, based on LVM in the supervised learning scenario, this study defines the explanation latent variable as the alignment in the form of a binary mask matrix, thereby proposing a post-hoc explanation method to obtain the alignment explanation of NLI models. The explanation method is based on three optimization goals: fidelity, simplicity and continuity. Experimental results demonstrate that the proposed explanation has better fidelity and understandability. Based on the proposed explanation method, this study diagnoses three existing typical co-attention-based NLI models at the alignment level, and finds that the performance of them mainly depend on their cross-sentence phrase alignment performance. Finally, this study proposes a simple strategy based semantic role labeling according to the explanation results. This strategy can significantly improve the performance of NLI models on adversarial datasets.

Semantic tree self-explanatory method for sentiment classification: Sentiment composition is the key to the task of sentiment analysis. In the context of sentiment composition, the sentiment of a constituent is determined by the semantics of its sub-constituents and semantic composition rules. Naturally, sentiment composition is often modeled through tree structures. However, existing tree structures lack the characterization of functional words and phrases, and hence cannot explain sentiment composition completely. To this end, this study proposes semantic tree and its corresponding sentiment composition grammar to completely model and explain sentiment composition. Since there is no annotation of semantic trees in the data, based on LVM in the supervised learning scenario, this study defines the explanation latent variable as the semantic tree, thereby proposing sentiment composition module to learn the semantic tree parser without annotations. Sentiment composition module not only can predict the sentiment label for a given input, but also be able to generate a semantic tree explanation for that prediction. Experimental results show that after adding sentiment composition module, the accuracy performance on sentiment classification in both vanilla and cross-domain settings is significantly improved. Furthermore, the generated semantic tree explanation shows that the model can explain complex sentiment combination phenomena well.

Secondly, to address the theoretical level challenge, based on LVM, this study focuses on the the foundational language models of NLP, exploring the in-context generation and learning abilities of them. Based on the explanation results, this study attempts to use the calibration method to improve the performance and stability of in-context learning. To be concrete:

On the in-context generation and topic generalization ability for language models: In-context generation/learning ability is one of the most important emergent abilities of large language models. Given an in-context prompt concatenating few-shot examples, the language model can implicitly infer the pattern of these examples and reuse it in subsequent completion. this study investigates the in-context generation ability of language models. First of all, this study proposes a LVM that conforms to real data to describe the data pre-trained distribution, and formalizes in-context generation phenomenon reasonably. The latent variables of this model refer to the mode and outline of the text. Next, theoretically, it is proven that if the language model can approximate the pre-trained distribution well, then when given in-context prompts, the explanation generative distribution will converge to the single-point distribution on the true mode and topic of the in-context samples in probability as the number of samples increases. An a macro level, the manifestation of this property is the in-context generation ability observed in the experiment. Finally, this study explores the impact of different settings of data and models on in-context generation and the corresponding topic generalization ability. Experimental results suggest that the combination of data, the proportion of repetitive topics, the number of model parameters, and the model's window size play an important role in determining the topic generalization of in-context generation. These conclusions provide insights for both dataset and model design. Note that in-context generation is the basis of in-context learning, so the conclusions can be easily transferred to in-context learning.

On the in-context learning stability analysis and calibration for language models: Existing studies have generally found that the performance of in-context learning is unstable with respect to the prompt setting. To this end, according to the theoretical results based on LVM in the above work, this study systematically explains that this phenomenon originates from the large variance of the language model's inference to the mode and topic when there are only a few samples. Meanwhile, through qualitative and quantitative analysis, this study also find that the instability mainly poses label shift, that is, the in-context label marginal distribution is very sensitive to the prompt setting. To alleviate this, this study proposes generative calibration method, which employs Monte-Carlo sampling to estimate the in-context label marginal distribution and uses this estimate to calibrate the in-context prediction distribution. Experimental results under different datasets, language models, numbers of shots, and prompt settings show that the performance of generative calibration generally exceeds in-context learning and existing calibration methods, and is competitive to the performance of complex prompt optimization algorithms based on large-scale data. Furthermore, generative calibration is robust with respect to the selection and order of samples in the in-context prompt.