Abstract
Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse tasks, yet their black-box nature raises concerns about transparency and faithfulness. Input attribution methods aim to highlight each input token's contributions to the model's output, but existing approaches are typically model-agnostic, and do not focus on transformer-specific architectures, leading to limited faithfulness. To address this, we propose Grad-ELLM, a gradient-based attribution method for decoder-only transformer-based LLMs. By aggregating channel importance from gradients of the output logit with respect to attention layers and spatial importance from attention maps, Grad-ELLM generates heatmaps at each generation step without requiring architectural modifications. Additionally, we introduce two faithfulneses metrics $\pi$-Soft-NC and $\pi$-Soft-NS, which are modifications of Soft-NC/NS that provide fairer comparisons by controlling the amount of information kept when perturbing the text. We evaluate Grad-ELLM on sentiment classification, question answering, and open-generation tasks using different models. Experiment results show that Grad-ELLM consistently achieves superior faithfulness than other attribution methods.
Abstract (translated)
大型语言模型（LLMs）在各种任务上展现了卓越的能力，但其黑盒特性引发了关于透明度和忠实性的担忧。输入归因方法旨在突出每个输入标记对模型输出的贡献，然而现有的方法通常是无模型特定的，不专注于变压器架构特有的属性，导致了忠实性方面的限制。为了解决这些问题，我们提出了Grad-ELLM，这是一种基于梯度的针对解码器独立的Transformer基础LLM的归因方法。通过聚合输出逻辑与注意力层相关的梯度所计算出的通道重要性和从注意力图谱中得出的空间重要性，Grad-ELLM能够在不进行架构修改的情况下，在每次生成步骤时产生热力图。 此外，我们还引入了两种忠实性衡量标准$\pi$-Soft-NC和$\pi$-Soft-NS，它们是对Soft-NC/NS的改进版本，通过控制在扰动文本时保留的信息量，提供了更公平的比较方式。我们在不同的模型上使用情感分类、问答和开放生成任务来评估Grad-ELLM的表现。 实验结果显示，与其它归因方法相比，Grad-ELLM在忠实性方面始终表现得更为优秀。
URL
https://arxiv.org/abs/2601.03089
