Abstract
In this paper, we explore the feasibility of using a transformer-based, spatiotemporal attention network (STAN) for gradient-based time-series explanations. First, we trained the STAN model for video classifications using the global and local views of data and weakly supervised labels on time-series data (i.e. the type of an activity). We then leveraged a gradient-based XAI technique (e.g. saliency map) to identify salient frames of time-series data. According to the experiments using the datasets of four medically relevant activities, the STAN model demonstrated its potential to identify important frames of videos.
Abstract (translated)
在本文中，我们探讨了使用基于Transformer的时空注意力网络（STAN）进行基于梯度的时序解释的可行性。首先，我们使用数据的全局和局部视图以及时间序列数据（即活动）的弱监督标签来训练STAN模型。然后，我们利用基于梯度的XAI技术（例如显着性图）来识别时间序列数据的显着帧。根据四个医学相关活动数据集的实验结果，STAN模型证明了其在识别视频重要帧方面的潜力。
URL
https://arxiv.org/abs/2405.17444
