Abstract
Event reasoning is a fundamental ability that underlies many applications. It requires event schema knowledge to perform global reasoning and needs to deal with the diversity of the inter-event relations and the reasoning paradigms. How well LLMs accomplish event reasoning on various relations and reasoning paradigms remains unknown. To mitigate this disparity, we comprehensively evaluate the abilities of event reasoning of LLMs. We introduce a novel benchmark EV2 for EValuation of EVent reasoning. EV2 consists of two levels of evaluation of schema and instance and is comprehensive in relations and reasoning paradigms. We conduct extensive experiments on EV2. We find that LLMs have abilities to accomplish event reasoning but their performances are far from satisfactory. We also notice the imbalance of event reasoning abilities in LLMs. Besides, LLMs have event schema knowledge, however, they're not aligned with humans on how to utilize the knowledge. Based on these findings, we introduce two methods to guide the LLMs to utilize the event schema knowledge. Both methods achieve improvements.
Abstract (translated)
事件推理是一个基本的能力，许多应用都建立在它之上。它需要事件模式知识来执行全局推理，并需要处理事件间关系的多样性和推理范式。LLM在各种关系和推理范式上实现事件推理的能力仍然是一个未知的问题。为了减轻这种不平等，我们全面评估了LLM在各种关系和推理范式上的事件推理能力。我们引入了一个新的基准EV2用于评估事件推理能力。EV2由模式和实例的两个级别的评估组成，在关系和推理范式上都是全面的。我们在EV2上进行了广泛的实验。我们发现，LLM具有实现事件推理的能力，但他们的表现并不令人满意。我们还注意到了LLM在事件推理能力方面的不平衡。此外，LLM具有事件模式知识，然而，他们在如何利用这些知识上与人类并不一致。基于这些发现，我们引入了两种方法来引导LLM利用事件模式知识。两种方法都取得了改进。
URL
https://arxiv.org/abs/2404.17513
