Abstract
Knowledge graphs (KGs) are vital for knowledge-intensive tasks and have shown promise in reducing hallucinations in large language models (LLMs). However, constructing high-quality KGs remains difficult, requiring accurate information extraction and structured representations that support interpretability and downstream utility. Existing LLM-based approaches often focus narrowly on entity and relation extraction, limiting coverage to sentence-level contexts or relying on predefined schemas. We propose a hierarchical extraction framework that organizes information at multiple levels, enabling the creation of semantically rich and well-structured KGs. Using state-of-the-art LLMs, we extract and construct knowledge graphs and evaluate them comprehensively from both structural and semantic perspectives. Our results highlight the strengths and shortcomings of current LLMs in KG construction and identify key challenges for future work. To advance research in this area, we also release a curated dataset of LLM-generated KGs derived from research papers on children's mental well-being. This resource aims to foster more transparent, reliable, and impactful applications in high-stakes domains such as healthcare.
Abstract (translated)
知识图谱（KG）对于知识密集型任务至关重要，并且在减少大型语言模型（LLM）中的幻觉方面展现出潜力。然而，构建高质量的知识图谱仍然具有挑战性，需要准确的信息提取和能够支持可解释性和下游应用的结构化表示。现有的基于LLM的方法通常仅专注于实体和关系抽取，覆盖范围局限于句子级别的上下文或依赖于预定义的模式。我们提出了一种分层抽取框架，该框架在多个层次上组织信息，以创建语义丰富且结构良好的知识图谱。利用最先进的LLM，我们提取并构建了知识图谱，并从结构和语义两个方面对其进行了全面评估。我们的研究结果突出了当前LLM在知识图谱构建中的优势与不足，并确定了未来工作的关键挑战。 为了推动该领域的研究，我们还发布了一个精心策划的数据集，其中包含来自儿童心理健康研究论文的LLM生成的知识图谱。这一资源旨在促进医疗等高风险领域中更透明、可靠和有影响力的应用。
URL
https://arxiv.org/abs/2510.11297
