Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion

Abstract: Temporal Knowledge Graph Completion (TKGC) is a complex task involving the prediction of missing event links at future timestamps by leveraging established temporal structural knowledge. This paper aims to provide a comprehensive perspective on harnessing the advantages of LLMs for reasoning in temporal knowledge graphs, presenting an easily transferable pipeline. In terms of graph modality, we underscore the LLMs' prowess in discerning the structural information of pivotal nodes within the historical chain. As for the generation mode of the LLMs utilized for inference, we conduct an exhaustive exploration into the variances induced by a range of inherent factors in LLMs, with particular attention to the challenges in comprehending reverse logic. We adopt a parameter-efficient fine-tuning strategy to harmonize the LLMs with the task requirements, facilitating the learning of the key knowledge highlighted earlier. Comprehensive experiments are undertaken on several widely recognized datasets, revealing that our framework exceeds or parallels existing methods across numerous popular metrics. Additionally, we execute a substantial range of ablation experiments and draw comparisons with several advanced commercial LLMs, to investigate the crucial factors influencing LLMs' performance in structured temporal knowledge inference tasks.

• The paper fine-tunes LLMs with historical temporal data and reverse logic to predict event links with state-of-the-art accuracy on TKGC benchmarks.
• It employs data augmentation techniques and a temporal sequential framework to capture evolving relationships in dynamic knowledge graphs.
• Experiments reveal that integrating structured historical context significantly enhances LLM performance in forecasting future events.

Overview of Temporal Knowledge Graph Completion

Temporal Knowledge Graphs (TKGs) are vital for understanding how entities and their relationships evolve over time. Predicting missing event links in TKGs, a task known as Temporal Knowledge Graph Completion (TKGC), can be quite complex due to the dynamic nature of TKGs. This paper introduces an innovative approach that reimagines the TKGC challenge by utilizing the capabilities of LLMs.

Methodology Employed in Research

The authors' method hinges on fine-tuning LLMs to better adapt to the temporal sequence prediction task. They fine-tune the models on known data from TKGs to enhance event generation. The process of fine-tuning includes structuring the LLM's knowledge of past events, which helps it learn relationships and patterns necessary for forecasting future ones. The researchers also propose data augmentation techniques and the integration of "reverse logic" to make the LLMs more effective at identifying the subtleties within structural knowledge.

Analysis and Experimental Results

Experiments comparing LLMs fine-tuned with these novel approaches to existing methods show promising results. On several TKGC benchmarks, the models outperformed current methods, achieving state-of-the-art performance. The experiments included various metrics and considered how factors like the amount of historical information available and the size of the LLM impact the model's performance in predicting temporal graph links.

Impact of Research and Future Work

The study demonstrates the potential of using LLMs not just as tools for natural language processing but as predictive models for temporal reasoning in knowledge graphs. The findings suggest that the predictive capabilities of LLMs can be significantly enhanced when equipped with structured, temporal, historical data. This research opens avenues to further apply LLMs to complex tasks in various domains and lays groundwork for future advancements in AI's ability to understand and predict the temporal dynamics of knowledge graphs.