Timeline-based Sentence Decomposition with In-Context Learning for Temporal Fact Extraction

Abstract: Facts extraction is pivotal for constructing knowledge graphs. Recently, the increasing demand for temporal facts in downstream tasks has led to the emergence of the task of temporal fact extraction. In this paper, we specifically address the extraction of temporal facts from natural language text. Previous studies fail to handle the challenge of establishing time-to-fact correspondences in complex sentences. To overcome this hurdle, we propose a timeline-based sentence decomposition strategy using LLMs with in-context learning, ensuring a fine-grained understanding of the timeline associated with various facts. In addition, we evaluate the performance of LLMs for direct temporal fact extraction and get unsatisfactory results. To this end, we introduce TSDRE, a method that incorporates the decomposition capabilities of LLMs into the traditional fine-tuning of smaller pre-trained LLMs (PLMs). To support the evaluation, we construct ComplexTRED, a complex temporal fact extraction dataset. Our experiments show that TSDRE achieves state-of-the-art results on both HyperRED-Temporal and ComplexTRED datasets.

• The paper introduces TSDRE, a method that employs timeline-based sentence decomposition with in-context learning for accurate temporal fact extraction.
• It combines large language models with fine-tuned PLMs to dissect complex sentences and improve recall on datasets like ComplexTRED.
• Experiments show that TSDRE outperforms traditional methods, paving the way for advanced temporal reasoning and dynamic knowledge graphs.

Timeline-based Sentence Decomposition with In-Context Learning for Temporal Fact Extraction

Introduction

Temporal fact extraction is vital for enriching knowledge graphs with time-dependent information. This paper presents a novel approach to temporal fact extraction through timeline-based sentence decomposition using LLMs with in-context learning. Traditional methods struggle with complex sentences where time-to-fact correspondences are obscured by intricate language structures. Addressing this, the authors propose TSDRE, a methodology combining LLMs' timeline-decomposition capabilities with the fine-tuning of smaller pre-trained LLMs (PLMs).

Figure 1: A difficult temporal fact extraction example, which contains 12 temporal facts in one sentence.

Methodology

The core of this research is the timeline-based sentence decomposition strategy, which dissects sentences based on their temporal references, enabling a nuanced understanding of time-fact associations. The decomposition leverages LLMs' in-context learning capabilities, alleviating the need for extensive training datasets. This technique is incorporated into TSDRE, merging LLM-driven timeline decomposition with PLM fine-tuning, thus enhancing temporal fact extraction across challenging datasets.

Figure 2: Framework comparison between Flan-T5 and TSDRE. Leveraging Timeline-based sentence decomposition for training can significantly improve the recall of the Flan-T5.

Experimentation

To evaluate the efficacy of TSDRE, the authors constructed ComplexTRED, a dataset laden with complex temporal narratives. TSDRE was benchmarked against HyperRED-Temporal and ComplexTRED datasets, achieving state-of-the-art performance. This improvement is attributed to TSDRE's ability to discern intricate temporal contexts better than existing methods, particularly in sentences with overlapping timelines or implicit temporal relations.

Implications and Future Work

The research outlines significant advancements in temporal fact extraction, offering practical enhancements for downstream applications like dynamic knowledge graphs and temporal reasoning systems. This method opens avenues for more sophisticated AI-driven comprehension of time-sensitive data, potentially informing future model architectures and integration approaches. An area for future exploration includes refining models to infer implicit temporal information, such as events specified by relative time markers like "three days later."

Figure 3: Player awards are presented in a timeline.

Conclusion

The paper introduces a robust method to tackle complexities in temporal fact extraction by incorporating advanced techniques in sentence decomposition and model training. TSDRE's performance demonstrates that integrating the parsing strength of LLMs with the focused calibration of smaller PLMs can significantly improve temporal fact extraction. This innovative approach marks a crucial step toward more nuanced AI interpretations of temporal contexts, promising enhanced capability for time-sensitive applications in artificial intelligence.