Fact in Fragments: Deconstructing Complex Claims via LLM-based Atomic Fact Extraction and Verification

Abstract: Fact verification plays a vital role in combating misinformation by assessing the veracity of claims through evidence retrieval and reasoning. However, traditional methods struggle with complex claims requiring multi-hop reasoning over fragmented evidence, as they often rely on static decomposition strategies and surface-level semantic retrieval, which fail to capture the nuanced structure and intent of the claim. This results in accumulated reasoning errors, noisy evidence contamination, and limited adaptability to diverse claims, ultimately undermining verification accuracy in complex scenarios. To address this, we propose Atomic Fact Extraction and Verification (AFEV), a novel framework that iteratively decomposes complex claims into atomic facts, enabling fine-grained retrieval and adaptive reasoning. AFEV dynamically refines claim understanding and reduces error propagation through iterative fact extraction, reranks evidence to filter noise, and leverages context-specific demonstrations to guide the reasoning process. Extensive experiments on five benchmark datasets demonstrate that AFEV achieves state-of-the-art performance in both accuracy and interpretability.

• The paper introduces a novel AFEV framework that iteratively decomposes complex claims into atomic facts for precise verification.
• It employs dynamic fact extraction, refined evidence retrieval, and adaptive verification to reduce error propagation and enhance reasoning fidelity.
• Experimental results show state-of-the-art performance on benchmarks like HOVER and PolitiHop, highlighting improved accuracy and interpretability.

Fact Verification Frameworks for Complex Claims

The paper "Fact in Fragments: Deconstructing Complex Claims via LLM-based Atomic Fact Extraction and Verification" presents a novel framework known as Atomic Fact Extraction and Verification (AFEV) for effectively verifying complex claims. This method primarily targets the challenges posed by multi-hop reasoning, which conventional fact verification models struggle with due to issues like evidence fragmentation and reasoning errors. AFEV proposes an iterative decomposition strategy where complex claims are broken down into atomic facts for improved retrieval and reasoning.

Introduction to Atomic Fact Extraction and Verification

The AFEV framework addresses limitations of existing methods by iteratively refining claim understanding, reducing error propagation, and ensuring adaptive reasoning through dynamic demonstrations. The framework comprises three main modules:

1. Dynamic Atomic Fact Extraction: Utilizes LLMs to decompose claims into atomic facts iteratively, aiding targeted verification and reducing potential errors.
2. Refined Evidence Retrieval: Employs pretrained evidence reranking to filter out irrelevant data, complemented by instance retrieval to showcase dynamic demonstrations and guide verification more accurately.
3. Adaptive Atomic Fact Verification: Integrates context-specific demonstrations and reranked evidence for generating both factuality labels and rationales for each atomic fact, which are then synthesized for final claim validation.

Methodology Detailed Analysis

Dynamic Atomic Fact Extraction

The iterative approach involves generating atomic facts sequentially, refining each fact based on previously verified outcomes to improve coherence and reduce redundancy. This strategy leverages LLM predictions and verification feedback for enhanced extraction fidelity.

Figure 1: Prompt for dynamic atomic fact extraction.

Refined Evidence Retrieval

To ensure accuracy in verification, AFEV employs a two-stage retrieval process:

• Evidence Retrieval and Reranking: Initial evidence snippets are filtered using pretrained models and ranked based on relevance scores computed through the cosine similarity measure.
• Dynamic Instance Retrieval: Claims are matched with similar instances using semantic similarity scores to provide contextual guidance, ensuring adaptive reasoning tailored to each atomic fact.

  Figure 2: Prompt for adaptive fact verification.

Adaptive Atomic Fact Verification

This module synthesizes evidence and demonstrations, producing both veracity labels and rationales which provide transparency in reasoning steps. Each atomic fact is verified independently, contributing to the overall claim judgment.

Figure 3: The reasoning process of AFEV for a specific case.

Experimental Findings

Comprehensive Performance Evaluation

AFE achieves state-of-the-art results across multiple benchmark datasets like HOVER and PolitiHop, demonstrating superior performance in accuracy and interpretability compared to existing single and multi-granular models.

Ablation Studies

Experiments reveal the importance of iteratively extracting atomic facts and employing a reranking strategy to filter evidence. Performance suffers when these components are removed, emphasizing their critical role in robust fact-checking.

Conclusion

AFEV introduces a novel approach to fact verification that iteratively breaks down complex claims for precise verification. Its framework, integrating dynamic atomic fact extraction, refined evidence retrieval, and adaptive reasoning, marks an advancement in handling nuanced verification tasks. This research underscores the efficacy of LLMs in facilitating multi-step reasoning and evidential synthesis, paving the way for future developments in AI-driven misinformation detection systems.