Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inference

Abstract: LLMs have achieved strong performance across a wide range of language-based tasks by leveraging both extensive parametric knowledge and in-context learning ability, enabling them to incorporate external information provided in the input prompt. However, the integration of external knowledge can introduce conflicts, not only between the model's internal parametric knowledge and the external information, but also among multiple pieces of external contexts. Existing approaches typically assume that either the model or the provided context is reliable, overlooking the possibility that both sources may contain errors, and avoid conflicts by privileging one source over the other, rather than actively resolving inconsistencies. To address these limitations, we propose a novel framework MACR for LLM knowledge conflict resolution that moves beyond the conventional binary choice paradigm and incorporates an explicit conflict-resolution mechanism based on a multi-agent reasoning approach. Specifically, we first propose an adaptive knowledge assessment and retrieval approach that employs a modified semantic entropy measure to quantify an LLM's confidence in its answer to a given query. Based on this confidence estimation, MACR either externalizes the model's internal knowledge as textual representations or retrieves relevant external knowledge when internal knowledge is insufficient, generating basic contexts for subsequent reasoning. Then we introduce an inductive multi-agent reasoning framework with three specialized agents that, respectively, induce explicit rules, analyze potential conflicts, and resolve inconsistencies across all available contexts. Empirical results demonstrate that MACR significantly outperforms state-of-the-art baselines across benchmarks, while also providing interpretable resolutions of explicit conflicts.

• The paper introduces MACR, a framework that explicitly resolves conflicts between parametric and contextual knowledge in LLM inference.
• It employs an adaptive knowledge assessment using semantic entropy and a multi-agent reasoning module to detect and adjudicate conflicts.
• Empirical results demonstrate MACR's superior performance with higher EM/ROUGE-L scores, robustly handling noisy and outdated information.

Explicit Knowledge Conflict Resolution for LLM Inference: The MACR Framework

Problem Context and Motivation

Parametric knowledge encoded during pre-training and contextual knowledge retrieved at inference are fundamental yet potentially discordant sources for LLMs. As observed in retrieval-augmented generation (RAG) and question-answering pipelines, retrieved context can diverge from the parametric beliefs of the LLM, and conflicts may also arise among the retrieved snippets themselves. Prior literature predominantly reduces this challenge to a source selection task, either forcing alignment to contextual knowledge (e.g., KAFT [15], IRCAN [17]) or preferring resilience against contextually injected errors (e.g., [19], [20]). More recent dynamic approaches rely on model confidence to select the trusted source [10], [9]. These methodologies are fundamentally limited by their binary nature: they either avoid or defer explicit conflict resolution, assuming at least one source is reliable.

The paper "Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inference" (2606.20245) targets this methodological gap, proposing robust multi-source conflict adjudication without reliance on a-priori assumptions about the trustworthiness of any source. This is particularly crucial as both parametric memory and retrieved context can be simultaneously erroneous or outdated due to phenomena such as temporal misalignment [4], [5], misinformation injection [6], or context drift.

Methodology: The MACR Framework

The Multi-Agent Conflict Resolution (MACR) framework systematically addresses multi-source conflict in LLM inference using three tightly integrated modules: an adaptive knowledge assessment and retrieval module, and an explicit inductive multi-agent reasoning engine.

Adaptive Knowledge Assessment and Retrieval

The first stage evaluates LLM confidence in its parametric response to the input query using a modified semantic entropy (H(q)) metric. Unlike prior approaches reliant on log-probability or raw sampling entropy, the framework computes semantic entropy based on both inter-sample consistency and answer relevance to the question—formally, using pairwise semantic similarity weighted by question-answer alignment, akin to enhanced SNNE [32]. The input prompt is explicitly augmented with temporal and disambiguation cues to mitigate spurious confidence arising from temporal drift or entity ambiguity.

Depending on whether the computed entropy is below or above a learned threshold $T$, the system either:

• Externalizes Parametric Knowledge: Explicitly verbalizes the LLM's internal beliefs as context, including cut-off dates and factual background, for scrutiny by downstream agents.
• Retrieves External Context: Invokes external knowledge retrieval, selecting the highest-relevance chunk (via embedding similarity) from the corpus as support for answer generation.

This explicit bifurcation prevents hallucination propagation by the base model and ensures sampled evidence is always available for cross-context conflict adjudication.

Inductive Multi-Agent Reasoning

The central innovation is the multi-agent reasoning module, comprising three specialized LLM-based agents:

• Observer: Operates offline, inducing generalizable conflict-resolution rules from demonstrations and validating them via support and coverage metrics on a hold-out set. Rules are typed (e.g., Temporal, Misinformation) and parameterized by semantic conditions and resolution logic.
• Analyzer: Online, identifies and localizes answer-level conflicts among all contexts (parametric and retrieved). It generates context-conditioned answers, detects semantic contradictions, and extracts the conflicting textual snippets, classifying each instance by conflict type.
• Reasoner: Applies the Observer's validated rules to the detected conflicts. For each triad (conflicting snippets, conflict type), the Reasoner checks rule applicability and executes resolution logic. For unhandled cases, it falls back to parametric adjudication and sends the instance for Observer retraining, ensuring continual adaptation.

This design allows for hierarchical, interpretable conflict detection and explicit application of conflicting evidence reconciliation procedures, unlike generic chain-of-thought prompting, which the ablation shows to be insufficiently robust.

Empirical Evaluation

Comprehensive experiments are conducted on three complementary benchmarks:

• ConflictBank 33
• ConFiQA 34
• MQuAKE 35

The principal metrics are Exact Match (EM) and ROUGE-L. The results show that MACR achieves the highest EM/ROUGE-L across all datasets and both Llama3.1-8B and Qwen2.5-7B models. For instance, on ConflictBank with Llama3.1-8B, MACR obtains EM/ROUGE-L of 0.549/0.678, significantly outperforming InstructRAG (0.312/0.506) and CK-PLUG (0.288/0.449).

Noise robustness is demonstrated by systematically increasing the proportion of noisy or contradictory contexts; MACR's performance degrades more gracefully than all baselines, retaining significantly higher accuracy even at high noise ratios. Ablation studies confirm that both semantic entropy-driven context selection and inductive multi-agent resolution are critical: replacing these with naive approaches produces drastic performance drops (e.g., EM falls from 0.549 to 0.229 with CoT reasoning).

Notably, in realistic temporal conflict scenarios (e.g., Tesla headquarters relocation), MACR overrides outdated parametric beliefs in favor of temporally relevant evidence, producing not only correct answers but explicit, interpretable conflict resolution rationales. In contrast, vanilla ICL and RAG strategies manifest confirmation bias and often propagate errors present in training data or spurious retrieved contexts.

Theoretical and Practical Implications

This work advances both the theoretical understanding and practical handling of knowledge conflict in LLMs:

• Beyond Binary Source Selection: MACR demonstrates that explicit, rule-based multi-agent resolution yields substantial gains over binary selection or dynamic switching architectures, especially as the number and noisiness of contexts increases.
• Rule-Based Interpretability: By externalizing parametric knowledge and applying type-aware, human-interpretable rules, the system offers principled explanations for its conflict adjudication, directly addressing the lack of transparency in black-box confidence heuristics and prompting chains.
• Adaptation to Unknown Conflict Modes: The framework’s feedback and continual rule induction architecture positions it for continual learning in open-world settings, where new conflict types and error modalities can be encountered.

Practically, MACR's approach can be deployed in RAG-based QA, knowledge-intensive dialogue, and factuality-critical generation systems where multi-source evidence and parametric memory must be robustly harmonized.

Limitations and Future Directions

Despite empirical gains, the framework currently incurs increased computational cost due to agent interaction and LLM call overhead. Rule induction quality is bounded by the representational and abstraction capacity of the underlying LLM and demonstration data—a limiting factor in highly ambiguous or underrepresented conflict patterns. Directions for optimization include compressing agent communication, integrating symbolic reasoning, and leveraging meta-learning for broader generalization of conflict resolution logic.

Conclusion

MACR provides a formally specified, empirically validated framework for explicit knowledge conflict resolution in LLM inference, moving definitively beyond binary allegiance to parametric or contextual sources. Through adaptive context selection, explicit externalization of parametric beliefs, and a collaborative inductive multi-agent reasoning system, MACR delivers robust, interpretable, and accurate answers in the presence of unreliable, contradictory information. This paradigm has significant implications for the reliability and transparency of next-generation retrieval-augmented LLM-based applications.