Knowledge-aware Fine-tuning (KaFT)

• Knowledge-aware Fine-tuning (KaFT) is a paradigm that integrates external structured knowledge and domain-specific priors into standard fine-tuning.
• It employs methods like knowledge graph injection, conflict-aware data weighting, and physics-based regularization to enhance model robustness.
• KaFT balances the retention of pretrained world knowledge with the targeted injection of new information, reducing hallucination and catastrophic forgetting.

Knowledge-aware Fine-tuning (KaFT) is a broad paradigm in machine learning and, in particular, in neural language modeling and mechanistic modeling, that augments standard fine-tuning procedures by explicitly encoding knowledge sources, constraining optimization to reflect priors, or selectively weighting domain- and model-specific information to balance adaptation and retention. Its purpose is to maximize task performance and knowledge transfer while minimizing catastrophic forgetting and hallucination, enabling robust and interpretable adaptation to data with complex domain-specific, contextual, or structured-knowledge characteristics.

1. Core Concepts and Definitions

KaFT departs from vanilla supervised fine-tuning (SFT), which simply fits model parameters $\theta$ to downstream labeled data by loss minimization (e.g., cross-entropy), treating all examples equally and providing no guarantee for the preservation or controlled manipulation of knowledge acquired during pretraining. KaFT encompasses a suite of methodologies in which (a) external, structured, or domain-specific knowledge (e.g., knowledge graphs, physics constraints, prior knowledge mastery) is injected, respected, or leveraged during fine-tuning, (b) the optimization objective is augmented to reflect knowledge-guided regularization, and (c) data selection or algorithmic weighting is controlled by the level of “conflict” or knowledge overlap between model and data (Ye et al., 20 Sep 2025, Zeng et al., 17 Dec 2025, Zhong et al., 21 May 2025, Lyu et al., 2024, Lv et al., 12 Jan 2026).

Typical goals are:

• Preservation of pre-trained world knowledge (avoidance of catastrophic forgetting)
• Targeted injection of new, beneficial knowledge while controlling hallucination and knowledge “overwriting”
• Enhancement of controllability (preference for context over parametric priors when context is relevant) and robustness (fallback to priors when no relevant context is provided) (Li et al., 2022)
• Improved calibration of model confidence relative to knowledge overlap (Wang et al., 27 May 2025)
• Interpretability of adaptation, e.g., by isolating site- or domain-specific deviations in analyzable submodules (Zeng et al., 17 Dec 2025)

2. Knowledge Sources and Regularization Mechanisms

KaFT implementations employ a variety of knowledge resources and mechanisms:

A. Structured Knowledge Bases: Fine-tuning can be guided by external KGs (e.g., Wikidata) via direct injection (augmenting input or hidden states), joint GNN-LM fusion, joint alignment losses, structural perturbation robustness, or rationale-based KL minimization. In biomedical applications, synonyms from knowledge bases are leveraged both in pre-training and fine-tuning (Yuan et al., 2022, Zhang et al., 20 Aug 2025, Lv et al., 12 Jan 2026).

B. Domain-specific Priors: For physical and environmental modeling (e.g., carbon-cycle quantification), KaFT incorporates explicit physics losses $L_\mathrm{phys}$, encoding hard constraints (mass-balance, non-negativity, monotonicity) that apply both at pretraining and fine-tuning (Zeng et al., 17 Dec 2025). In knowledge calibration, known versus unknown data is tracked to regularize overconfident adaptation (Wang et al., 27 May 2025).

C. Context and Mastery-based Curation: KaFT often weights or filters training data according to the fidelity of model mastery or knowledge conflict. Query diversification and response sampling are used to evaluate model agreement with training targets, stratifying examples into “right,” “might”, and “wrong” categories with distinct weighting (e.g., $\alpha, \beta$) in the optimization loss (Zhong et al., 21 May 2025, Li et al., 2024, Gekhman et al., 2024, Ye et al., 20 Sep 2025).

D. Parameter-efficient Mechanisms: Knowledge-preserving and context-adaptive fine-tuning is sometimes realized via PEFT techniques such as SVD-based adapter positioning (KaSA, CorDA), in which singular-value reweighting or context-oriented factorization isolates world-knowledge subspaces for freezing, while allowing adaptation in task- or context-aligned components (Wang et al., 2024, Yang et al., 2024).

3. Methodological Frameworks

A. Losses and Update Objectives

KaFT methods frequently employ composite losses of the form: $$L_\mathrm{total} = L_\mathrm{task} + \lambda L_\mathrm{knowledge} + \mu L_\mathrm{anchor} + \rho L_\mathrm{penalize}$$ where:

• $L_\mathrm{task}$ is a standard prediction loss (cross-entropy, regression error),
• $L_\mathrm{knowledge}$ measures constraint or alignment with external knowledge (e.g., physics loss, KG alignment, rationale KL divergence),
• $L_\mathrm{anchor}$ penalizes deviation from pre-trained weights (e.g., $\|\theta_s - \theta^*\|_2^2$ for site-specific adaptation (Zeng et al., 17 Dec 2025)),
• $L_\mathrm{penalize}$ regularizes adapters or redundant parameter changes.

In models such as FTBSC-KGML, the two-stage procedure involves both global pretraining and site-specific adaptation with explicit regularization to control the degree of local-specialization versus global generalization (Zeng et al., 17 Dec 2025).

B. Data Selection and Example Weighting

KaFT leverages stratified or curriculum-based selection, focusing on:

• Mastery-based curation: Training on mid-mastery or partially known samples (as opposed to fully unknown or trivially known) to maximize retained and acquired knowledge (Gekhman et al., 2024, Li et al., 2024, Ye et al., 20 Sep 2025).
• Conflict-aware weighting: Assigning dynamic sample weights according to model's propensity for agreement or conflict with new labels, typically suppressing highly conflicting examples to prevent harmful overwriting but not excluding them altogether (Zhong et al., 21 May 2025).

C. Adapter and Parameter-efficient Fine-tuning

Low-rank adapters or SVD-based decompositions (e.g., in KaSA, CorDA) are tailored to preserve or adapt knowledge depending on context orientation, via selective freezing or singular-value scaling. The context can be representative of either world-knowledge (to be preserved) or the downstream instruction/task (to be injected) (Wang et al., 2024, Yang et al., 2024).

D. KG-enabled Reasoning and Rationale Distillation

Highly-structured domains (QA requiring multi-hop reasoning) benefit from composite architectures: GNN-based graph encoders fused with LMs, joint optimization for both task and structural objectives, and KL-based rationale distillation (e.g., KALE), where rationales derived from KGs are used to shape the predictive distribution of the fine-tuned model even in the absence of explicit rationales at inference (Lv et al., 12 Jan 2026, Zhang et al., 20 Aug 2025, Ye et al., 2023).

4. KaFT in Application Domains

A. Environmental and Physical Modeling

FTBSC-KGML demonstrates KaFT in knowledge-guided carbon cycle estimation, using a GRU-based architecture with modular subunits (e.g., GRU_Ra, GRU_Rh, attention module), mass-balance physics guidance, and site-specific calibration heads. The two-stage (pretrain+fine-tune) procedure yields marked reductions in validation MSE (up to –43.6%) compared to non-KaFT baselines. Fine-tuning is regularized to anchor site-specific parameters near global optima, ensuring gains are not at the cost of plausibility or interpretability (Zeng et al., 17 Dec 2025).

B. LLM Knowledge Injection and Control

In LLMs, KaFT encompasses mastery-based curation, parameter update filtering, and integration of knowledge graphs or rationale-generating mechanisms. For instance, parameter restoration experiments demonstrate up to 90% of SFT-induced parameter changes fail to support knowledge enhancement, and their reversal can yield 8–10% gains in closed-book QA accuracy. Small, high-quality, and mid-mastery-targeted data yields best improvement of model knowledge, whereas excessive or low-mastery (fully unknown) data degrades performance (Ye et al., 20 Sep 2025, Gekhman et al., 2024, Li et al., 2024).

C. Structured Knowledge Fusion

Graph-based KaFT frameworks achieve state-of-the-art in structured reasoning via end-to-end fusion of token-level LM representations and KG-derived entity embeddings, using attention-based or gated mechanisms and joint task-structural alignment loss. Ablation studies show that disabling KG fusion or structural loss degrades performance by 1.5–4 pp in QA accuracy (Zhang et al., 20 Aug 2025, Ye et al., 2023).

D. Robustness, Controllability, and Calibration

KaFT methodologies address hallucination and poor calibration by explicit anti-hallucination objectives and online identification of knowledge overlap. For example, inclusion of counterfactual and irrelevant context in the training mix, with targets derived from the model’s own priors, greatly enhances both controllability (up to 80%, from <5%) and robustness (up to 80% on SQuAD 2.0 “impossible” queries). Knowledge-aware calibration frameworks such as CogCalib apply regularization only to “known” examples as determined by online NLL thresholds, yielding 57% ECE reductions with minimal accuracy loss (Wang et al., 27 May 2025, Li et al., 2022).

5. Quantitative Impact and Empirical Trends

System/Domain	KaFT Mechanism	Key Gains vs. Baseline	Reference
LLM QA (closed-book)	Mastery-based SFT; parameter filtering	ΔACC +13.69% (240 vs. 1920 samples); 8–10% restoration gain	(Ye et al., 20 Sep 2025)
Carbon-cycle modeling (multi-state)	Physics-guided KaFT	Up to –43.6% MSE vs. baseline	(Zeng et al., 17 Dec 2025)
LLM hallucination control	Knowledge filtering	Each new “unknown” example fit reduces test EM by 8.3%	(Gekhman et al., 2024)
Structured KG-LM fusion (T-REX QA)	GNN fusion + alignment	QA-Acc +4 pp over prior; F1 up to 82.1	(Zhang et al., 20 Aug 2025)
PEFT adaptation (KaSA)	SVD, singular-value adaptation	+1–3% NLU/NLG gains; better instruction following	(Wang et al., 2024)
Calibration (CogCalib)	Knowledge-biased loss	–57% ECE, ACC preserved	(Wang et al., 27 May 2025)
Customer service dialog	KAFT with retrieved KB	Inform Rate ×2–3 vs. prompting	(Cai et al., 28 Jun 2025)
Rationale distillation (KALE)	KG path + KL alignment	Up to +11.72% QA accuracy	(Lv et al., 12 Jan 2026)

Sensitivity analyses consistently demonstrate that stratified or context-filtered data, rationale-guided learning, adapter alignment, and targeted regularization all outperform naive SFT, simple prompting, or full-parameter unrestricted adaptation.

6. Interpretability, Limitations, and Design Recommendations

Interpretability is enhanced in KaFT frameworks that (a) retain modular or factorized model structures, (b) assign adaptation to isolated adapter heads or explicit rationale-generation components, and (c) maintain analyzable links between learned deviations and domain-specific structures (e.g., calibration heads, KG reasoning paths, singular-value components) (Zeng et al., 17 Dec 2025, Wang et al., 2024, Lv et al., 12 Jan 2026).

Limitations:

• KaFT generally requires auxiliary resources (expert KGs, prior knowledge curation, or task-oriented sample selection).
• Oversuppression of conflicting samples can inhibit model’s ability to learn genuinely new knowledge if not balanced properly.
• The optimal number and weighting of conflict splits, adapter ranks, or external knowledge components is empirical and task-specific.
• In PEFT schemes, automatic determination of decomposition ranks is not yet addressed (Wang et al., 2024).

Recommended KaFT pipeline includes: preclassification of knowledge overlap, representative context sampling, adapter design to isolate knowledge and task subspaces, and dynamic or stratified loss weighting. For tasks requiring strict preservation (e.g., environmental or compliance-critical modeling), stronger anchoring and knowledge-constrained objectives (e.g., physics or structure alignment) are necessary (Zeng et al., 17 Dec 2025, Yang et al., 2024).

7. Extensions and Future Directions

Emerging directions include:

• Modular multi-task KaFT, training adapters tuned for distinct knowledge types or domains and dynamically fusing outputs (Wang et al., 2024, Yang et al., 2024).
• Zero-latency rationale internalization via KL alignment and rationale-free inference (Lv et al., 12 Jan 2026).
• Real-time or online knowledge overlap detection to steer continual learning, calibration, and data curation (Wang et al., 27 May 2025, Li et al., 2024).
• Application-specific KaFT in domains requiring interpretable, reliable adaptation without prohibitive cost (e.g., healthcare, environmental policy, legal reasoning) (Zeng et al., 17 Dec 2025, Yuan et al., 2022, Zhong et al., 21 May 2025).

KaFT thus crystallizes a set of practical principles for the controlled, knowledge-informed adaptation of machine learning systems, combining advances in parameter-efficient updates, knowledge distillation, structural learning, and robust optimization.