HIN-LLM Enhanced Knowledge Tracing

• The paper introduces HISE-KT, a novel framework combining HIN modeling with LLM-driven meta-path quality assessment for accurate prediction of student performance.
• HISE-KT constructs a multi-relationship heterogeneous network and employs automated meta-path selection alongside student similarity retrieval to enhance interpretability.
• Empirical results show up to +9% AUC improvement on public datasets, with clear, evidence-backed explanations that address meta-path noise and improve prediction transparency.

HIN-LLM Synergistic Enhanced Knowledge Tracing (HISE-KT) is a knowledge tracing framework integrating heterogeneous information networks (HINs) with LLMs to achieve accurate and evidence-based prediction of student performance, together with interpretable explanations. HISE-KT departs from earlier models prone to meta-path noise and explanation inconsistency by introducing automated meta-path quality assessment, student-similarity retrieval, and structured prompt engineering, each informed by educational psychology and optimized with LLM capabilities. The method demonstrates substantial predictive and interpretative improvement on public knowledge tracing benchmarks (Duan et al., 19 Nov 2025).

1. Multi-Relationship Heterogeneous Information Network Construction

HISE-KT models educational data as a multi-relationship heterogeneous information network (MRHIN)

$G = (V, E, \mathcal{T}, \mathcal{R}),$

where $V = \bigcup_{T \in \mathcal{T}} V_T$ is the set of nodes with the following type set:

• $U$: students,
• $Q$: questions,
• $K$: knowledge concepts,
• $A$: student-ability levels (Low/Medium/High),
• $D$: question-difficulty levels (Low/Medium/High).

The edge-type set $$\mathcal{R} = \{r_{Q-U}, r_{Q-K}, r_{Q-D}, r_{U-A}\}$$ encodes relationships: student answered question ($r_{Q-U}$), question involves concept ($r_{Q-K}$), question difficulty ($r_{Q-D}$), student ability level ($r_{U-A}$). Each relation $r\in\mathcal{R}$ is associated with a binary adjacency matrix $A^{(r)}$ such that $A^{(r)}_{i,j}=1$ iff $(v_i \xrightarrow{r} v_j) \in E$.

This formalization enables joint encoding of interactions, content, skill level, and student ability, forming the substrate for cross-semantic meta-path reasoning (Duan et al., 19 Nov 2025).

2. Meta-Path Specification and Instantiation

A family of meta-path templates $\Phi = \{\phi_1, \phi_2, ..., \phi_L\}$ defines possible cross-type traversals:

• Basic instances include $\phi_1 = Q$–$U$–$Q$, $\phi_2 = Q$–$K$–$Q$, $\phi_3 = Q$–$D$–$Q$, $\phi_4 = Q$–$U$–$A$–$U$–$Q$.
• Composite meta-paths such as $\phi_5 = Q$–$K$–$Q$–$U$–$Q$–$D$–$Q$ encode more complex semantic traversals.

Instantiated paths $p = (v_0, v_1, ..., v_L)$ conform to both type and edge constraints: $\mathrm{type}(v_i)=T_i$, $(v_{i-1},v_i)\in E$. Meta-paths encode both direct (single-hop) and higher-order relational semantics (multi-hop, cross-domain), supporting nuanced aggregation beyond classic neighbor-based HIN analytics.

3. LLM-Based Meta-Path Quality Assessment and Selection

HISE-KT employs an LLM to automatically quantify the quality of instantiated meta-paths along four axes:

• Question Centrality ($C_q(p)$): Encourages paths tightly centered on the target question $q_0$ by penalizing average shortest-path distance.
• Knowledge-Concept Relevance ($R_{KC}(p)$): Measures overlap between questions on the path and the target concept $k^*$.
• Informativeness ($I_{\mathrm{info}}(p)$): Rewards distinct node instances on the path excluding $q_0$ and $k^*$.
• Node-Type Diversity ($D_{\mathrm{type}}(p)$): Penalizes homogeneity in ability/difficulty subtypes using a level entropy term.

Each dimension is scored in $[0,5]$, and summed to a total $S(p) \in [0,20]$:

$$S(p) = C_q(p) + R_{KC}(p) + I_{\mathrm{info}}(p) + D_{\mathrm{type}}(p).$$

For each meta-path template $\phi$, only the Top-K paths $p$ with maximal $S(p)$ are retained, replacing earlier heuristic, random, or manual selection strategies (Duan et al., 19 Nov 2025).

4. Meta-Path-Aware Student Similarity Retrieval

For a target student $u$ and question $q_0$, HISE-KT extracts all students $s$ that co-occur in high-quality Top-K meta-paths. Each candidate $s$ is represented by a feature vector:

$$z_{u,s} = [| \theta_u - \theta_s |,\ \tfrac{c}{|K|}\sum\limits_{k\in K}| \mathrm{acc}_u(k) - \mathrm{acc}_s(k) |,\ (1+N_Q)^{-c},\ (1+N_K)^{-c},\ (1+f)^{-c}]^\top$$

where $\theta$ is IRT ability, $\mathrm{acc}_\cdot(k)$ is per-concept accuracy, $N_Q$ is number of shared questions, $N_K$ is number of shared concepts, $f$ is co-occurrence frequency on $q_0$, $c$ is a decay constant.

Student similarity is measured by Mahalanobis distance $$d(u,s)=\sqrt{(z_{u,s}-\mu)^\top\Sigma^{-1}(z_{u,s}-\mu)}$$ (parameters estimated from the population), with $\mathrm{Sim}(u,s)=1/(1+d(u,s))$. Top-S most similar students are selected to yield a context pool $\mathcal{C}_u = \bigcup_s H_s$, where $H_s$ is the full historical trajectory of $s$ (Duan et al., 19 Nov 2025).

5. Structured Prompt Engineering and Explainable Prediction

HISE-KT leverages a structured prompt which concatenates:

• Target student summary: student ID, ability $\theta_u$, interaction history $H_u$,
• Target question summary: question ID, concept, difficulty, discrimination, prior student accuracy,
• Similar-students context: for each Top-S student $s$, their ability, history on $k^*$, and accuracy.

The full prompt ends with an instruction:

Based on the above, predict: 1. Will student $<$u_id$>$ answer $<$q_0$>$ correctly? (correct/wrong) with probability. 2. Provide a three-sentence analysis citing evidence from $H_u$ and $\mathcal{C}_u$.

The LLM produces both a point prediction and an explanation referencing evidence paths and similar students, thereby coupling performance and interpretability. The design enforces zero-shot generality and supports automated, evidence-citing explanations.

6. Complete Model Workflow

The HISE-KT pipeline proceeds as follows:

1. Construct $G(U,Q,K,A,D; \mathcal{R})$ from dataset and IRT calculations.
2. For each $(q_0,\phi)$: a. Enumerate (sample) $N$ path instances, b. For each path, query LLM for scores $C_q$, $R_{KC}$, $I$, $D$ to obtain $S(p)$, c. Retain Top-K by $S(p)$.
3. Aggregate all students $U$ from retained paths.
4. For each candidate $s$: compute $z_{u,s}$, $d(u,s)$. Select Top-S $s$ for context.
5. Compile the prompt with $H_u$, $\mathcal{C}_u$, $q_0$ meta-data. Call LLM for prediction and explanation.

This systematic workflow supports joint optimization of both knowledge-tracing accuracy and interpretability, unifying the strengths of HIN modeling and LLM-based reasoning (Duan et al., 19 Nov 2025).

7. Empirical Performance and Interpretability

HISE-KT was evaluated on four public datasets (Assistment09, Slepemapy, Statics2011, Frcsub). Table 1 displays peak AUC results for HISE-KT (Qwen variant) and leading baselines:

Dataset	HISE-KT_Qwen AUC	Best Previous Baseline (Method, AUC)
Assistment09	0.8703	CoKT 0.8211
Slepemapy	0.9749	STHKT 0.8574
Statics2011	0.8888	TCL4KT 0.8357
Frcsub	0.9482	CoKT 0.9238

Accuracy improvements reach up to +9%. Interpretability, judged via human assessments and path-citation metrics, consistently exceeded all baselines. Explanatory outputs explicitly cited relevant meta-paths and student trajectories. For example, in Assistment09, the output LLM explanation referenced both meta-path evidence and similar peers’ incorrect answers to motivate its prediction (wrong, $P=0.72$ for the target item) (Duan et al., 19 Nov 2025).

8. Context and Comparison to Related Work

SINKT (Fu et al., 2024) also deploys a heterogeneous graph and LLM-based message-passing, but focuses primarily on student-inductive generalization and relies on LLMs for semantic initialization and graph expansion, with predictions realized through machine-learned encoders rather than prompt-driven explanation. In contrast, HISE-KT systematically integrates LLMs for both meta-path instance selection and final explanatory prediction, and introduces automated, fine-grained path scoring and student-similarity context aggregation. This suggests a broader applicability in environments where explainable, evidence-grounded predictions are requisite.

A plausible implication is that the HISE-KT paradigm can be extended to other domains where HINs and LLMs can be co-optimized for both prediction and interpretability, especially in educational recommender and adaptive tutoring systems. Current results represent a significant development in LLM-assisted, interpretable, evidence-backed knowledge tracing (Duan et al., 19 Nov 2025, Fu et al., 2024).