PMFR: Fast, Adaptive Dialogue Architecture

• PMFR Architecture is a temporal decoupling framework for dialogue systems that separates fast response generation from asynchronous knowledge refinement.
• It employs a three-module design with a Knowledge Adequacy Evaluator, Lightweight Response Generator, and Asynchronous Knowledge Refinement Agent to balance latency and quality.
• Empirical results show PMFR achieves comparable quality to large models while reducing latency by approximately 95.3%, ensuring robust user interaction.

The PMFR architecture (Prepared Mind, Fast Response) is a temporal decoupling framework for adaptive knowledge orchestration in open-domain dialogue systems. It aims to reconcile the latency-quality tradeoff encountered in conversational AI by combining rapid, always-on user interaction with asynchronous, knowledge-intensive processing. This separation enables PMFR to achieve sub-second response latency while maintaining response quality comparable to much larger and slower tool-augmented agents (Gan et al., 9 Oct 2025).

1. System Architecture and Functional Decomposition

PMFR is structured around three tightly coordinated core modules:

• Knowledge Adequacy Evaluator ($\mathcal{E}$): Assesses whether the existing knowledge base ($K_t$) at turn $t$ suffices to answer the user query ($q_t$) and history ($H_{t-1}$). It outputs a binary adequacy signal $s_t \in \{0,1\}$ along with a reformulated query $\widetilde q_t$ for downstream retrieval.
• Lightweight Response Generator ($G$): Utilizes an instruction-tuned 4B parameter model (Qwen3-4B), producing immediate responses either as fully grounded answers (if $s_t=0$) or as brief transitional replies (“holding” messages) when background retrieval is triggered.
• Asynchronous Knowledge Refinement Agent ($A$): Invoked only on $s_t=1$ (KB-Miss), this heavyweight ReAct-style agent (Qwen3-235B with Chain-of-Thought) retrieves, reasons over, and synopsizes new external evidence in background threads, incrementally updating $K_{t+1}$ for future turns.

The high-level dataflow is captured below:

[User Query q_t, History H_{t-1}, KB K_t] | v +-- Knowledge Adequacy Evaluator (𝔈) --+ | s_t, ~q_t | v v KB-Hit (s_t=0): KB-Miss (s_t=1): [G: fast answer r_t] +--> [G: polite placeholder r_t] | [A: async retrieval + reasoning → KB update]

This decoupling ensures conversational flow is never blocked by slow retrieval.

2. Component Design and Gating Mechanism

2.1 Knowledge Adequacy Evaluator

• Input: $(q_t, H_{t-1}, K_t)$.
• Decision: $s_t = \mathcal{E}(q_t, H_{t-1}, K_t) \in \{0,1\}$, where $0$ signifies a knowledge-base “hit” and $1$ a “miss”.
• Formulation: Implicitly modeled as a scoring function:

$$\text{Score}(q_t, H_{t-1}, K_t) = \sigma(W_s[\operatorname{emb}(q_t);\operatorname{emb}(H_{t-1});\operatorname{emb}(K_t)] + b_s)$$

$s_t$ is set to $1$ if Score $<\tau$, otherwise $0$.

• Query Reformulation: Improves retrieval accuracy:

$$\widetilde q_t = \mathrm{decode}\left(W_c \left[\operatorname{emb}(q_t); \operatorname{emb}(\widehat H_{t-1}) \right] + b_c\right)$$

2.2 Lightweight Response Generator ($G$)

• Direct Mode: On KB-Hit ($s_t=0$), $G$ generates a fully grounded response via a single forward pass, with deterministic decoding and $<$1 s latency.
• Transition Mode: On KB-Miss ($s_t=1$), $G$ provides a short, user-friendly reply (“Let me check...”) to maintain interaction fluidity while $A$ is running.
• Model Backbone: Qwen3-4B, optimized for real-time, edge deployment.

2.3 Asynchronous Knowledge Refinement Agent ($A$)

• Trigger: Invoked exclusively when $s_t=1$.
• Pipeline:

1. Knowledge Acquisition: Uses $\widetilde q_t$ to retrieve external sources (web APIs, document repositories, KBs).
2. Evidence Reasoning: Employs Chain-of-Thought to synthesize and disambiguate facts.
3. Synopsis & Caching: Produces confidence-weighted, provenance-tagged summaries, updating $K_{t+1}$ asynchronously for later use.

• Model Backbone: Qwen3-235B, running only in background threads.

3. Temporal Decoupling and Update Policy

PMFR explicitly factorizes dialogue response as follows:

• Fast path: $r_t = f_\text{fast}(q_t, H_{t-1}, K_t)$ sent immediately via $G$
• Asynchronous path: $$K_{t+1} = \mathrm{async}\{ f_\text{slow}(q_t, H_{t-1}, K_t) \}$$ performed by $A$ only on demand.

The gating function $T(q_t, H_{t-1}, K_t)=s_t$ dictates when background refinement is triggered. Multiple background updates may queue across turns, but only the most recent KB is surfaced at the next user interaction.

4. Turn-by-Turn Workflow

A typical PMFR dialogue turn proceeds as:

1. Query Intake: User sends $q_t$; $H_{t-1}$ and $K_t$ supplied to $\mathcal{E}$.
2. Adequacy Check: $\mathcal{E}$ computes $s_t$, produces $\widetilde q_t$.
3. Fast Response Path:
   • If $s_t = 0$ (KB-Hit): $G$ generates full answer $r_t$.
   • If $s_t = 1$ (KB-Miss): $G$ issues a transition reply $r_t$.
4. Async Retrieval Path: For $s_t = 1$, $A$ is launched in background with $\widetilde q_t$; $K_{t+1}$ is updated and will be available in subsequent turns.
5. Response Delivery: User receives $r_t$ with sub-second latency; knowledge coverage increases adaptively over subsequent turns.

5. Quality, Latency, and Pareto Performance

Empirical results on TopiOCQA validate the efficacy of PMFR’s temporal decoupling strategy:

Method	GEval-C	Latency (s)	P95 Latency (s)
Qwen-4B (ins., no tools)	0.481	1.155	1.844
Qwen-4B (CoT, no tools)	0.511	8.710	20.137
ReAct (Qwen-4B, CoT)	0.460	13.668	28.515
ReAct (Qwen-235B, CoT)	0.620	23.375	49.443
PMFR (Ours)	0.613	1.090	1.810

• Latency Reduction: PMFR achieves a mean response latency of $1.09$ s versus $23.38$ s for synchronous ReAct agents ($\approx 95.3\%$ reduction).
• Quality Retention: PMFR reaches a GEval-C score of $0.613$—indistinguishable from the $0.620$ achieved by the 235B ReAct agent, despite using fast lightweight models for most turns.
• Stability: $95$th percentile latencies remain tightly bounded ($1.81$ s for PMFR) compared to $49.44$ s for synchronous agents, enabling robust user experience (Gan et al., 9 Oct 2025).

6. Discussion: Strengths, Tradeoffs, and Future Directions

Key advantages:

• The temporal split prevents conversational stalls by decoupling slow retrieval/tool use from user interaction.
• The gating mechanism ensures that external retrieval is performed only when essential, mitigating resource overhead.
• Model heterogeneity capitalizes on the respective strengths of small (latency) and large (reasoning depth) models.

Identified limitations:

• Hard binary gating ($s_t$) can lead to under- or over-triggering, affecting completeness or efficiency.
• Asynchronous updates create brief windows where new knowledge is not instantly reflected in responses.
• The system complexity increases due to concurrent background jobs, dynamic caching, and inter-model orchestration.

Potential improvements:

• Transition from binary to learnable, continuous adequacy scoring for smoother retrieval triggering.
• Reinforcement learning to fine-tune the gating threshold $\tau$ and synopsis caching strategy.
• Integration of real-time knowledge graphs or multimodal retrieval.
• Closed-loop user feedback mechanisms for online adaptation and error correction (Gan et al., 9 Oct 2025).

PMFR represents a substantive architectural advancement in dialogue AI by achieving near-optimal response quality at real-time latencies, leveraging temporal decoupling, asynchronous knowledge refinement, and dynamic model selection.