PMFR: Fast, Adaptive Dialogue Architecture

• PMFR Architecture is a temporal decoupling framework for dialogue systems that separates fast response generation from asynchronous knowledge refinement.
• It employs a three-module design with a Knowledge Adequacy Evaluator, Lightweight Response Generator, and Asynchronous Knowledge Refinement Agent to balance latency and quality.
• Empirical results show PMFR achieves comparable quality to large models while reducing latency by approximately 95.3%, ensuring robust user interaction.

The PMFR architecture (Prepared Mind, Fast Response) is a temporal decoupling framework for adaptive knowledge orchestration in open-domain dialogue systems. It aims to reconcile the latency-quality tradeoff encountered in conversational AI by combining rapid, always-on user interaction with asynchronous, knowledge-intensive processing. This separation enables PMFR to achieve sub-second response latency while maintaining response quality comparable to much larger and slower tool-augmented agents (Gan et al., 9 Oct 2025).

1. System Architecture and Functional Decomposition

PMFR is structured around three tightly coordinated core modules:

• Knowledge Adequacy Evaluator ($\mathcal{E}$): Assesses whether the existing knowledge base ($K_t$) at turn $t$ suffices to answer the user query ($q_t$) and history ($H_{t-1}$). It outputs a binary adequacy signal $s_t \in \{0,1\}$ along with a reformulated query $\widetilde q_t$ for downstream retrieval.
• Lightweight Response Generator ($G$): Utilizes an instruction-tuned 4B parameter model (Qwen3-4B), producing immediate responses either as fully grounded answers (if $s_t=0$) or as brief transitional replies (“holding” messages) when background retrieval is triggered.
• Asynchronous Knowledge Refinement Agent ($A$): Invoked only on $K_t$0 (KB-Miss), this heavyweight ReAct-style agent (Qwen3-235B with Chain-of-Thought) retrieves, reasons over, and synopsizes new external evidence in background threads, incrementally updating $K_t$1 for future turns.

The high-level dataflow is captured below: $\widetilde q_t$6 This decoupling ensures conversational flow is never blocked by slow retrieval.

2. Component Design and Gating Mechanism

2.1 Knowledge Adequacy Evaluator

• Input: $K_t$2.
• Decision: $K_t$3, where $K_t$4 signifies a knowledge-base “hit” and $K_t$5 a “miss”.
• Formulation: Implicitly modeled as a scoring function:

$K_t$6

$K_t$7 is set to $K_t$8 if Score $K_t$9, otherwise $t$0.

• Query Reformulation: Improves retrieval accuracy:

$t$1

2.2 Lightweight Response Generator ($t$2)

• Direct Mode: On KB-Hit ($t$3), $t$4 generates a fully grounded response via a single forward pass, with deterministic decoding and $t$51 s latency.
• Transition Mode: On KB-Miss ($t$6), $t$7 provides a short, user-friendly reply (“Let me check...”) to maintain interaction fluidity while $t$8 is running.
• Model Backbone: Qwen3-4B, optimized for real-time, edge deployment.

2.3 Asynchronous Knowledge Refinement Agent ($t$9)

• Trigger: Invoked exclusively when $q_t$0.
• Pipeline:

1. Knowledge Acquisition: Uses $q_t$1 to retrieve external sources (web APIs, document repositories, KBs).
2. Evidence Reasoning: Employs Chain-of-Thought to synthesize and disambiguate facts.
3. Synopsis & Caching: Produces confidence-weighted, provenance-tagged summaries, updating $q_t$2 asynchronously for later use.

• Model Backbone: Qwen3-235B, running only in background threads.

3. Temporal Decoupling and Update Policy

PMFR explicitly factorizes dialogue response as follows:

• Fast path: $q_t$3 sent immediately via $q_t$4
• Asynchronous path: $q_t$5 performed by $q_t$6 only on demand.

The gating function $q_t$7 dictates when background refinement is triggered. Multiple background updates may queue across turns, but only the most recent KB is surfaced at the next user interaction.

4. Turn-by-Turn Workflow

A typical PMFR dialogue turn proceeds as:

1. Query Intake: User sends $q_t$8; $q_t$9 and $H_{t-1}$0 supplied to $H_{t-1}$1.
2. Adequacy Check: $H_{t-1}$2 computes $H_{t-1}$3, produces $H_{t-1}$4.
3. Fast Response Path:
   • If $H_{t-1}$5 (KB-Hit): $H_{t-1}$6 generates full answer $H_{t-1}$7.
   • If $H_{t-1}$8 (KB-Miss): $H_{t-1}$9 issues a transition reply $s_t \in \{0,1\}$0.
4. Async Retrieval Path: For $s_t \in \{0,1\}$1, $s_t \in \{0,1\}$2 is launched in background with $s_t \in \{0,1\}$3; $s_t \in \{0,1\}$4 is updated and will be available in subsequent turns.
5. Response Delivery: User receives $s_t \in \{0,1\}$5 with sub-second latency; knowledge coverage increases adaptively over subsequent turns.

5. Quality, Latency, and Pareto Performance

Empirical results on TopiOCQA validate the efficacy of PMFR’s temporal decoupling strategy:

Method	GEval-C	Latency (s)	P95 Latency (s)
Qwen-4B (ins., no tools)	0.481	1.155	1.844
Qwen-4B (CoT, no tools)	0.511	8.710	20.137
ReAct (Qwen-4B, CoT)	0.460	13.668	28.515
ReAct (Qwen-235B, CoT)	0.620	23.375	49.443
PMFR (Ours)	0.613	1.090	1.810

• Latency Reduction: PMFR achieves a mean response latency of $s_t \in \{0,1\}$6 s versus $s_t \in \{0,1\}$7 s for synchronous ReAct agents ($s_t \in \{0,1\}$8 reduction).
• Quality Retention: PMFR reaches a GEval-C score of $s_t \in \{0,1\}$9—indistinguishable from the $\widetilde q_t$0 achieved by the 235B ReAct agent, despite using fast lightweight models for most turns.
• Stability: $\widetilde q_t$1th percentile latencies remain tightly bounded ($\widetilde q_t$2 s for PMFR) compared to $\widetilde q_t$3 s for synchronous agents, enabling robust user experience (Gan et al., 9 Oct 2025).

6. Discussion: Strengths, Tradeoffs, and Future Directions

Key advantages:

• The temporal split prevents conversational stalls by decoupling slow retrieval/tool use from user interaction.
• The gating mechanism ensures that external retrieval is performed only when essential, mitigating resource overhead.
• Model heterogeneity capitalizes on the respective strengths of small (latency) and large (reasoning depth) models.

Identified limitations:

• Hard binary gating ($\widetilde q_t$4) can lead to under- or over-triggering, affecting completeness or efficiency.
• Asynchronous updates create brief windows where new knowledge is not instantly reflected in responses.
• The system complexity increases due to concurrent background jobs, dynamic caching, and inter-model orchestration.

Potential improvements:

• Transition from binary to learnable, continuous adequacy scoring for smoother retrieval triggering.
• Reinforcement learning to fine-tune the gating threshold $\widetilde q_t$5 and synopsis caching strategy.
• Integration of real-time knowledge graphs or multimodal retrieval.
• Closed-loop user feedback mechanisms for online adaptation and error correction (Gan et al., 9 Oct 2025).

PMFR represents a substantive architectural advancement in dialogue AI by achieving near-optimal response quality at real-time latencies, leveraging temporal decoupling, asynchronous knowledge refinement, and dynamic model selection.