QA Pair Reassembler

• QA Pair Reassembler is a system that reorganizes isolated QA pairs using knowledge graph algorithms to create semantically coherent dialogue sequences.
• It employs triple extraction, mapping to knowledge graph nodes, and heuristic-guided graph traversal to simulate multi-turn conversations from single-turn data.
• Empirical results show that models pre-trained with reassembled data achieve significant improvements in F1 and coherence on conversational QA benchmarks.

A QA Pair Reassembler is a system or module that organizes isolated question–answer (QA) pairs—typically generated or extracted from documents—into coherent, sequential multi-turn dialogues by leveraging knowledge graph-based algorithms and structural mapping techniques. The QA Pair Reassembler is most prominently instantiated as the second stage in frameworks such as S2M ("Single-turn to Multi-turn") for conversational question answering (CQA), where it reformulates standalone QA pairs from single-turn datasets (e.g., SQuAD) into logical, dialogue-style conversational QA sequences suitable for training multi-turn CQA models (Li et al., 2023).

1. Function in Conversational QA Frameworks

The principal function of a QA Pair Reassembler is to transform an unordered or independent collection of QA pairs—extracted or generated from a context—into a semantically ordered sequence that captures natural conversational flow. In the S2M framework, the Reassembler mediates the conversion of a single-turn QA corpus into a simulated dialogue dataset by:

• Ingesting a context passage and its associated QA pairs (both original and model-generated),
• Establishing a latent structure encoding the relationships among factual statements, and
• Sequencing the QA pairs along this structure to mimic the dependency and progression of questions and answers in real-world conversations.

This approach specifically addresses the performance gap (distribution divergence) observed when training CQA models exclusively on single-turn data, refining synthetic data for multi-turn training.

2. Knowledge Graph Construction and Triples Join Algorithm

Central to the QA Pair Reassembler's operation is the induction of a document-specific knowledge graph that formalizes semantic and factual dependencies between sentences and clauses:

• Triple Extraction: Each sentence in the context, $C = [s_1, s_2, ..., s_N]$, is processed by OpenIE6 to generate a set of $(subject, relation, object)$ triples, $T = [T_1, T_2, ..., T_N]$.
• Triples Join Algorithm: The algorithm traverses these triples, connecting them according to three principles:
  1. Semantic Overlap: Link triples where subjects/objects match or are nested.
  2. Adjacency: Fill in edges between previously unconnected triples in adjacent sentences, maintaining topical coherence.
  3. Inter-Sentence Bridging: Ensure global connectivity across the context by linking terminal triples of one sentence to initial triples of the next.

Formally, the process is documented as:

\begin{algorithm}[tb] \caption{Triple Join Algorithm} \label{alg:triple join} Input: %%%%3%%%% Parameter: %%%%4%%%% is a sentence of context %%%%5%%%%. %%%%6%%%% is the number of sentences in the context %%%%7%%%% Output: Knowlege Graph %%%%8%%%% \begin{algorithmic}[1] \STATE Initialize Graph %%%%9%%%% = dict() \STATE Initialize Triples %%%%10%%%% = list() \FOR{%%%%11%%%% in %%%%12%%%%} \STATE %%%%13%%%%.append(OpenIE6(%%%%14%%%%)) \ENDFOR % Intra-Sentence Level Connections \FOR{%%%%15%%%% in %%%%16%%%%} \IF{%%%%17%%%% satisfy Principle 1} \STATE connect(%%%%18%%%%) \ENDIF \ENDFOR % Principle 2 and 3 omitted for brevity \STATE return %%%%19%%%% \end{algorithmic} \end{algorithm}

This results in a graph $G$ where nodes represent contextual entities/events and edges encode relationships, laying the groundwork for sequential dialogue synthesis.

3. Mapping QA Pairs to Knowledge Graph Nodes

After constructing the knowledge graph, each QA pair—represented by its principal triple(s) as extracted via OpenIE6—is mapped to graph nodes:

• Each pair's principal fact (or desired information need) is matched against entities/relations in the knowledge graph.

• This mapping enables the marking and subsequent identification of nodes that are both covered by the document and engaged by the QA pairs, including both original and generated pairs.

This alignment ensures that each QA pair's content directly corresponds to a precise semantic entity or relation within the underlying document structure.

4. Dialogue Assembly via Graph-Guided Traversal and Heuristics

Sequential QA pair assembly proceeds by traversing the knowledge graph:

• Root Selection: Initiate traversal at a graph root, frequently corresponding to a “starting fact” or a key background entity.
• Node Traversal and Substitution: For each node encountered along the traversal path, the system inserts its corresponding QA pair. If a node is masked as a conversational topic, the system dynamically replaces it with the appropriate QA pair, thus generating a “turn” in the synthetic conversation.
• Redundancy and Continuity Management: Heuristics ensure central or frequently-referenced nodes are prioritized and repetitive QA pairs (from generation artifacts or overlapping facts) are merged or eliminated. Dialogue construction halts on detecting discontinuities or excessive unanswerable QA pairs (e.g., >3).

The resultant ordered QA sequence thus mirrors the semantic and causal progressions salient in the original text.

5. Bridging the Single-Turn to Multi-Turn Data Gap

The QA Pair Reassembler directly addresses the distribution mismatch between single-turn and multi-turn QA datasets:

• Single-turn pairs lack inter-question dependency and context carryover.
• By aligning QA pairs along a knowledge graph traversal, each question is anchored in the context of previous turns, yielding dialogic coherence and simulating real multi-turn QA sessions.

Empirical studies validate that models pre-trained with S2M Reassembler-generated data achieve substantial F1 and HEQ improvements on Conversational QA benchmarks (QuAC), both in unsupervised and supervised scenarios (Li et al., 2023). This substantiates the necessity of structural reassembly—beyond simple aggregation—for effective multi-turn modeling.

6. Quantitative and Qualitative Efficacy

Quantitative results indicate that models pre-trained (even with smaller synthetic data size) on S2M Reassembler outputs outperform alternatives using existing single-turn datasets (SQuAD, CoQA) in both unsupervised and fine-tuned settings. Key metrics include:

• F1 improvements, e.g., up to 59.2 (DEBERTA+S2M, unsupervised) vs. 13.2 (baseline),
• Enhanced overlap (F1) between questions and answers across dialogue turns,
• Higher human-rated adequacy, contextual relevance, and accuracy relative to earlier synthetic and even some manual datasets,
• Superior coherence and logical flow compared to surface-level or context-only synthetic sequencing.

This demonstrates that knowledge graph-based QA pair reassembly delivers structurally faithful and high-utility multi-turn training data, sensitive to both topical and semantic progression.

7. Applications, Limitations, and Extensions

The QA Pair Reassembler paradigm is most effective for:

• Augmenting conversational QA training when only single-turn QA sources are available,
• Enabling synthetic yet realistic dialogue creation across domains (with domain-adapted entity/relation extraction),
• Serving as a template for dialogic data creation in non-QA tasks where structural/hierarchical dependencies govern information flow.

Potential constraints include dependence on coverage and accuracy of triple extraction (OpenIE6 or equivalents), and diminished performance if the knowledge graph fails to recover the underlying semantic structure. Extensions could integrate more sophisticated relation extraction, contextual masking (for context-aware question rewriting), or model-driven traversal strategies.

---

In summary, a QA Pair Reassembler leverages document-derived knowledge graphs and principled traversal algorithms to reconstruct a collection of QA pairs into logically sequenced, multi-turn dialogues, thereby enabling robust augmentation of conversational QA data and bridging the inherent structural gap between single-turn and multi-turn resources (Li et al., 2023).