RPAM: Merging Reasoning Patterns

• RPAM is a framework that leverages parameter-space merging to control and blend reasoning patterns in neural models.
• It employs techniques like interpolation, Fisher masking, and graph alignment to optimize reasoning depth, efficiency, and domain specificity.
• RPAM enables adaptive integration of meta-abilities and domain features, significantly reducing computational costs while preserving critical reasoning traits.

Reasoning Pattern Alignment Merging (RPAM) refers to a suite of frameworks and algorithms developed to address the integration, adaptation, and controllability of reasoning behaviors in neural models—particularly LLMs, multimodal transformers, and provenance-graph-based reasoners—by explicit alignment and fusion of reasoning patterns in parameter space. Rather than relying on retraining or prompt engineering, RPAM postulates that reasoning depth, style, and domain specificity can be tuned, blended, or preserved by principled model merging. These methods draw on a range of mathematical tools (feature alignment, Fisher information, task-vector interpolation, and pattern-guided graph alignment) to yield merged models that exhibit target reasoning traits, adaptive behavior, or multimodal/attack scenario integration.

1. Formalization and Motivations

RPAM addresses the challenge that powerful models trained for advanced reasoning (long-chain-of-thought, meta-abilities, domain-specific analysis) tend to incur higher computational costs or lose domain/perceptual fidelity when fused naïvely. The goal is to produce a merged model that retains or adaptively blends the desirable reasoning patterns of specialized parents, while minimizing degradation in efficiency, language, or domain performance (Zhong et al., 7 Jan 2026, Lan et al., 26 Sep 2025, Hu et al., 15 May 2025, Yang et al., 5 Aug 2025).

The generic formalism is: given parent models (or weights configurations) $\{\theta_i\}$, each exhibiting a target reasoning pattern (e.g., deduction, Long-CoT, domain-specificity), find a parameter set $\theta_{\mathrm{merge}}$ such that, for critical inputs or tasks, the model expresses the optimal combination of parent reasoning patterns. This paradigm eschews expensive retraining and allows "zero-shot" tuning of reasoning depth, query adaptivity, or meta-ability coverage.

2. Core Merging Algorithms and Mechanisms

Several algorithmic variants instantiate RPAM across domains:

• Weight Interpolation (Arithmetic, SLERP): Baseline approaches involve direct or spherical interpolation between parameter sets, possibly relative to a shared base $\theta_{\text{base}}$:

$$w_{\mathrm{merged}}(\alpha) = (1 - \alpha)\theta_{\text{direct}} + \alpha \theta_{\text{think}}$$

or per-layer/task-vector interpolation using

$$w_{\mathrm{merged}}(\alpha) = \theta_{\text{base}} + (1-\alpha)\Delta_{\text{direct}} + \alpha\Delta_{\text{think}}$$

These produce smooth transitions from fast, shallow reasoning to deep, complex reasoning. The merging strength $\alpha$ is swept to chart accuracy-efficiency trade-offs (Lan et al., 26 Sep 2025).

• Layer-wise and Ability-aware RPAM: RPAM is extended to layer-wise merging with per-layer coefficients $\alpha^l$ or blockwise policies (early/late layer splits), facilitating preservation of low-level (e.g., language) features while blending higher-level reasoning (Pipatanakul et al., 13 Feb 2025, Zhong et al., 7 Jan 2026). In ability-aware scenarios, coefficients may decay or be learned for maximal retention of language performance or target reasoning ability.
• Feature Alignment and Contrastive Objectives: Adaptive RPAM formulates the per-layer merge formula as

$$h_M^{(l)} = \alpha_L^l h_L^{(l)} + \alpha_S^l h_S^{(l)}$$

and learns $\alpha^l$ so that $h_M^{(l)}$ reproduces the positive parent (e.g., Long-CoT or Short-CoT) as measured on a pattern-labeled calibration set, while being contrastively pushed from the negative parent (Zhong et al., 7 Jan 2026).

• Fisher-based and Masked Merging: In domain-reasoning fusion, RPAM uses the Fisher Information Matrix (FIM) of the reasoning model as a prior to block most parameter movement away from high-curvature reasoning directions, while permitting domain-specific drift elsewhere. The merging mask $M_i$ identifies which weights may be shifted without disrupting reasoning patterns (Yang et al., 5 Aug 2025).
• Multi-parent Linear Merging: For compositional reasoning alignment, parameter-space merging can generalize to an unrestricted linear combination of specialists:

$$\Theta_{\text{merge}} = \lambda_d \Theta^{(d)} + \lambda_i \Theta^{(i)} + \lambda_a \Theta^{(a)}$$

for deduction/induction/abduction specialists, e.g., with $\lambda$'s tuned on development performance (Hu et al., 15 May 2025).

• Pattern-guided Graph Alignment (Provenance Analysis): In the context of cyber reasoning over provenance graphs, RPAM denotes the process of aligning mined tactic/technique sequential patterns to compressed anomaly subgraphs, then scoring and merging high-confidence paths to reconstruct concise, semantically accurate attack scenarios (Sheng, 25 Oct 2025).

3. Calibration, Optimization, and Pattern-Selection

RPAM implementations utilize auxiliary calibration sets or pattern labels to optimize the blending strategy:

• Pattern-Labeled Calibration for Query Adaptivity: A small calibration set is constructed to identify for each query the optimal parent (long vs. short chain-of-thought), allowing layer-wise or element-wise coefficients to be learned so the merged model can express either style depending on input (Zhong et al., 7 Jan 2026).
• Self-verifiable Synthetic Tasks for Meta-Ability Specialists: For meta-ability alignment (deduction, induction, abduction), synthetic training with automatic checking allows for the production of specialist models, which are then merged in parameter space, with blending coefficients chosen via validation (Hu et al., 15 May 2025).
• PL-set Size and Hyperparameter Sensitivity: Empirical results confirm that even small calibration sets (N=32–256) suffice for near-optimal alignment, and tuning of merging coefficients is not computationally intensive compared to full retraining (Zhong et al., 7 Jan 2026, Pipatanakul et al., 13 Feb 2025).

4. Empirical Outcomes and Benchmark Findings

RPAM has been validated across LLMs, multimodal transformers, and provenance reasoning systems. Key findings include:

• Tunable Accuracy-Efficiency Frontiers: RPAM-swept models produce a Pareto frontier between reasoning accuracy and computational cost (tokens generated), with "phase changes" at predictable $\alpha$ thresholds where complex reasoning suddenly activates (Lan et al., 26 Sep 2025, Zhong et al., 7 Jan 2026).
• Adaptive Reasoning with Minimal Retraining: Adaptive RPAM models achieve substantial reductions in average inference cost (40–65% fewer tokens) while preserving $>95\%$ of long-chain-of-thought accuracy. On Qwen3-4B, RPAM yields 75.9% accuracy at 48% of the computational cost of Long-CoT (Zhong et al., 7 Jan 2026).
• Domain and Multilingual Integration: For low-resource LLMs (e.g., Thai), RPAM achieves both high reasoning accuracy and strong language/task fidelity by balancing deep-reasoning weights in early layers and preserving language-specific content in later layers (Pipatanakul et al., 13 Feb 2025).
• Preservation of Critical Reasoning Patterns: Fisher-masked RPAM merging drastically reduces output degradation (gibberish rate drops from $>50\%$ to 14.3%) and outperforms baseline merges by 9.5–12% in domain accuracy (Yang et al., 5 Aug 2025).
• Multimodal Restoration and Grounding: Plateau-guided RPAM (PlaM) for multimodal models restores text reasoning in late layers by merging vision-LLM weights with base LLM weights only after modal alignment occurs, yielding up to 22.86 absolute gains in visual-language benchmarks (Wang et al., 12 Jan 2026).
• Graph Reasoning Scenario Merging: In provenance/attack scenario analysis, RPAM-aligned merging of anomaly subgraph paths with tactic/technique patterns achieves 99.9% graph simplification, 91% critical node preservation, and F1 up to 64%—significantly exceeding prior methods (Sheng, 25 Oct 2025).

5. Design Choices, Limitations, and Practical Guidelines

RPAM is subject to specific design decisions and certain fundamental constraints:

• Identical Architecture Requirement: Most RPAM approaches require that all parent networks share identical architectures and vocabulary to enable parameter-wise interpolation or merging (Hu et al., 15 May 2025, Zhong et al., 7 Jan 2026).
• Hyperparameter and Mask Tuning: Performance is sensitive to merging strength ($\alpha$, $\lambda$), pattern-label set size, and mask thresholding (e.g., Fisher-based importance weights). Tuning is done per-architecture or per-domain (Yang et al., 5 Aug 2025, Zhong et al., 7 Jan 2026).
• Diagonal Approximations: Some variants rely on diagonal Fisher Information, neglecting cross-parameter dependencies. Extending RPAM to richer curvature or blockwise alignment is a stated direction (Yang et al., 5 Aug 2025).
• No Optimizer/Memory Merge: Only raw model weights are merged; optimizer states and RL memories are excluded in these paradigms (Hu et al., 15 May 2025).
• Computational Efficiency: RPAM merging and calibration typically require orders of magnitude less computation than retraining, often scaling with a few validation sweeps or <1 h per model (Zhong et al., 7 Jan 2026, Pipatanakul et al., 13 Feb 2025).

6. Generalizations, Cross-Domain Extensions, and Future Directions

RPAM has been extended and conjectured as broadly applicable:

• Multimodal and Cross-pattern Merging: Extensions to vision-LLMs (Wang et al., 12 Jan 2026, Chen et al., 8 May 2025), meta-abilities beyond the canonical reasoning triad, and multi-domain/mixture-of-experts architectures are suggested (Hu et al., 15 May 2025, Yang et al., 5 Aug 2025).
• Dynamic and Fine-grained Adaptivity: Future work includes dynamic per-token or per-query merging coefficients, richer activation/attention-based alignment, blockwise multi-domain fusions, and meta-learned parameter policies (Yang et al., 5 Aug 2025, Zhong et al., 7 Jan 2026).
• Pattern-guided Graph Reasoning: RPAM provides a blueprint for aligning complex, real-world sequential patterns (e.g., tactics/techniques in security graphs) directly to extracted graph paths, guiding scenario reconstruction with interpretable, confidence-scored merges (Sheng, 25 Oct 2025).
• Meta-Ability and Task-General Merging: The parameter-space merging paradigm has demonstrated that merged models can inherit complementary behaviors—suggesting that even more meta-abilities, inference modes, and cross-modal traits could be captured by convex or structured combinations in parameter space (Hu et al., 15 May 2025, Chen et al., 8 May 2025).

7. Summary Table of Key RPAM Techniques

Technique / Variant	Domain	Procedure/Criteria
Arithmetic/SLERP interpolations	LLM reasoning	Linear/spherical blend pos. base models
Fisher-masked (MAP) merging	Reasoning + domain LLMs	Masked update, preserve reasoning via FIM
Feature alignment (contrastive)	Adaptive LLM reasoning	Learn per-layer merge α via PL precedents
Layer-split/decay merging	Multilingual, multimodal	Decay/increase α across low/high layers
Multi-ability linear merge	Meta-abilities (ded/ind/abd)	Weighted convex sum of specialist weights
Plateau-guided layer merge	Vision-language (MLLM)	Detect modal alignment, merge after plateau
Pattern-aligned path merging	Provenance (APT) graphs	CTI-mined pattern graph-path alignment

RPAM frameworks have established that reasoning style, depth, and specificity can be robustly controlled, adaptively blended, or preserved across diverse neural architectures and reasoning contexts by principled parameter-space merging grounded in feature and pattern alignment (Zhong et al., 7 Jan 2026, Lan et al., 26 Sep 2025, Yang et al., 5 Aug 2025, Hu et al., 15 May 2025, Pipatanakul et al., 13 Feb 2025, Sheng, 25 Oct 2025, Wang et al., 12 Jan 2026, Chen et al., 8 May 2025).