Dual-Cluster Memory Agent: Resolving Multi-Paradigm Ambiguity in Optimization Problem Solving

Abstract: LLMs often struggle with structural ambiguity in optimization problems, where a single problem admits multiple related but conflicting modeling paradigms, hindering effective solution generation. To address this, we propose Dual-Cluster Memory Agent (DCM-Agent) to enhance performance by leveraging historical solutions in a training-free manner. Central to this is Dual-Cluster Memory Construction. This agent assigns historical solutions to modeling and coding clusters, then distills each cluster's content into three structured types: Approach, Checklist, and Pitfall. This process derives generalizable guidance knowledge. Furthermore, this agent introduces Memory-augmented Inference to dynamically navigate solution paths, detect and repair errors, and adaptively switch reasoning paths with structured knowledge. The experiments across seven optimization benchmarks demonstrate that DCM-Agent achieves an average performance improvement of 11%- 21%. Notably, our analysis reveals a ``knowledge inheritance'' phenomenon: memory constructed by larger models can guide smaller models toward superior performance, highlighting the framework's scalability and efficiency.

• The paper introduces a dual-cluster memory framework that decouples modeling and coding experiences to resolve multi-paradigm ambiguity in optimization.
• It employs a bipartite graph linking modeling strategies and coding instantiations, enabling flexible retrieval and targeted error repair.
• Experimental results demonstrate an 11–21% accuracy improvement over benchmarks, with significant gains for smaller LLMs through knowledge inheritance.

Dual-Cluster Memory Agent: Disentangling Multi-Paradigm Ambiguity in Optimization Problem Solving

Motivation and Problem Formulation

LLMs have demonstrated remarkable proficiency in the automatic modeling and solution of optimization problems. However, a foundational challenge persists: structural ambiguity arises when a single problem admits multiple, and often conflicting, modeling paradigms such as CP, ILP, or DP. This ambiguity induces cognitive interference in LLMs, resulting in either the mechanical application of memorized templates or the selection of suboptimal solution paths, and ultimately high error rates. Existing fine-tuning or agentic frameworks remain limited—they cannot adaptively navigate paradigm ambiguity nor robustly verify paradigm-specific correctness due to the reliance on static prompts or parameter updates. The core challenge is therefore to build reasoning architectures capable of dynamically detecting, verifying, and switching between algorithmic paradigms, ensuring both flexibility and paradigm-specific rigor.

Figure 1: Illustration of a single production planning problem formalized via distinct algorithmic paradigms.

Dual-Cluster Memory Agent Framework

The Dual-Cluster Memory Agent (DCM-Agent) resolves multi-paradigm ambiguity through an explicit decoupling of modeling and coding experience, formalized as two distinct memory architectures: the Modeling Cluster and the Coding Cluster. Experience nodes (problem-solution trajectories) are bifurcated and stratified according to their solution status (always correct, recovered, persistent failure). Each node is then atomized into three structured guidance tiers: Approach (canonical solution strategies), Checklist (verification criteria), and Pitfall (common error modalities). Through bottom-up clustering, clusters in both modeling and coding space are synthesized by semantic similarity and LLM-verification, and continuously abstracted as generalized cluster knowledge.

A bipartite graph is constructed which encodes statistically weighted associations between Modeling and Coding Clusters, leveraging observed compatibilities in historical solution data. This dual-cluster bipartite structure captures the natural, but historically ambiguous, mapping from mathematical modeling logic to solver code instantiation, facilitating flexible recombination and hierarchical knowledge re-use.

Figure 2: Overview of the Dual-Cluster Memory Agent (DCM-Agent): construction process (memory stratification, clustering, abstraction) and inference pipeline (dual-retrieval, path synthesis, knowledge-guided solution).

Two qualitative case studies from the memory illustrate the cluster abstraction process and divergent error modes captured (e.g., integrality gaps vs. recurrence failures).

Figure 3: Two representative historical solution clusters within DCM-Agent's memory system.

Memory-Augmented Inference

DCM-Agent performs inference with a two-level dual-retrieval scheme. Given a new problem, embeddings are used to retrieve both exemplar nodes and generalized modeling clusters. These retrieved modeling clusters are expanded via the bipartite association graph to enumerate feasible coding clusters, yielding a ranked candidate set of modeling-coding paths.

The solution search pipeline then executes each path iteratively, strictly conditioning generation and verification on the associated cluster knowledge rather than node-specific information. Solutions are generated and verified by approach and checklist guidance; on code execution error, targeted repair is initiated via pitfall guidance. If repair is unsuccessful, systematic backtracking through the candidate path set is conducted, ensuring both coverage and robustness.

Experimental Evaluation

Experiments were conducted on seven diverse optimization benchmarks (NL4Opt, NLP4LP, OptiBench, OptMATH, ComplexLP, IndustryOR, ComplexOR) using multiple LLMs (Qwen3 series, DeepSeek, GPT-5.1). DCM-Agent exhibits 11–21% absolute improvements in end-to-end solving accuracy over strong multi-agent, search-based, and retrieval-augmented competitors at all model scales. Notably, the margin is accentuated for smaller models, highlighting DCM-Agent's ability to amortize reasoning deficiencies via external memory.

DCM-Agent maintains computational cost comparable to other retrieval-augmented methods, while dramatically outperforming exhaustive tree search algorithms such as AF-MCTS in efficiency.

Robustness and ablation studies empirically validate that both Modeling and Coding Clusters are necessary, with modeling cluster removal causing a pronounced drop in performance, confirming that fine-grained mathematical logic abstraction is the critical bottleneck.

Memory Construction and Knowledge Inheritance Phenomena

A key property revealed is "knowledge inheritance": cluster memories constructed by larger LLMs can be effectively transferred to guide smaller LLMs, resulting in up to 25–40% accuracy improvements for weak models. Performance degrades only when cluster density from exceptionally strong models exceeds the smaller model's processing capacity. Increasing the historical memory budget (i.e., more clustered experiences) provides monotonic gains, confirming the generalization benefits of broader experience curation.

Cluster count statistics show that advanced LLMs form more fine-grained modeling clusters, further substantiating the correlation between model scale and abstraction granularity.

Figure 4: Distribution of modeling and coding cluster cardinalities yielded by LLMs of varying sizes.

Detailed Analysis and Ablation

Ablation on cluster guidance, retrieval breadth ($K$), cluster update thresholds ($N$), and candidate path number ($M$) reveals that solution quality peaks under moderate cluster granularity and retrieval scope, while excessive values induce context overload. Case studies underscore the agent's ability to enforce subtle modeling requirements such as integrality—baseline LLMs often generate spurious yet numerically superior fractional solutions, whereas DCM-Agent applies correct constraints.

Figure 5: Ablation study—accuracies resulting from the removal of dual-cluster components and parameter variants.

Figure 6: Discrete optimization solution: baseline's infeasible fractional output vs. DCM-Agent's integral, physically valid solution.

Implications and Future Directions

DCM-Agent shifts the paradigm in LLM-based optimization modeling from static parameter reliance or template fitting to externalized, dynamically evolving reasoning archives. The stratification of knowledge into abstract modeling and concrete coding domains, coupled with explicit error mode documentation, enables both robust paradigm switching and high-fidelity solution verification. The training-free logic and memory transferability enable efficient adaptation to deployed, resource-constrained LLMs.

From a broader standpoint, DCM-Agent’s dual-cluster architecture is readily extensible to other domains characterized by entangled reasoning modalities, and its bipartite graph-based knowledge organization potentially facilitates self-supervised continual memory evolution, meta-reasoning, and integration with human-in-the-loop correction protocols.

Conclusion

The Dual-Cluster Memory Agent provides a rigorous, computationally efficient approach for resolving multi-paradigm ambiguity in automated optimization problem solving. By decoupling modeling and coding experiences, stratifying guidance, and explicitly modeling cross-domain associations, DCM-Agent achieves robust, state-of-the-art performance across diverse LLMs and benchmarks. The empirical identification of knowledge inheritance underpins future developments in scalable, memory-augmented general reasoning architectures for AI.

Reference: "Dual-Cluster Memory Agent: Resolving Multi-Paradigm Ambiguity in Optimization Problem Solving" (2604.20183).