FactorMiner: A Self-Evolving Agent with Skills and Experience Memory for Financial Alpha Discovery

Abstract: Formulaic alpha factor mining is a critical yet challenging task in quantitative investment, characterized by a vast search space and the need for domain-informed, interpretable signals. However, finding novel signals becomes increasingly difficult as the library grows due to high redundancy. We propose FactorMiner, a lightweight and flexible self-evolving agent framework designed to navigate this complex landscape through continuous knowledge accumulation. FactorMiner combines a Modular Skill Architecture that encapsulates systematic financial evaluation into executable tools with a structured Experience Memory that distills historical mining trials into actionable insights (successful patterns and failure constraints). By instantiating the Ralph Loop paradigm -- retrieve, generate, evaluate, and distill -- FactorMiner iteratively uses memory priors to guide exploration, reducing redundant search while focusing on promising directions. Experiments on multiple datasets across different assets and Markets show that FactorMiner constructs a diverse library of high-quality factors with competitive performance, while maintaining low redundancy among factors as the library scales. Overall, FactorMiner provides a practical approach to scalable discovery of interpretable formulaic alpha factors under the "Correlation Red Sea" constraint.

• The paper introduces a self-evolving agent that employs a structured experience memory to overcome signal redundancy and guide orthogonal alpha factor discovery.
• It details a modular skill pipeline that enforces stringent IC and correlation thresholds, achieving up to three times more high-quality candidates compared to memoryless methods.
• Empirical results demonstrate superior cross-market performance and computational efficiency, validating its applicability in high-compliance, high-frequency trading environments.

FactorMiner: Self-Evolving Agent Architecture for Formulaic Alpha Discovery

Motivation and Context

The construction of interpretable formulaic alpha factors remains a foundational task in quantitative finance, particularly in high-frequency trading regimes where signal redundancy escalates as libraries grow. Prevailing search paradigms—genetic programming, reinforcement learning, LLM-based proposal—fail to maintain persistent knowledge across sessions, leading to repetition and exploration inefficiency. The necessity for domain-informed, transparent signal architectures is exacerbated by compliance and risk management requirements. "FactorMiner: A Self-Evolving Agent with Skills and Experience Memory for Financial Alpha Discovery" (2602.14670) addresses the combinatorial search complexity and knowledge decay by instantiating a memory-augmented agent skill system, leveraging modular evaluation components and a structured experience memory, thus facilitating scalable discovery under tight library orthogonality constraints.

System Design and Agent Loop

FactorMiner operationalizes formulaic factor mining as an agent skill within the Ralph Loop framework, which iteratively cycles through memory retrieval, candidate generation, evaluation, and strategic knowledge distillation. The architecture consists of three primary modules: Experience Memory, Agent Skill, and Dynamic Factor Library.

Figure 1: FactorMiner architecture: memory retrieval, modular skill pipeline, and evolving orthogonal factor library.

The experience memory stores both successful factor templates and forbidden regions, capturing structural knowledge about factor families and their redundancy relationships. Agent Skill encapsulates a multi-stage pipeline: initial IC screening, inter-factor correlation checks, intra-batch deduplication, and full validation. The factor library grows with strict pairwise orthogonality enforcement, optimizing global library diversity rather than individual factor quality. This system supports modular tool upgrades and market/domain transfer via configuration abstraction.

Experience Memory and Search Dynamics

Experience memory is formally defined as a knowledge base $\mathcal{M}$, evolving through three operators: Formation (distilling mining trajectories into symbolic patterns), Evolution (integrating new artifacts, consolidating redundancy), and Retrieval (context-sensitive extraction of guidance signals for candidate generation). The memory state includes mining logs, recommended directions (high-success templates), forbidden directions (high-correlation zones), and strategic lessons regarding operator selection and regime-switch logic.

The agent's search policy $\pi(\alpha \mid m)$ is conditioned on memory priors, contracting the exploration probability mass onto the orthogonal manifold $\mathcal{P}_{\text{orth}}$. Memory evolution is driven by a distillation operator $\Psi$, updating $\mathcal{M}$ based on mining trajectory outcomes and inter-factor correlation structure.

Ablation studies on memory utility revealed significant improvements: the "Have Memory" variant generated three times more high-quality candidates (60% vs. 20% yield) and achieved more aggressive redundancy filtration, confirming that memory operators $F, E, R$ enable precise navigation and strategic diversity enhancement.

Figure 2: Memory ablation: memory enables yield and diversity, outperforming memoryless baselines.

Modular Skill Pipeline and Computational Efficiency

FactorMiner's modular skill architecture decouples symbolic program synthesis from factor evaluation, mitigating calculation hallucination and supporting cross-market portability. The operator library comprises 60+ typed primitives, including statistical, time-series, cross-sectional, smoothing, regression, and logical operators. The validation pipeline enforces IC and correlation thresholds, and admits replacements for higher-quality factors superseding correlated incumbents.

Computational efficiency is achieved via GPU acceleration, C-compiled numerical operations, and multi-process parallelization. GPU backends deliver 8–59× speedup relative to Python/Pandas, rendering large-scale iterative discovery tractable with commodity hardware.

Figure 3: Benchmark: GPU operator/factor evaluation shows consistent order-of-magnitude speedup.

Factor Library Structure and Redundancy Analysis

The final A-share library comprises 110 rigorously validated formulaic factors, each annotated with explicit expressions and operator lineage. Library diversity is quantified using pairwise Spearman correlation. The average off-diagonal absolute correlation $|\rho|$ is 0.203, with localized clusters but overall moderate dependence, indicating successful navigation of the "Correlation Red Sea" and avoidance of high-redundancy zones.

Figure 4: Pairwise Spearman correlation heatmap of the factor library, illustrating moderate redundancy and controlled tail dependence.

Forbidden directions—e.g., VWAP deviation variants, standardized returns, simple Delta reversals—encode negative knowledge essential for avoiding mathematical equivalence traps and dense correlation clusters. Memory-guided mining ensures continual avoidance of these regions, focusing exploration on orthogonal pattern families such as higher-moment regime-switching, trend regression adaptivity, and robust efficiency interactions.

Empirical Performance and Cross-Market Robustness

FactorMiner was benchmarked against Alpha101 (Classic/Adapted), Random Formula Exploration, GPLearn, and AlphaAgent across four datasets (CSI500, CSI1000, HS300, Crypto). In strict out-of-sample evaluations, FactorMiner exhibits superior performance:

• CSI500: IC/ICIR of 8.25%/0.77 (Factor Library), 14.95%/1.29 (Equal-weight combination)
• Crypto: IC/ICIR of 3.82%/0.28 (Factor Library), 9.48%/0.61 (Equal-weight combination)

These results are consistently higher than all baselines. The mined libraries retained cross-market generalization, with robust IC/ICIR under varied microstructure and asset universes, indicating capture of fundamental price-volume dynamics rather than overfitting to market-specific idiosyncrasies.

Downstream Factor Combination and Selection

FactorMiner libraries are amenable to downstream combination and selection via equal-weight, IC-weighted, and orthogonalization strategies, as well as Lasso and XGBoost models. Learned models provide uplift relative to simple ensembles for most baselines, but for FactorMiner, ensemble combinations nearly saturate predictive utility. Lasso identifies a minimal set (8 factors) capturing ~95% of predictive signal; XGBoost leverages nonlinear interactions among regime-switching and distribution-based factors for maximal ICIR.

Theoretical and Practical Implications

The FactorMiner paradigm synthesizes continual learning, neural-symbolic synthesis, and modular skill architectures. The agent’s self-evolving loop institutionalizes structural knowledge, enabling meta-learning across mining sessions and scaling library diversity beyond standard RL/GP/LLM baselines. The open library facilitates mechanistic inspection and reproducible hypothesis testing, supporting research into market microstructure and regime dynamics.

Practically, FactorMiner supports interpretable, auditable signal discovery suitable for high-compliance trading environments. The modular architecture is adaptable to new markets/frequencies via skill parameterization and memory updates.

Theoretically, the memory-guided search provides a framework for scalable symbolic program discovery with persistent structural knowledge, addressing the correlation saturation phenomenon and information redundancy intrinsic to combinatorial discovery spaces.

Speculation on Future Directions

Prospective work may involve transaction-cost-aware backtesting, real-time online memory update for non-stationary markets, integration with end-to-end forecasting models, and expansion to other asset classes (fixed income, FX, derivatives). The modular skill-memory-agent paradigm is extendable to symbolic regression, scientific discovery, and general program synthesis tasks, suggesting broader applications in AI systems requiring interpretability and continual knowledge accumulation.

Conclusion

FactorMiner exemplifies a lightweight, self-evolving agent architecture for interpretable alpha factor discovery. Combining modular skill execution with structured experience memory, the system scales symbolic alpha mining under orthogonality constraints, delivering a reproducible library for hypothesis-driven financial research. Memory-guided search yields superior yield and diversity, affirming the value of institutionalized structural knowledge and continual learning mechanisms within neural-symbolic synthesis frameworks.