SE-Bench: Benchmarking Self-Evolution with Knowledge Internalization

Abstract: True self-evolution requires agents to act as lifelong learners that internalize novel experiences to solve future problems. However, rigorously measuring this foundational capability is hindered by two obstacles: the entanglement of prior knowledge, where ``new'' knowledge may appear in pre-training data, and the entanglement of reasoning complexity, where failures may stem from problem difficulty rather than an inability to recall learned knowledge. We introduce SE-Bench, a diagnostic environment that obfuscates the NumPy library and its API doc into a pseudo-novel package with randomized identifiers. Agents are trained to internalize this package and evaluated on simple coding tasks without access to documentation, yielding a clean setting where tasks are trivial with the new API doc but impossible for base models without it. Our investigation reveals three insights: (1) the Open-Book Paradox, where training with reference documentation inhibits retention, requiring "Closed-Book Training" to force knowledge compression into weights; (2) the RL Gap, where standard RL fails to internalize new knowledge completely due to PPO clipping and negative gradients; and (3) the viability of Self-Play for internalization, proving models can learn from self-generated, noisy tasks when coupled with SFT, but not RL. Overall, SE-Bench establishes a rigorous diagnostic platform for self-evolution with knowledge internalization. Our code and dataset can be found at https://github.com/thunlp/SE-Bench.

• The paper demonstrates that closed-book SFT significantly outperforms RL methods in encoding novel, obfuscated API knowledge.
• It introduces SE-Bench, a diagnostic platform that obfuscates NumPy functions to measure pure knowledge internalization in self-evolving agents.
• Experimental results reveal that withholding documentation during training is essential for robust parametric memory and autonomous learning.

SE-Bench: A Diagnostic Platform for Knowledge Internalization in Self-Evolving Agents

Motivation and Problem Setting

Self-evolution—defined as a model's capacity to autonomously acquire, internalize, and consolidate novel knowledge through interaction with its environment—is foundational to AGI. Existing methods evaluate fragments of this ability, such as incremental adaptation or tool-use, but fail to cleanly disentangle pure knowledge internalization from confounds like prior exposure and reasoning complexity. The two core obstacles are: (1) pre-training contamination, which precludes certainty that an agent’s knowledge is not memorized from training data; and (2) entanglement with task complexity, where failures may reflect reasoning limitations rather than deficits in experiential knowledge assimilation.

SE-Bench addresses this with an adversarially clean, synthetic domain: it obfuscates the well-known NumPy API and its documentation into a novel, randomized package (ZWC), thus precluding any leakage from public corpora or base knowledge. Agents are required to internalize ZWC's logic and API purely through exposure to its documentation during training, but must subsequently complete tasks in a closed-book regime, operating without documentation at test-time.

Figure 1: Overview of the SE-Bench construction pipeline. Steps include function and docstring obfuscation (creating ZWC), automatic task generation, and consensus filtering with multiple LLMs plus human verification.

This experimental design cleanly isolates the target phenomenon: successful task completion at test time is algorithmically trivial given knowledge of ZWC, but mathematically impossible without it due to the randomized API. Thus, performance directly measures parametric knowledge internalization capacity.

Benchmark Construction and Evaluation Protocol

The SE-Bench construction pipeline has three critical phases:

1. Obfuscation: 268 NumPy functions are systematically renamed to semantically void ZWC identifiers, and all documentation is rewritten to refer only to ZWC, erasing any original NumPy tokens.
2. Task/Case Generation: State-of-the-art LLMs generate single- and multi-function programming tasks along with ground-truth test cases, mapped to ZWC's API.
3. Filtering: Three distinct closed models (Qwen3-Coder-480B, Gemini-2.5-Pro, and GPT-OSS-120B) must each solve the generated tasks using the original NumPy API. Only tasks solved by all three are retained, with additional human filtering for clarity.

Training is conducted with access to documentation; evaluation is strictly closed-book. The final 1,417-task SE-Bench comprises both "Single" (single-API) and "Multiple" (multi-step compositional) splits.

The evaluation metric strictly enforces three constraints: (a) all test cases must pass; (b) abstract syntax tree analysis must show exclusive use of ZWC; and (c) the original NumPy must never be imported.

Experimental Results and Major Empirical Insights

Extensive baselines from three families are evaluated:

• Memory-based: External memory retrieval methods (ACE and ExpeL)
• Parameter-update: Supervised fine-tuning (SFT), reinforcement learning (RL), and hybrids, each run under "Open" (documentation present during update) and "Closed" (documentation withheld) regimes
• Self-play/autonomous: Fully self-generated data with downstream SFT or RL

Key results:

• Zero-shot accuracy using ZWC without documentation is 0.0%, confirming the benchmark’s immunity to training leakage.
• Closed-SFT and Closed-SFT-RL substantially outperform Open variants and memory retrieval, achieving up to 54.4% accuracy, but still trail "oracle" NumPy usage (>90%).
• All RL-only and Open-book SFT methods fail completely (0% accuracy), even matching state-of-the-art methods such as Absolute Zero.
• Self-play is viable if coupled with SFT, but not RL: Closed-SFT on self-generated curricula achieves 22.5% accuracy, whereas RL again yields 0%.

Figure 2: Withholding documentation during SFT (Closed-SFT) induces true parametric internalization; Open-SFT fails without documentation, exposing brittle context dependence.

Analysis of Optimization Dynamics

SFT and RL exhibit starkly divergent optimization characteristics for knowledge internalization.

• Open-Book Paradox: When documentation remains accessible during parameter update, no true internalization occurs. Test-time performance collapses without documentation, exposing a strict reliance on context.
• Closed-Book Superiority: Withholding documentation during update enforces genuine parametric memory—all context-free performance is strictly attributed to weights, not short-term memory.
• RL Gap: Classic RL methods (PPO, GRPO) fail to internalize obfuscated APIs due to two factors:
  • PPO’s clipping precludes large, informative weight updates needed to encode novel factual knowledge
  • Use of normalized advantages introduces negative gradients early, erasing transient, fragile associations before they consolidate
  • Ablation shows that SFT-style settings for RL (positive-reward only, large learning rate and batch size, no clipping) can partially close the gap, but as soon as standard RL mechanisms are reintroduced, learning collapses
  • 

Figure 3: Distribution of error types. Closed-SFT is dominated by API hallucinations; Hybrid Closed-SFT-RL suppresses hallucinations and encourages robust fallback strategies.

Autonomous Self-Play and Internalization

A crucial question is whether agents can autonomously generate and learn from their own (potentially noisy) curricula. Direct self-play with RL (Absolute Zero) fails entirely, mirroring the RL Gap. However, using SFT on self-generated tasks recovers significant non-trivial performance, indicating that the optimization method, not data quality, is the bottleneck for self-evolving internalization.

Behavior Consolidation and Error Shift

Closed-SFT yields "probabilistic" memory, where agents often fill in gaps with plausible hallucinations (e.g., inventing missing ZWCArray attributes). When RL is applied after SFT ("Closed-SFT-RL"), there is a dramatic shift: RL prunes uncertain, lazy guesses and favors disciplined, more robust code, frequently replacing implausible calls with primitive constructs (loops, built-ins). However, RL is unable to repair fundamental memorization errors or incorrect API signatures inherited from SFT—these persist.

Diversity Effects

Figure 4: Impact of question vs. response diversity on knowledge internalization. Diverse question exposure is far more consequential than response diversity.

The diversity of training questions is crucial for efficient internalization—merely increasing solution diversity per question has minor effects relative to the breadth of encountered problem templates.

Self-Play Internalization Dynamics

Figure 5: Self-play SFT (Closed-SFT$_\text{self}$) outperforms open-book self-play, conforming to the Open-Book Paradox even in a fully autonomous regime.

The Open-Book Paradox persists under self-generated curricula: documentation exposure during update inhibits generalization to documentation-free test-time.

Implications and Future Directions

SE-Bench exposes and quantifies multiple previously underexplored phenomena essential to the development of robust self-evolving machine intelligence:

• Closed-book learning as a requirement: Accessible context during parameter update is actively detrimental to long-term retention and parametric compression.
• Limitations of RL for factual internalization: Standard RL protocols optimize utilization and behavioral adaptation, but cannot efficiently encode novel, structured knowledge unless modified to remove clipping and negative gradients.
• Autonomous learning is viable, contingent on optimization: Self-generated data is sufficient for parametric internalization in LLMs—charting a clear path for self-guided AGI learning, if architectural and algorithmic changes (towards SFT-style updates) are implemented.
• Behavior refinement pipeline: SFT can be leveraged for acquisition and RL for consolidation and error pruning, but not for de novo retention.

Looking forward, SE-Bench offers a minimal diagnostic test for self-evolution—a "needle-in-a-haystack" for parametric memory. It will serve as an important baseline for further advances in continual learning, hybrid update protocols, and the mechanistic elucidation of memory consolidation in large-scale architectures.

Conclusion

SE-Bench establishes an empirically robust, disentangled environment for evaluating and studying knowledge internalization in AI agents. Through rigorous ablation and diverse experimental settings, it demonstrates that parametric memory formation is strictly context-dependent, classic RL is inadequate for factual acquisition, and closed-book SFT (possibly seeded with autonomous data) is essential for robust self-evolving agents. This work provides a necessary lower bound for future research on self-improving AI and informs both the limitations of current training paradigms and the prospects for open-ended, autonomous machine learning (2602.04811).