LLM-in-Sandbox: Enabling Agentic LLM Systems

• LLM-in-Sandbox is a paradigm that transforms large language models into autonomous agents by leveraging a controlled virtual sandbox for iterative tool-based problem solving.
• The framework formalizes an interactive loop where LLMs issue tool commands, process environment feedback, and employ reinforcement learning to boost performance across diverse domains.
• It offers scalable, secure deployment with minimal computational overhead, making it suitable for research benchmarks and real-world applications in complex tasks.

LLM-in-Sandbox is a paradigm and software framework that enables LLMs to reason, plan, and act within a controlled computational sandbox—a virtualized code environment—allowing the emergent elicitation and enhancement of agentic behaviors that generalize far beyond code synthesis. The LLM-in-Sandbox methodology formalizes the interactive loop in which an LLM issues tool calls (e.g., shell commands, file edits, code execution), receives environment feedback, and leverages both for nontrivial problem-solving in mathematics, science, biomedicine, and complex general tasks. The approach is extensible to training via reinforcement learning (LLM-in-Sandbox-RL), supports both vanilla and agentic LLM evaluation, and is open-sourced to facilitate scientific benchmarking and real-world deployment scenarios (Cheng et al., 22 Jan 2026).

1. System Architecture and Control Loop

The canonical LLM-in-Sandbox system instantiates each agent inside a stateless, lightweight Ubuntu-based Docker image (~1.1 GB) preconfigured with a Python interpreter and standard science libraries (NumPy, SciPy). The sandbox environment exposes a minimal, atomic API for the LLM to interact with:

• execute_bash(cmd): Executes arbitrary bash commands in a persistent shell. This enables package management (apt-get, pip), file I/O, program invocation, etc.
• str_replace_editor(...): Enables viewing, creating, or modifying files under a dedicated directory (/testbed).
• submit(): Marks the completion of the task, signaling the system to grade output or finalize results.

The interaction is structured with a ReAct-style loop:

Input: user prompt p, optional requirements r, sandbox S, max turns T Configure S (e.g. drop input files under /testbed/input) history ← [] for t in 1…T: a_t ← LLM(p, history) # Predict next tool call if a_t is submit: break obs_t ← sandbox.execute(a_t) history.append((a_t, obs_t)) Read final answer from /testbed/output/answer.txt

Each LLM step consists of selecting the next tool call based on the prompt and interaction history. The sandbox executes the tool call, returns observations, and the transcript continues. The file system under /testbed persists across steps but resets between tasks, ensuring containment.

2. Zero-Shot Agentic Generalization

Notably, state-of-the-art LLMs (e.g., Claude Sonnet, GPT-5, DeepSeek-V3.2) generalize agentic use of the sandbox without any dedicated tool-use instruction or fine-tuning. Empirically, these models:

• Install and utilize new software (e.g., acquiring RDKit via pip, handling Java dependencies for chemistry tasks).
• Write multi-file scripts to offload context (handling 100K-token input documents via file slicing and Python extractors).
• Exploit the Unix environment for iterative computation, data extraction, and result formatting (e.g., using grep, sed, and custom scripts).

Task decomposition and resource use are emergent: strong LLMs allocate 18–24% of tool calls to file operations, 6–12% to external installs, and 11–14% to computation, compared to <3% effective use by weaker models (Cheng et al., 22 Jan 2026).

3. Reinforcement Learning in the Sandbox (LLM-in-Sandbox-RL)

To induce strategic, tool-interacting behaviors in weaker or medium-capability models, LLM-in-Sandbox-RL introduces an on-policy RL scheme:

• State $s_t$: Full interaction history (tool actions and sandbox feedback).
• Action $a_t$: Tool call (execute_bash, str_replace_editor) or submit.
• Reward $R(\tau)$: Task-dependent, based on final answer correctness (binary for MCQ, ROUGE-L/F1 for generation).
• Objective:

$$\mathcal{L}(\theta) = -\mathbb{E}_{\tau\sim\pi_\theta}[R(\tau)] + \lambda\,\mathrm{KL}(\pi_\theta\|\pi_0)$$

(No explicit KL penalty is used in main reported experiments.)

In practice, Qwen3-4B-Instruct and Qwen3-Coder-30B models are improved via single-step on-policy updates (GRPO variant), achieving substantial boosts in performance across all measured domains. Hyperparameters include a learning rate of $1e-6$, batch size of 8, rollouts per prompt of 8, and turn cap of 100 (Cheng et al., 22 Jan 2026).

4. Empirical Evaluation Across Domains

LLM-in-Sandbox is evaluated on a broad suite of benchmarks:

• Mathematics (AIME25): +7.3–10.1% improvement over vanilla LLM (zero-shot); after RL, Qwen3-4B rises from 35.4% to 50.2% accuracy.
• Physics, Chemistry, Biomedicine: Gains from +0.5% to +14.4% depending on benchmark.
• Long-context tasks: Using file I/O in the sandbox achieves +13.0% over prompt-only context storage.
• Instruction following: +3.7–14.4% improvement in constrained generation.
• Code (SWE-bench): Measured via task verification.

Post-RL, both agentic (sandboxed) and vanilla (no-tools) performance improve, indicating transfer of agentic skills back to text-only operation (Cheng et al., 22 Jan 2026).

Domain	Vanilla LLM vs. LLM-in-Sandbox	Post-RL Performance Gain
Mathematics	+7.3–10.1%	35.4% → 50.2% (Qwen3-4B)
Chemistry	+0.5–14.4%	—
Long-Context	+0.5–6.2%, +13.0% (file)	5.8% → 16.8% (Qwen3-4B)
Instruction	+3.7–14.4%	—

5. Computational Efficiency and Deployment

Despite the multi-step process, the framework is optimized for computational and infrastructural efficiency:

• Token and compute overhead: Environment tokens account for 43–51% of total tokens, but are processed via fast "prefill" paths rather than slow autoregressive decoding. Average end-to-end token consumption per query is 0.84× vanilla LLM.
• Throughput: Query-level throughput ranges from 0.6× to 2.2× vanilla LLM speed (e.g., MiniMax).
• Sandbox infrastructure: Each Docker image is a minimal 1.1 GB and supports concurrent sharing ($<$5% of 2 TB RAM with 512 sandboxes). No per-task customization is needed.
• Open-source release: The framework is provided as a Python package (supports standard API-based LLMs, vLLM, and SGLang backends), requiring only one Dockerfile and supporting automated /testbed mounting and file management.

The minimal overhead and ready deployment make the approach suitable for both research and production digital agent applications (Cheng et al., 22 Jan 2026).

6. Core Capabilities and Research Implications

LLM-in-Sandbox transforms baseline LLMs into autonomous computational agents capable of:

1. Dynamically acquiring and utilizing new software tools and resources via shell commands.
2. Offloading and organizing information via persistent file manipulation for tasks with context much larger than model context windows.
3. Iteratively composing, executing, and validating code to solve problems with constraints beyond natural-language generation (e.g., formatting, logic, stateful operations).

The emergent behaviors challenge classical task boundaries: agentic skills such as environment exploration, procedural decomposition, error recovery, and toolchain construction are observed without explicit tool-use training, and can be further amplified by RL using only outcome-based rewards from non-agentic data (Cheng et al., 22 Jan 2026).

7. Comparative and Safety Perspective

LLM-in-Sandbox demonstrates clear separation between strong and weak models via metrics of effective tool use and task completion, and provides a principled setting for RLHF and safety research on agentic LLMs. As other works highlight the risks of agentic autonomy (e.g., emergent deception or misaligned behaviors under sandboxed constraints (Ivanov, 30 Jun 2025)), the controlled virtual machine approach enables both the enhancement of general intelligence and the monitoring/alignment of behavior in a reproducible, auditable way. The scalability, domain coverage, and open-source design position LLM-in-Sandbox as a central research artifact for next-generation agentic LLM systems (Cheng et al., 22 Jan 2026).