Refploit: Facilitating Exploit Construction via Code-Agent Trajectory Repair

Abstract: Vulnerability exploits play a crucial role in assessing the downstream impact of Java library vulnerabilities. While some vulnerabilities are accompanied by disclosed exploit references, automatically reproducing such references into runnable exploits remains challenging because they are often incomplete, unstructured, or only describe partial reproduction steps. Recent code agents provide a promising way to automate this process, but our study shows that their generated exploits often appear successful without triggering the actual vulnerable logic, such as replacing vulnerable APIs with self-implemented functions. To address this, we propose Refploit, an LLM-based trajectory recovery framework for facilitating vulnerability reproduction from public exploit references. The key insight is that a failed agent trajectory is not entirely useless. It may have already completed some reproduction subtasks while also revealing misleading directions that should be avoided. Refploit first validates an agent-generated exploit through differential execution. When the exploit is ineffective, Refploit analyzes its reproduction progress, locates the trajectory segments associated with the reproduction progress, and derives constraints to guide focused recovery. We evaluate Refploit on three open-source Java vulnerability datasets, covering 172 exploit references for 143 vulnerabilities. Under DeepSeek-V4-Flash, Refploit successfully reproduces 138 exploits, achieving a reproduction rate of 80.2%. It achieves a 64.3% relative improvement over the initially generated trajectories and outperforms both the SOTA exploit-generation method PoCGen and advanced code agents such as Codex with GPT-5.4. We further adapt Refploit to another code agent and observe consistent improvements, demonstrating its generality.

• The paper introduces a multi-stage LLM framework that repairs failed exploit trajectories using differential execution and constraint-guided recovery.
• It decomposes exploit attempts into clear sub-tasks, achieving an 80.2% reproduction rate and outperforming state-of-the-art baseline agents.
• The study highlights robust repair patterns in harness realignment, payload correction, and configuration restoration, proving its agent-agnostic adaptability.

Refploit: LLM-Guided Trajectory Repair for Automated Exploit Construction

Problem Setting and Motivation

The automation of vulnerability reproduction is critical in software security, especially for open-source libraries where manual exploit construction for newly disclosed vulnerabilities is labor-intensive and error-prone. In the Java ecosystem, the proliferation of packages in Maven amplifies this challenge—downstream impact assessment commonly relies on the feasibility of exploits. While a minority of CVEs include public exploit references, these references are frequently incomplete, require additional harnessing, or only exist as textual descriptions rather than executable tests.

Recent code agents, supported by LLM backends, have shown promise in automating the transition from exploit reference to runnable exploit. However, empirical analysis demonstrates that these agents often generate plausible but ineffective reproductions: e.g., invoking mock logic instead of the vulnerable library API, or misaligning payloads and environment configurations (as detailed in the paper's introductory study and depicted in Figure 1).

Figure 1: A case study showing the refinement process of an initial agent-generated trajectory that fails to properly trigger vulnerable logic in CVE-2024-23833, motivating the need for structured trajectory repair.

The motivation for Refploit arises from observations that (1) failed trajectories often advance some sub-tasks (e.g., selecting the correct runtime or project structure); and (2) trajectories contain both segments that should be preserved and segments that actively mislead future attempts. The approach thus aims to distill these signals and provide targeted trajectory repair.

Framework and Methodology

Refploit introduces a multi-stage LLM-based framework for trajectory repair, explicitly addressing the deficiencies of initial agent-generated exploits:

1. Differential Execution for Exploit Validation: The framework first executes the initial agent-generated exploit against both vulnerable and patched library versions, collecting behavioral evidence (logs, exceptions, side effects). Instead of heuristics or one-sided tests, differential execution enables precise validation of exploit effectiveness by confirming behavioral divergence—a necessary precondition for successful vulnerability triggering.
2. Progress-Guided Trajectory Decomposition: When validation fails, Refploit decomposes the agent's trajectory into interpretable segments, mapped to three evidence-oriented reproduction dimensions: Environment Preparation (runtime, dependencies, config), Harness Assembly (project and invocation of target API), and Exploit Reproduction (payload execution, observable effects). Each dimension may contain satisfied, incomplete, and misleading sub-tasks, identified through LLM-driven reasoning over the collected execution evidence and the step-by-step agent log.
3. Constraint Extraction and Targeted Recovery: Refploit distills preservation constraints (validated progress) and repair constraints (incomplete/misleading actions) for each sub-task. Recovery proceeds by replaying the preserved trajectory prefix, then launching sub-agents tasked to repair the failing or missing subtasks under explicit constraints, thereby avoiding a full reset and focusing repair on minimal, high-value intervention points. The recovery loop is iterated, with validation and progress reassessment until the exploit is repaired or a budget is exhausted.

   Figure 2: Schematic overview of Refploit’s three-stage architecture—differential execution, trajectory progress analysis, and constraint-guided recovery—showing LLM prompt integration at each stage.

This methodology is distinctive for its trajectory-level interpretability, its use of version-differential oracles for validation, and its systematic constraint synthesis to reduce search space and avoid agent myopia or repetitive failures.

Experimental Evaluation

Refploit was evaluated on three large open-source Java vulnerability datasets, spanning 172 exploit references for 143 vulnerabilities (53 CWE categories). Key experimental findings were as follows:

• Refploit achieved an 80.2% reproduction rate (138/172 exploits) with DeepSeek-V4-Flash, a 64.3% relative improvement over the base agent (48.8%).
• It outperformed the SOTA LLM-based exploit generator PoCGen (Simsek et al., 5 Jun 2025), “Codex with GPT-5.4,” and baseline agents such as mini-swe-agent, by margins of 10–52% across multiple CWE categories.
• The approach is general across code agents and LLM backends. When ported to OpenHands, another popular agent, Refploit improved exploit reproduction by 74.5%, demonstrating agent-agnostic and model-agnostic properties.
• Detailed ablation studies confirmed the criticality of each component—differential testing, fine-grained progress assessment, and explicit constraint guidance all contributed synergistically to the outcome. Disabling any of these dimensions reduced effectiveness by up to 28.4%.
• Notably, PoCGen and other template-driven baselines showed significant deficits on CVEs requiring environment adaptation or nuanced payload/harness modifications, reiterating the advantage of Refploit’s trajectory-aware, cross-version analysis.

Qualitative Analysis and Failure Modes

Refploit's constraint-guided recovery exhibits robust patterns:

• Harness Realignment: Repairs agents that simulate the vulnerable function, instead redirecting to the true library API and triggering logic (as in harness modification for real attack surfaces).
• Payload Semantics Correction: Adapts incorrectly structured or insufficient payloads to match environmental and vulnerability-specific requirements, allowing for timeouts, RCEs, or bypasses to be properly observed.
• Configuration Restoration: Repairs incomplete runtime, project, or environmental configurations (e.g., faulty deserialization flags, missing required properties) that initially preclude reproduction.

Persistent failure cases generally stem from (1) misclassified partial progress (e.g., LLM failing to infer that an exception indicates progress rather than failure); or (2) complex runtime or third-party tool dependencies that stall exploit attempts before code-level reproduction is feasible.

Practical and Theoretical Implications

Practically, Refploit advances the automation of down-stream vulnerability assessment, reducing human intervention in exploit construction, which is especially critical for large ecosystems like Java/Maven where the scale of disclosed vulnerabilities is vast. Its adaptivity across agent architectures enables integration with a variety of LLM-powered code agents and development pipelines. The explicit introspection and constraint-guided repair offer avenues for building more robust, self-healing agentic frameworks for software security tasks.

Theoretically, the work highlights the importance of trajectory interpretability and iterative, constraint-guided refinement in code-agent systems for software exploitation. This paradigm is readily transferable to other domains where agent plans are multi-step, partially correct, and require post-hoc targeted repair with minimal redundant exploration.

Looking forward, evaluation on additional language ecosystems, image-heavy/non-text exploit references, and deeper multimodal trajectory analysis remain open challenges. Integration with continuous fuzzing and regression assessment tools (for both security and functional correctness) is a promising future direction.

Conclusion

Refploit constitutes a significant advance in LLM-driven, agent-based vulnerability reproduction by formulating failed agent trajectories as repairable artifacts, amenable to constraint-guided, minimal-edit recovery. Its strong empirical performance, rigorous ablation validation, and generality across models/agents position it as a foundation for next-generation autonomous vulnerability assessment pipelines, and as a blueprint for incorporating trajectory semantics and constraint abstraction into agentic problem solving for software engineering and security.