Section-Local Textual Gradients in MPO

• Section-local textual gradients are a modular prompt optimization technique that refines distinct prompt sections independently.
• They use critic language models to generate natural-language updates, preserving key instructions and reducing cross-section interference.
• Empirical evaluations show improved reasoning and performance on LLM benchmarks without altering model parameters.

Section-local textual gradients are a prompt optimization technique applied within the Modular Prompt Optimization (MPO) framework, introduced to address the limitations of monolithic prompt editing for LLMs. This approach treats prompts as structured objects composed of discrete, semantically coherent sections, such as system role, context, task description, constraints, and output format. Section-local textual gradients allow for targeted, independent refinement of each prompt section via language-model-generated natural-language edits, thereby improving interpretability, preserving critical instructions, and mitigating cross-section interference. This technique has demonstrated consistent reasoning performance improvements on LLM benchmarks without modifying model parameters or overall prompt structure (Sharma et al., 7 Jan 2026).

1. Definition and Motivation

Monolithic textual gradient methods optimize prompts by treating the entire instruction sequence as a single block. In this paradigm, a critic LLM (LM) assesses the full prompt and generates a natural-language global delta, Δp, which is then applied in its entirety. This can result in the loss of important instructions, unintended side effects, and unmanageable prompt growth.

Section-local textual gradients, by contrast, operate under a fixed prompt schema $p = \{ s^{(1)}, \ldots, s^{(K)} \}$, where each $s^{(k)}$ is a distinct section. The critic LM generates an independent update Δ$s^{(k)}$ for each section, conditioned on the content of the remaining sections. This modular approach allows precise localization of feedback, preserves each section's semantic role, and supports scalable, interpretable, and non-destructive prompt optimization (Sharma et al., 7 Jan 2026).

2. Mathematical Formalization

Let $p_t = \{ s_t^{(1)}, \ldots, s_t^{(K)} \}$ represent the structured prompt at iteration $t$. The optimization process is guided by a critic loss $L_{\mathrm{critic}}(p)$, defined as the negative log-likelihood of correct answers $y$ given inputs $x$ under a solver model $S$:

$$L_{\mathrm{critic}}(p) = -\frac{1}{|D|}\sum_{(x,y)\in D}\log P_S(y \mid p,x)$$

where $D$ is a held-out development set. Lower $L_{\mathrm{critic}}$ indicates better expected model performance.

For section $k$, the embeddings for section tokens are denoted $E_t^{(k)}$. The (theoretical) embedding-space gradient is

$$g_t^{(k)} = \nabla_{E_t^{(k)}} L_{\mathrm{critic}}(p_t)$$

Due to the discreteness of text tokens, direct embedding updates are not feasible. MPO addresses this by letting the critic LM $C$ generate a natural-language update $\Delta s_t^{(k)}$ that approximates a step in the negative gradient direction. This discrete gradient is then additively applied:

$$\tilde{s}_{t+1}^{(k)} = s_t^{(k)} \oplus \Delta s_t^{(k)}$$

$$s_{t+1}^{(k)} = \mathcal{D}\bigl(\tilde{s}_{t+1}^{(k)}\bigr)$$

where $\mathcal{D}$ denotes a de-duplication module to consolidate redundant or overlapping instructions (e.g., via LLM-based summarization or overlap heuristics).

3. Critic LM Gradient Generation Process

For each section $s_t^{(k)}$, the critic LM $C$ receives as input:

• The current section: $s_t^{(k)}$.
• The remainder of the prompt: $p_t \setminus s_t^{(k)}$.
• A directive to propose concise improvements to $s_t^{(k)}$ that are expected to lower solver error.

The critic outputs $\Delta s_t^{(k)}$—often a short, actionable edit or addition (e.g., “Add a reminder to think step by step”). These updates serve as a text-space approximation to a negative gradient step on $L_{\mathrm{critic}}$. No direct manipulation of model parameters or embedding vectors is required, as the process leverages the intrinsic generative capabilities of LLMs for both evaluation (by $L_{\mathrm{critic}}$) and edit proposal (by $C$).

Candidate updates can be further filtered or scored: when $C$ generates multiple $\Delta s$ alternatives (either by beam search or stochastic sampling), each resulting prompt can be temporarily constructed and scored under $L_{\mathrm{critic}}$ to select the edit that minimizes loss.

4. Algorithmic Workflow

The MPO algorithm using section-local textual gradients proceeds as follows:

1. Initialization:
   • Start from $p_0 = \{ s_0^{(1)}, \ldots, s_0^{(K)} \}$.
   • Specify critic LM $C$, de-duplication module $\mathcal{D}$, and number of optimization steps $T$.
2. Iterative Optimization (for $t = 0 \ldots T-1$):
   • For each section $k = 1 \ldots K$:
     • Generate section-local textual gradient: $$\Delta s \leftarrow C.\text{generate\_delta}(s^{(k)}, p - s^{(k)})$$.
     • Form the tentative update: $\tilde{s}^{(k)} \leftarrow s^{(k)} \oplus \Delta s$.
     • Consolidate via de-duplication: $$s_\text{next}^{(k)} \leftarrow \mathcal{D}(\tilde{s}^{(k)})$$.
   • Reassemble $$p \leftarrow \{ s_\text{next}^{(1)}, \ldots, s_\text{next}^{(K)} \}$$.
3. Termination: Return $p_T$ as the optimized prompt.

If multiple candidate deltas are generated per section, each resulting prompt can be scored using $L_{\mathrm{critic}}$ to select the changes that most improve solver accuracy.

5. Illustrative Example and Empirical Outcomes

Consider a prompt with three sections:

Section	Initial Text	Critic Delta	Updated Text
System Role	You are a helpful assistant.	Always think step by step.	You are a helpful assistant. Always think step by step.
Task Details	Solve math word problems.	Show intermediate equations.	Solve math word problems. Show intermediate equations.
Output Format	Give the answer only.	Include units where appropriate.	Give the answer only, including units where appropriate.

In this workflow:

• The System Role is refined to encourage stepwise reasoning.
• The Task Details section is extended to make explicit the expectation of shown work.
• The Output Format evolves to prevent omission of units, addressing domain-specific errors.

De-duplication ensures concise, non-redundant instructions. Empirical evaluation on ARC-Challenge and MMLU with LLaMA-3 8B-Instruct and Mistral-7B-Instruct demonstrates notable performance gains (e.g., an increase from 75% to 79.1% accuracy) (Sharma et al., 7 Jan 2026). This improvement is attributable to increased clarity and targeted augmentation of prompt sections rather than model retraining or architectural changes.

6. Theoretical Properties and Interpretability

Section-local textual gradients provide an interpretable, modular approximation to continuous gradient-based prompt tuning applied in embedding space. Each $\Delta s^{(k)}$ is both human-interpretable and traceable to an explicit schema component. The process supports granular error analysis, preservation of essential instructions (e.g., hard constraints or formatting requirements), and controlled growth of prompt length via de-duplication.

By constraining prompt edits to fixed schema sections, MPO avoids destructive interference observed in monolithic gradient methods, where global deltas can inadvertently delete or dilute vital information. This is particularly relevant for instruction-tuned small LMs, where explicit structure governs model reliability and reasoning performance.

7. Relationship to Prior Work and Practical Implications

Section-local textual gradients represent an evolution of textual gradient and LLM-based self-refinement techniques, addressing key shortcomings in structure-awareness and prompt interpretability. Unlike monolithic optimization approaches, this method retains alignment with transparent, user-defined schemas and supports safe, modular deployment in practical, safety-critical applications.

A plausible implication is that the section-local paradigm, by providing modular control over prompt optimization, may facilitate hybrid human-in-the-loop workflows where certain sections remain fixed or are subject to human review, while others are amenable to LLM refinement. This approach is extensible to a broad class of reasoning and instruction-following tasks and operates independently of model scale or architecture, as it requires no modification of solver parameters (Sharma et al., 7 Jan 2026).