Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

Abstract: Pass@k is a widely used performance metric for verifiable LLM tasks, including mathematical reasoning, code generation, and short-answer reasoning. It defines success if any of $k$ independently sampled solutions passes a verifier. This multi-sample inference metric has motivated inference-aware fine-tuning methods that directly optimize pass@$k$. However, prior work reports a recurring trade-off: pass@k improves while pass@1 degrades under such methods. This trade-off is practically important because pass@1 often remains a hard operational constraint due to latency and cost budgets, imperfect verifier coverage, and the need for a reliable single-shot fallback. We study the origin of this trade-off and provide a theoretical characterization of when pass@k policy optimization can reduce pass@1 through gradient conflict induced by prompt interference. We show that pass@$k$ policy gradients can conflict with pass@1 gradients because pass@$k$ optimization implicitly reweights prompts toward low-success prompts; when these prompts are what we term negatively interfering, their upweighting can rotate the pass@k update direction away from the pass@1 direction. We illustrate our theoretical findings with LLM experiments on verifiable mathematical reasoning tasks.

• The paper identifies prompt interference as a key factor causing Pass@1 degradation during Pass@k optimization in LLM post-training.
• It presents a formal gradient conflict analysis, illustrating how negative inner products between prompts degrade single-try performance.
• Empirical results on code generation and mathematical reasoning tasks highlight critical trade-offs for inference-aware finetuning strategies.

Theoretical and Empirical Analysis of Pass@k Optimization-Induced Pass@1 Degradation via Prompt Interference

Introduction

The study rigorously analyzes the empirically observed trade-off between pass@k and pass@1 metrics in LLM post-training, with a focus on verifiable tasks such as code generation and mathematical reasoning. Pass@k, which measures the probability that at least one of $k$ independent samples is correct, has motivated inference-aware training protocols that directly optimize the metric anticipated for deployment. However, a recurring phenomenon is that such optimization often degrades pass@1—the single-try success probability—despite substantial gains in pass@k. This effect is operationally impactful since high reliability with pass@1 is frequently required in cost-sensitive or fallback settings where multiple attempts are infeasible or verifiers are imperfect.

Prompt Interference: Definitions and Gradient Characterization

The central contribution is the formalization of prompt interference, extending prior notions of negative transfer in multi-task learning to LLM post-training. Prompt interference is operationalized via the gradient inner product between the per-prompt pass@1 gradients: two prompts $x$, $x'$ are said to be negatively interfering if updates that improve one prompt’s pass@1 degrade the other. Formally, with the kernel function $$\kappa_\theta(x, x') = \langle \nabla p_\theta(x), \nabla p_\theta(x') \rangle$$, negative values correspond to interfering prompts.

This phenomenon is illustrated in a minimal contextual bandit setup where prompts with overlapping features but different optimal responses (easy vs hard) yield nearly antiparallel pass@1 gradients, as evidenced by a cosine kernel heatmap:

Figure 1: Cosine kernel heatmap illustrating prompt pairs—substantial regions in blue highlight negative interference where gradient directions are opposed.

Through an explicit decomposition of pass@k gradients, the work reveals that optimizing pass@k induces an implicit importance weighting toward lower-success prompts (harder instances), with $w_{k,\theta}(x) = k(1-p_\theta(x))^{k-1}$. When these hard prompts are negatively interfering, the aggregate pass@k gradient can rotate into direct conflict with the mean pass@1 gradient—as shown in a schematic:

Figure 2: Schematic of empirical trade-off and geometric gradient interaction; prompt 3’s negative interference skews the overall update direction away from pass@1 improvement.

In such cases, even a single step of pass@k optimization provably increases pass@k while decreasing pass@1, a result both theoretically quantified and numerically validated in the toy domain and LLM finetuning.

Formal Gradient Conflict Analysis

The analysis proceeds by precisely characterizing when gradient conflict occurs. The inner product $$\langle \nabla J_k(\theta), \nabla J_1(\theta) \rangle$$ is negative (i.e., the gradients are conflicting) if and only if the average agreement score $a_\theta(x)$ (prompt-level alignment with the average pass@1 direction) is negative under the pass@k-induced prompt weighting. This is cast as

$$\langle \nabla J_k(\theta), \nabla J_1(\theta)\rangle = \mathbb{E}_{x \sim \mathcal{D}}[w_{k,\theta}(x) a_\theta(x)]$$

A sufficient condition for conflict is when the weighted influence of negative-interference prompts dominates that of positive ones. The threshold value of $k$ beyond which this transition occurs is determined as a closed-form function of gradient norms, agreement-score statistics, and hard/easy prompt response probabilities.

The analysis is supported by contour plots visualizing the parameter space where pass@k and pass@1 gradients are aligned versus in conflict:

Figure 3: Contour plots in parameter space; gray area marks regions of gradient conflict between pass@1 and pass@k.

The framework also makes explicit the nontriviality of the effect: absent negative interference, all pass@k objectives are mutually aligned, but the presence and upweighting of negatively interfering regions is both necessary and sufficient for conflict.

Empirical Validation: Mathematical Reasoning with LLMs

Experiments employ MATH dataset benchmarks and LLMs (DeepSeek-R1-Distill-Llama-8B, Qwen-7B) to empirically verify theoretical claims. Prompts are partitioned into easy/hard via pass@1 thresholds. Agreement scores and pass@k weights are computed from response samples.

A key result—consistent across multiple threshold configurations—is that pass@k reweighting yields a dramatic shift: hard prompts, even if vastly outnumbered by easy prompts, receive exponentially higher importance, and their negative agreement scores collectively dominate the weighted average:

Figure 4: DeepSeek-Llama empirical result, showing pronounced dominance of hard, negatively aligned prompts in pass@k weighted distribution.

Figure 5: Llama agreement/weight pattern with negative pass@k/pass@1 gradient inner product, consistent with theoretical prediction.

Figure 6: Illustrative case ($\delta_1 = 0.80$, $\delta_2 = 0.10$): despite high easy prompt prevalence, weighted negative alignment prevails.

Analysis of the scatter of weights versus alignment demonstrates the mechanism: pass@k weights are consistently concentrated on low pass@1, negatively aligned (hard) prompts.

Figure 7: Distribution of prompt-level agreement scores, with prominent mass of negative values indicating strong potential for gradient conflict.

Figure 8: Population-level pass@5 vs pass@1: pass@5 improves, pass@1 degrades under pass@k updates.

Implications, Limitations, and Future Directions

The results provide necessary theoretical caution for deployment of inference-aware finetuning. If pass@1 is an operational requirement (due to cost, latency, or fallback constraints), naive pass@k optimization may be counterproductive or even detrimental. The effect emerges from shared parameterization and heterogeneity of prompt statistics in the target domain, paralleling analogous results in multi-task learning.

Practically, this suggests that careful monitoring for negative interference and tailored prompt reweighting or gradient surgery (using similarity metrics or interference structure) is essential. Alternatives such as risk-sensitive or convex-combination inference objectives, or methods that directly circumvent gradient conflict (e.g., task-specific adapters or multi-objective optimization), merit further investigation. The theoretical framework developed also invites connections to advances in multi-task interference mitigation and controlled policy update schedules.

Conclusion

The work provides a comprehensive theoretical and empirical platform for understanding the origin and pervasiveness of pass@1 degradation under pass@k policy optimization, rooting it in the amplification of negatively interfering prompts by the pass@k objective. The results highlight a fundamental caveat for application of inference-time-aware finetuning, especially as LLMs are deployed in increasingly complex and mission-critical domains. Further work on mitigation strategies tailored to the identified prompt interference structure represents a crucial avenue for robust LLM deployment.