What If We Allocate Test-Time Compute Adaptively?

Abstract: Test-time compute scaling allocates inference computation uniformly, uses fixed sampling strategies, and applies verification only for reranking. In contrast, we propose a verifier-guided adaptive framework treating reasoning as iterative trajectory generation and selection. For each problem, the agent runs multiple inference iterations. In each iteration, it optionally produces a high-level plan, selects a set of reasoning tools and a compute strategy together with an exploration parameter, and then generates a candidate reasoning trajectory. A process reward model (PRM) serves as a unified control signal: within each iteration, step-level PRM scores are aggregated to guide pruning and expansion during generation, and across iterations, aggregated trajectory rewards are used to select the final response. Across datasets, our dynamic, PRM-guided approach consistently outperforms direct test-time scaling, yielding large gains on MATH-500 and several-fold improvements on harder benchmarks such as AIME24 and AMO-Bench. We characterize efficiency using theoretical FLOPs and a compute intensity metric penalizing wasted generation and tool overhead, demonstrating that verification-guided allocation concentrates computation on high-utility reasoning paths.

• The paper presents a verifier-guided adaptive inference framework that dynamically allocates compute using a Process Reward Model (PRM) to score candidate reasoning trajectories.
• It achieves significant empirical gains with accuracy improvements of 10% to over 20% on benchmarks like MATH-500, AIME24, and AMO-Bench while optimizing compute utilization.
• The approach redefines LLM reasoning as an iterative decision-making process, shifting from static uniform computation to dynamic candidate-based strategies.

Adaptive Test-Time Compute Allocation

Introduction

The paper "What If We Allocate Test-Time Compute Adaptively?" (2602.01070) proposes a novel approach to inference in LLMs by considering adaptive test-time compute allocation. Traditional inference methods often apply computation uniformly, which can be inefficient, particularly for tasks requiring variable difficulty and diverse reasoning strategies. This work addresses these limitations by implementing a verifier-guided, adaptive framework where reasoning is treated as an iterative decision-making process.

Methodology

The core innovation lies in rethinking how inference time computation is allocated by using a dynamic, verifier-guided approach. The proposed system comprises multiple rounds of inference, where each iteration involves planning, tool selection, compute strategy definition, and candidate trajectory generation. In this context, a Process Reward Model (PRM) is central to guiding both intra-iteration and inter-iteration decisions. PRM scores are utilized to aggregate step-level rewards, which are crucial in deciding the final response from multiple candidate reasoning trajectories. This methodology ensures that computation is concentrated on high-utility reasoning paths, leading to performance improvements.

Figure 1: Universal reasoning agent architecture. The system generates $K$ candidate trajectories $\{I_1,\ldots,I_K\}$ scored by a PRM, with each iteration following a structured workflow.

Empirical Results

The application of this adaptive framework shows significant empirical gains over traditional static methods. Key results include improved accuracy in datasets such as MATH-500, AIME24, and AMO-Bench, with gains ranging from 10% to over 20%. The paper demonstrates that adaptive inference not only increases accuracy but also enhances compute efficiency. Two metrics, theoretical FLOPs and a compute intensity score, are introduced to evaluate efficiency, revealing that the proposed method optimizes compute utilization by focusing resources on promising trajectories.

Figure 2: Llama-3.1-8B-Instruct model performance comparison.

Figure 3: MATH-500: Tool usage shows model-dependent behavior with dynamic adaptation.

Discussion

The implications of this work are multifaceted. Practically, the framework can be employed in any reasoning-intensive domain where intermediate verification signals can be extracted. Theoretically, it provides a compelling case for treating reasoning within LLMs as a trajectory-level decision-making process rather than a linear generation task. The adaptive approach shifts the paradigm in LLM inference from static, uniform compute strategies to dynamic, verifier-assisted learning, promising more reliable and efficient model performance.

Future Work

Several areas are ripe for exploration following this study. Enhancing the quality and robustness of PRMs, especially for complex problems, remains a priority, as their efficacy is critical to the proposed framework. Exploring domain-specific adaptations, expanding beyond mathematical reasoning into areas such as code generation and complex decision-making, would further validate the versatility of the approach. Additionally, integrating learning mechanisms for adaptive controllers and policies could further optimize compute allocation.

Conclusion

This paper establishes a new direction in LLM inference by introducing adaptive test-time compute allocation guided by a PRM. The framework delivers substantial improvements in accuracy and compute efficiency, demonstrating the potential of adaptive strategies to enhance reasoning performance. Such innovations underscore the importance of rethinking inference methodologies to harness the full potential of LLMs in solving complex reasoning problems.