Adaptive Inference-Time Scaling via Cyclic Diffusion Search

Abstract: Diffusion models have demonstrated strong generative capabilities across domains ranging from image synthesis to complex reasoning tasks. However, most inference-time scaling methods rely on fixed denoising schedules, limiting their ability to allocate computation based on instance difficulty or task-specific demands adaptively. We introduce the challenge of adaptive inference-time scaling-dynamically adjusting computational effort during inference-and propose Adaptive Bi-directional Cyclic Diffusion (ABCD), a flexible, search-based inference framework. ABCD refines outputs through bi-directional diffusion cycles while adaptively controlling exploration depth and termination. It comprises three components: Cyclic Diffusion Search, Automatic Exploration-Exploitation Balancing, and Adaptive Thinking Time. Experiments show that ABCD improves performance across diverse tasks while maintaining computational efficiency.

• The paper presents the Adaptive Bi-directional Cyclic Diffusion (ABCD) framework, which dynamically allocates computational resources during inference.
• It introduces cyclic diffusion search and automatic exploration-exploitation balancing to enhance solution quality across tasks like Sudoku and maze navigation.
• Experimental results demonstrate improved accuracy and efficiency, indicating substantial potential for practical adaptive inference applications.

Adaptive Inference-Time Scaling via Cyclic Diffusion Search

The paper "Adaptive Inference-Time Scaling via Cyclic Diffusion Search" (2505.14036) presents Adaptive Bi-directional Cyclic Diffusion (ABCD), a novel framework aimed at improving inference-time scaling for diffusion models. The central premise is to allow adaptive allocation of computational resources based on task difficulty and input complexity, thus overcoming limitations of traditional fixed-step inference approaches.

Problem and Motivation

Diffusion models excel in generating complex multimodal distributions through hierarchical denoising processes, making them suitable for tasks ranging from image synthesis to strategic planning. Despite their effectiveness, a significant challenge is adaptive inference-time scaling—adjusting computational intensity dynamically during inference. Existing methods predominantly leverage static denoising sequences, limiting their responsiveness to instance-specific demands, thus often making inefficient use of computational resources for varying task complexities.

Methodology: Adaptive Bi-directional Cyclic Diffusion

ABCD reframes diffusion model inference as an efficient search-based process, optimizing computation allocation through three key components:

1. Cyclic Diffusion Search (CDS): This mechanism iteratively refines outputs by alternating between denoising and re-noising steps, facilitating exploration of alternative solutions and avoiding local minima entrapment.

• Process: Initiating with a diverse particle set from a Gaussian prior, a coarse denoising trajectory via DDIM is executed swiftly to acquire initial solutions. This iterative process allows the exploration of a broader solution space by reintroducing stochastic elements through a controlled re-noising step.

1. Automatic Exploration-Exploitation Balancing (AEEB): AEEB dynamically adjusts exploration depth by varying the re-noising levels, allowing instance-specific computation allocation. The process detects optimal exploration depths by distributing particles among predefined 'temperature' levels, enhancing adaptability across diverse tasks.
2. Adaptive Thinking Time (ATT): ATT terminates the search process when solution quality metrics indicate diminishing refinement returns. This component ensures the judicious use of computational resources by halting further computation upon detecting stalled improvement in solution quality.

Experimental Evaluation

ABCD’s effectiveness was validated across multiple challenging domains:

• Sudoku Puzzle Completion: Displayed superior accuracy with reduced computational time compared to baselines, achieving consistent performance across varied difficulty levels.
• Pixel Maze Navigation: Demonstrated faster convergence to high-success-rate solutions in unfamiliar and complex environments.
• Molecule Generation and Text-to-Image Generation: Achieved higher stability and human preference alignment, respectively, within competitive computational budgets.

In comparative assessments, ABCD outperformed several state-of-the-art methods such as Best-of-N, Sequential Monte Carlo Diffusion, Beam Search, and Search-over-Path by showcasing enhanced adaptive computation scaling and superior performance metrics. Notably, ABCD achieved near-perfect success rates in tasks emphasizing logical consistency and structured spatial navigation.

Implications and Future Work

The scalability up to moderately complex domains coupled with an empirical demonstration of performance gains suggests the potential for practical application in environments where adaptive resource allocation is key. Addressing scalability to high-dimensional tasks like high-resolution image synthesis may involve incorporating hierarchical enhancements or structural priors.

Future research might explore learning-based approaches to amortize the adaptive search performed by ABCD, potentially reducing inference complexity through offline optimization strategies that capture task-specific dynamics.

Conclusion

ABCD represents a significant advancement in adaptive inference-time scaling for diffusion models, offering a robust framework that judiciously balances computational efficiency with task-specific demands. By integrating cyclic refinement with adaptive search and termination strategies, it significantly enhances performance across diverse domains while maintaining computational efficiency.