DFlash: Block Diffusion for Flash Speculative Decoding

Abstract: Autoregressive LLMs deliver strong performance but require inherently sequential decoding, leading to high inference latency and poor GPU utilization. Speculative decoding mitigates this bottleneck by using a fast draft model whose outputs are verified in parallel by the target LLM; however, existing methods still rely on autoregressive drafting, which remains sequential and limits practical speedups. Diffusion LLMs offer a promising alternative by enabling parallel generation, but current diffusion models typically underperform compared with autoregressive models. In this paper, we introduce DFlash, a speculative decoding framework that employs a lightweight block diffusion model for parallel drafting. By generating draft tokens in a single forward pass and conditioning the draft model on context features extracted from the target model, DFlash enables efficient drafting with high-quality outputs and higher acceptance rates. Experiments show that DFlash achieves over 6x lossless acceleration across a range of models and tasks, delivering up to 2.5x higher speedup than the state-of-the-art speculative decoding method EAGLE-3.

• The paper presents a block diffusion speculative decoding approach that leverages a lightweight diffusion model to draft token blocks in parallel, achieving significant speedups.
• It employs conditioned injection of target model context via KV projections, ensuring high-fidelity context integration and robust draft quality.
• Empirical evaluations on models like Qwen3 and LLaMA demonstrate over 6× speedup compared to AR decoding, outperforming previous methods such as EAGLE-3.

DFlash: Block Diffusion for Flash Speculative Decoding

Motivation and Problem Setting

Despite the progress of LLMs in numerous tasks, production deployment of state-of-the-art LLMs is hindered by high inference latency due to the inherently sequential nature of autoregressive (AR) decoding. Speculative decoding attempts to mitigate this bottleneck by leveraging lightweight draft models for parallel token speculation, followed by verification by the main LLM. However, the ceiling of practical speedup in conventional approaches—including EAGLE-3—is limited by the sequential autoregressive nature of the drafting model and its constrained model capacity.

Recent work in diffusion LLMs (dLLMs) suggested a path to parallel decoding, but these models have either underperformed AR LLMs or incurred substantial costs due to the need for large-draft models and/or many denoising steps. The challenge is to design a speculative decoding protocol that supports high-quality, parallel token drafting with lightweight models while achieving high speedups and preserving exact generation quality.

DFlash: Methodology

DFlash introduces a speculative decoding framework using a lightweight block diffusion model as a parallel drafter, departing from prevailing sequential AR strategies. The primary architectural and algorithmic contributions include:

• Block Diffusion Drafting: The core of DFlash is a block diffusion model that generates a block of future tokens in parallel. This overcomes the sequential bottleneck that limits AR-based speculative decoders.
• Conditioned Draft Model via Target Context Injection: DFlash extracts hidden states from multiple layers of the target AR LLM and fuses them into a condensed context embedding. Crucially, these features are then injected directly into the Key-Value (KV) projections of every layer in the diffusion drafter, ensuring persistent access to high-fidelity context information throughout drafting (Figure 1).

  Figure 1: DFlash inference design conditions all draft layers on target model context features by direct KV cache injection, enabling highly effective speculation.

• Block-Diffusion Training Objective Adapted for Speculative Settings: Instead of dividing responses into fixed-length blocks, DFlash randomly samples anchor positions and constructs masked blocks dynamically. This simulates the inference-time context where each drafted block starts with a verified token from the target model (Figure 2).

  Figure 2: DFlash training overview with context feature injection, dynamic anchored block construction, and blockwise sparse causal attention.

• Loss Decay for Efficient Training: The cross-entropy loss is exponentially weighted to prioritize prediction accuracy at earlier positions in drafted blocks, reflecting the fact that errors earlier in the block cascade to invalidate larger portions of the draft (Figure 3).

  Figure 3: Loss decay accelerates model convergence and improves acceptance length by emphasizing token accuracy at the start of the block.

• Draft Model Reuse and Parameter Sharing: DFlash shares the input-output embeddings with the target LLM, keeping only the inner diffusion transformer layers trainable. This reduces the total trainable parameter count and ensures close representational alignment.

Analysis of Parallel Diffusion Speculation

In speculative decoding, speedup is driven both by increasing the expected number of accepted tokens per cycle ($\tau$) and by reducing drafting overhead. AR-based drafters suffer a drafting latency that scales linearly with the drafted block size, requiring shallow models to keep latency in check—this limits block sizes and degrades draft quality.

In contrast, the parallelism of the diffusion drafter enables deep, expressive block models without incurring additional inference cost (Figure 4).

Figure 4: Drafting costs for DFlash remain nearly constant as diffusion depth and block size increase; autoregressive EAGLE-3 drafters' cost grows linearly.

Crucially, richer context conditioning via KV injection allows draft quality and acceptance length to scale with increased model depth, unlike in EAGLE-3 where token embedding conditioning rapidly saturates.

Empirical Evaluation

Throughput and Speedup

DFlash delivers strong lossless acceleration for Qwen3 and LLaMA models. On Qwen3-8B, it achieves more than 6× speedup over standard AR decoding with up to 2.5× higher acceleration than EAGLE-3 (Figure 5). Average acceptance length per speculative cycle also increases significantly, supporting larger block sizes without sacrificing verifiability.

Figure 5: Speedups for DFlash and EAGLE-3 relative to autoregressive decoding for Qwen3-8B across multiple benchmarks.

DFlash's benefits persist across model sizes, deployment settings (including SGLang, FlashAttention-4, and high-concurrency scenarios), and various generation temperatures. The design is robust, as larger block sizes, deeper draft architectures, and richer context features reliably increase acceptance length at minimal speed cost.

Instruct, Reasoning, and Coding Benchmarks

Across math (GSM8K, MATH-500, AIME25), code (HumanEval, MBPP, LCB), and chat (MT-Bench) benchmarks, DFlash consistently surpasses EAGLE-3 by a wide margin for both greedy and stochastic sampling. The empirical Pareto frontier for draft quality and cost is strictly better than prior art.

Ablations

• Number of draft layers: Acceptance length continues to improve with more drafter layers, while speedup is maximized at a moderate depth (typically five layers).
• Number of target context features: Extracting from more layers boosts acceptance at the cost of additional compute during training.
• Training-to-inference block size transfer: Models trained with larger blocks generalize downwards smoothly, enabling dynamic scheduling in production.
• Conditioning mechanism: Eliminating target context or substituting input embedding fusion for KV injection substantially degrades quality and speedup.

Theoretical and Practical Implications

By exploiting parallel block diffusion, DFlash fundamentally shifts the speculative decoding design space. The mechanism of dense target context injection enables aggressive reduction in drafter model size without hurting acceptance length—a property not previously attainable in AR or unconstrained diffusion settings. The result is a scalable, low-latency speculative decoding protocol that democratizes high-performance LLM inference for both serving and edge scenarios.

On the theoretical side, DFlash reframes the utility of diffusion LLMs: the hybrid paradigm treats diffusion drafting as a specialized, context-conditioned adapter strictly for speculative acceleration, sidestepping known quality limitations in end-to-end diffusion generation. This division of labor—diffusion for batched speculation, autoregression for final quality—suggests further directions in model hybridization and hardware-efficient LLM serving.

On the practical side, DFlash is deployable within existing open-source frameworks, requires only small additional trainable modules, and demonstrates substantial wall-clock cost reductions in high-throughput settings. Dynamic scheduling of block size, context extraction, and model depth can be used to optimize for device-specific latency or throughput objectives.

Conclusions

DFlash presents a principled and empirically validated framework for block diffusion speculative decoding, achieving lossless acceleration that exceeds previous AR and diffusion-based methods by a significant margin. It demonstrates that a lightweight, highly-conditioned block diffusion model can serve as an optimal spearhead for speculative decoding. Future research could explore adaptive block sizing, tighter integration with target model training, and the application of similar hybrid diffusion-adapter techniques to other sequence modeling domains.