Deploying DFlash block diffusion on NVIDIA hardware accelerates autoregressive LLMs during latency-sensitive inference.
Autoregressive large language models generate tokens sequentially. This sequential execution restricts GPU utilisation and constrains throughput in latency-sensitive serving scenarios, particularly as AI systems transition from single-turn interactions to coordinated multi-agent workflows.
Speculative decoding addresses this restriction by deploying a lightweight model to draft future tokens, allowing the larger target model to verify them in parallel. The standard speculative approach relies on autoregressive draft models that still generate tokens sequentially. Drafting costs escalate quickly as the volume of speculative tokens grows, imposing a hard limit on system throughput.
DFlash, an open-source lightweight block diffusion model, replaces the autoregressive drafter entirely. The DFlash drafter predicts an entire block of masked future tokens in a single forward pass instead of producing tokens individually. This turns sequential drafting into block-parallel GPU compute while verifying against the target model to preserve output quality.
NVIDIA hardware throughput metrics
Testing on an eight NVIDIA DGX B300 system running the gpt-oss-120b model with TensorRT-LLM demonstrates the production viability of this method. Utilising the SPEED-Bench coding dataset, DFlash delivers higher throughput across production-relevant latency targets compared with standard autoregressive decoding.
Operating at high interactivity ranges of 500-600 tokens per second per user, DFlash increases throughput on NVIDIA Blackwell architecture by more than 15x compared with autoregressive decoding. This output rate achieves 1.5x higher throughput than EAGLE-3 speculative decoding. Executing at the lowest concurrency point with a batch size of one, the DFlash model more than doubles interactivity on Blackwell hardware.
Engineering teams serving interactive coding, reasoning, and agentic workflows must maintain strict per-user token latency parameters while scaling concurrency. DFlash addresses this requirement by parallelising the speculative decode path. The block-diffusion drafter generates multiple candidate tokens simultaneously, passing them to the target model for parallel verification.
Large language model inference in decode-constrained environments often faces limitations from memory movement and sequential token generation rather than raw compute availability. Relocating execution into parallel block drafting and verification enables the hardware to execute more available compute operations while sustaining the targeted interactivity rate.
The NVIDIA Blackwell Ultra GPU architecture aligns with this parallelised execution model. Each Blackwell Ultra GPU features two reticle-sized dies connected by 10tbps of high-bandwidth chip-to-chip interconnect. This hardware configuration forms a unified compute domain containing 160 streaming multiprocessors and 640 fifth-generation Tensor Cores.
DFlash exposes more parallel operations to Blackwell’s 15 petaflops of dense NVFP4 compute, allowing the system to serve up to 15x more users concurrently at identical interactivity rates. Performance gains apply equally to smaller models. DFlash nearly doubles the interactivity of EAGLE-3 on Llama 3.1 8B when processing the Speed-Bench multilingual dataset.
Comparative performance of DFlash across specialised datasets
DFlash consistently outperforms EAGLE-3 when measuring interactivity speedups at matched user concurrency levels.
Processing the gpt-oss-120b model, DFlash achieves a 2.6x speedup on coding tasks compared to EAGLE-3’s 1.8x. Retrieval-augmented generation workloads observe DFlash reaching a 2.3x speedup versus 1.7x for EAGLE-3. Multilingual processing demonstrates a 2.6x improvement with DFlash, contrasting with the 1.8x recorded by EAGLE-3.
These impressive workload metrics yield an average speedup of 2.3x across tested datasets for DFlash on the 120-billion parameter model, scaling consistently across reasoning, writing, and summarisation tasks.
The smaller Llama 3.1 8B Instruct model exhibits similar performance scaling characteristics under comparative testing. Coding tasks process at a 3.0x speedup utilising DFlash, while EAGLE-3 achieves 2.3x. Retrieval-augmented generation hits a 3.1x speedup with DFlash. The average speedup across all tested datasets for the eight billion parameter model reaches 2.8x with DFlash, compared to 2.2x for EAGLE-3.
DFlash integration in vLLM and SGLang
The research team released 20 DFlash model checkpoints on Hugging Face, providing recipes for both NVIDIA Blackwell and Hopper GPUs. These cover several major model families, including Qwen, Kimi K2.6, Llama, Gemma, and gpt-oss.
Engineers managing vLLM environments can replace EAGLE-3 with a DFlash checkpoint through a simple configuration update. The integration operates via the open-source Speculators library, connecting the DFlash drafter to the target model’s hidden states within the vLLM inference path on NVIDIA GPUs.
Benchmarking Gemma 4 31B on a single Blackwell Ultra GPU using vLLM demonstrates up to 5.8x higher throughput at matched concurrency over standard autoregressive decoding. This vLLM configuration yields a 5.8x throughput increase on the Math500 dataset. HumanEval tasks register a 5.6x improvement, and GSM8K benchmarks record a 5.3x gain.
Migrating from EAGLE in deployments running SGLang requires updating the speculative decoding algorithm to DFlash and supplying the corresponding DFlash draft model checkpoint.
Evaluating Qwen3 8-B on SGLang using a single NVIDIA B200 GPU yields up to 5.1x throughput improvements at matched concurrency over autoregressive decoding. The SGLang integration produces a 5.1x throughput increase for Math500 tasks and a 4.2x gain for HumanEval tasks when compared to standard autoregressive processing.
The internal architecture of DFlash relies on three technical mechanisms:
- Block-diffusion drafting predicts multiple future tokens in parallel.
- Target hidden-state conditioning extracts context features directly from the target model.
- Key-value injection inserts these extracted target context features into the draft model’s key-value projections across its operational layers.
The key-value injection mechanism maintains high acceptance rates during the verification phase. The target model exclusively performs the final validation pass, ensuring the system preserves the exact output distribution of the primary model while accelerating the overall generation sequence.
Open model checkpoints remain accessible across NVIDIA hardware through integrated support within SGLang, vLLM, and TensorRT-LLM frameworks.
See also: OpenAI deploys GPT-5.5-Cyber for open-source vulnerability fixes
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events including the Cyber Security & Cloud Expo. Click here for more information.
Developer is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.