TheModelDesk

A Reddit user has shared early production numbers for Qwen3.8-Flash-Next running on a Dell Precision 7960 Rack workstation with two Xeon Platinum 8562Y+ CPUs, 512 GB DDR5 ECC RAM and two RTX PRO 6000 Blackwell Max-Q GPUs with 96 GB VRAM each. The FP8 model runs under vLLM with tensor parallelism of 2 and a 262K context window.

VLLM_PLE_CPU_OFFLOAD=1 vllm serve Qwen/Qwen3.8-Flash-Next-FP8 --tensor-parallel-size 2

The model includes a 51B-parameter PLE/n-gram embedding table, which the user offloads to system RAM via VLLM_PLE_CPU_OFFLOAD=1. GPU-side model load ended up around 67.5 GiB per GPU, leaving room for the KV cache.

Speculative decoding configuration proved critical. With MTP3, generation speed dropped to 40-48 tok/s and acceptance rates fell sharply across positions. Switching to MTP1 produced sustained generation of about 123-126 tok/s, with prompt processing peaking around 2,185 tok/s. Acceptance reached 99-100%, with a mean acceptance length of roughly 2.0.

The author notes the speed is close to what Qwen3.5-122B-A10B achieved on the same hardware, despite the different architecture. Further tests are planned: MTP1 versus no speculative decoding, tuning the missing MoE kernel config for E=512 / N=320 / FP8, higher GPU memory utilization, NUMA effects given the CPU-resident PLE table, longer-context performance, concurrent-user throughput and a quality comparison against Qwen3.5-122B-A10B.