boltbeam.fyi

Qwen3.8-Flash-Next UD-Q4_K_XL on H200

GGUF Q8_0+Q4_K

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 23:06 UTC

100.4 tok/s

14% of the roofline709.1 tok/s possible

now, a token
9.96 ms
floor
1.41 ms
read, a token
6.57 GB

Speed at context

128 tokens
llama.cpp100.9 tok/s of 710.1
512 tokens
llama.cpp100.4 tok/s of 709.1
The pink tick is the roofline at that context.

Where the time goes

Now, a token
9.96 ms
Roofline floor, a token
1.41 ms
Gap to the floor
8.55 ms
Kernels a token
498

Launch heavy: 3472 kernel launches per token

Cannot split: ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, attn_qo BF16, attn_kv BF16, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, attn_qo BF16, attn_kv BF16, moe_router F32. No kernel group matched their calls and bytes

Not classified: blk.*.hc_attn_down.weight Q8_0, blk.*.hc_attn_up.weight Q8_0, blk.*.ple_key.weight Q8_0, blk.*.ple_value.weight Q8_0, output_hc_down.weight Q8_0, output_hc_up.weight Q8_0. 100 tensors, 0.38 GB per token, not classified; counted in the limit

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

moe_expert_gate_up Q4_K0.98 / 0.19 ms 19%
attn_qkv Q8_00.75 / 0.22 ms 29%
moe_shared_expert_gate_up Q8_00.44 / 0.04 ms 8%
attn_gate Q8_00.48 / 0.13 ms 27%
moe_expert_down Q5_10.41 / 0.11 ms 28%
lm_head Q8_00.44 / 0.14 ms 33%
ffn_gate_up Q8_00.26 / 0.04 ms 14%
ssm_out Q8_00.33 / 0.13 ms 39%
ssm_alpha_beta F320.20 / 0.01 ms 4%
attention0.13 / 0.00 ms 2%
attn_kv Q8_00.07 / 0.01 ms 10%
moe_expert_down Q8_00.05 / 0.02 ms 41%
moe_expert_gate_up Q5_K0.02 / 0.00 ms 21%
weight kernels, unattributed1.21 ms · no floor
other0.93 ms · no floor
kernel gaps, at most0.65 ms · no floor
quantize0.61 ms · no floor
attn qo0.46 ms · no floor
8 other small kernels1.54 ms · no floor
Per kernel (3): ms now / ms at the roofline, launches a token
mul_mat_vec_q · Q8_05.27 ms · no floor ×402
mul_mat_vec_q · Q4_K0.70 / 0.19 ms ×94
mul_mat_vec_q · Q5_K0.02 / 0.00 ms ×2

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run h200-flash-next-q4kxl-llama

Tested on

GPU
H200 · 140 GB
System RAM
1511 GB
CPU threads
256
Memory, measured
4662 GB/s
Temp, max
see notes
Power
see notes
SM clock
–
Driver
595.91.07
Measured
2026-11-09 23:06 UTC · context 512 tokens · measured, in-model (nsys): 12 of 21 weight roles timed

Qwen3.8-Flash-Next on other GPUs