boltbeam.fyi

Qwen3.8-27B UD-Q4_K_M on L40S

GGUF IQ4_XS+Q5_K

The tested file was 16.46 GB. The repo's current file is 16.56 GB: it was re-uploaded after the run.

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 19:35 UTC

40.0 tok/s

83% of the roofline48.4 tok/s possible

now, a token
24.97 ms
floor
20.66 ms
read, a token
15.74 GB
More
memory, measured
762 GB/s

What was run

llama-bench -m Qwen3.8-27B-UD-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1

Speed at context

128 tokens
llama.cpp40.3 tok/s of 48.5
512 tokens
llama.cpp40.0 tok/s of 48.4
The pink tick is the roofline at that context.

Where the time goes

Now, a token
24.97 ms
Roofline floor, a token
20.66 ms
Gap to the floor
4.31 ms
Kernels a token
497

Launch heavy: 1914 kernel launches per token

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up IQ4_XS4.20 / 3.80 ms 90%
ffn_gate_up Q5_K2.53 / 2.25 ms 89%
ffn_gate_up Q4_K2.07 / 1.84 ms 89%
ssm_alpha_beta Q8_00.20 / 0.03 ms 17%
attention0.18 / 0.05 ms 25%
ffn_down Q5_K1.87 / 1.77 ms 95%
ffn_down IQ4_XS1.59 / 1.49 ms 94%
ssm_out Q5_K1.03 / 0.94 ms 91%
attn_qkv Q4_K1.20 / 1.12 ms 94%
ffn_down Q4_K0.72 / 0.66 ms 91%
lm_head Q6_K1.44 / 1.37 ms 95%
attn_gate Q4_K0.62 / 0.56 ms 90%
ffn_gate_up Q3_K0.35 / 0.30 ms 87%
attn_qkv Q5_K0.61 / 0.57 ms 93%
attn_gate Q5_K0.37 / 0.34 ms 91%
attn_kv Q6_K0.09 / 0.07 ms 74%
attn_gate IQ4_XS0.29 / 0.26 ms 92%
ffn_down IQ4_NL0.22 / 0.20 ms 90%
ffn_down IQ3_S0.17 / 0.15 ms 88%
attn_kv Q5_K0.07 / 0.05 ms 74%
attn_qo Q4_K0.30 / 0.28 ms 94%
ffn_gate_up IQ4_NL0.21 / 0.20 ms 92%
attn_qo Q6_K0.15 / 0.14 ms 89%
attn_kv Q8_00.07 / 0.06 ms 80%
ssm_out Q6_K0.18 / 0.17 ms 93%
ssm_out Q4_K0.13 / 0.12 ms 91%
ffn_down Q6_K0.11 / 0.10 ms 90%
ssm_out IQ4_XS0.12 / 0.11 ms 92%
attn_qkv IQ4_XS0.23 / 0.22 ms 96%
ffn_down Q3_K0.06 / 0.05 ms 88%
ffn_gate_up Q6_K0.10 / 0.10 ms 94%
ffn_gate_up IQ3_S0.05 / 0.05 ms 94%
attn_qkv IQ4_NL0.04 / 0.04 ms 94%
attn_kv Q4_K0.01 / 0.00 ms 70%
norm0.72 ms · no floor
copy0.55 ms · no floor
attn qo0.54 ms · no floor
quantize0.53 ms · no floor
weight kernels, unattributed0.31 ms · no floor
7 other small kernels0.75 ms · no floor
Per kernel (8): ms now / ms at the roofline, launches a token
mul_mat_vec_q · Q5_K7.64 ms · no floor ×131
mul_mat_vec_q · IQ4_XS7.23 ms · no floor ×117
mul_mat_vec_q · Q4_K5.59 / 4.58 ms ×103
mul_mat_vec_q · Q6_K2.22 / 1.93 ms ×24
mul_mat_vec_q · Q8_01.05 / 0.09 ms ×104
mul_mat_vec_q · IQ4_NL0.51 / 0.43 ms ×7
mul_mat_vec_q · Q3_K0.45 / 0.35 ms ×7
mul_mat_vec_q · IQ3_S0.25 / 0.20 ms ×4

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run l40s-in-model

Tested on

GPU
L40S · 45 GB
System RAM
2016 GB
CPU threads
192
Memory, measured
762 GB/s
Temp, max
54 °C
Power
129 W mean of 350 W
SM clock
2501 MHz, throttled
Driver
570.211.01
Measured
2026-11-09 19:35 UTC · context 512 tokens · measured, in-model (nsys): 33 of 39 weight roles timed

Qwen3.8-27B on other GPUs