boltbeam.fyi

Qwen3.8-27B UD-Q4_K_M on RTX 4090

GGUF IQ4_XS+Q5_K

The tested file was 16.46 GB. The repo's current file is 16.56 GB: it was re-uploaded after the run.

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 19:21 UTC

50.4 tok/s

82% of the roofline61.1 tok/s possible

now, a token
19.85 ms
floor
16.36 ms
read, a token
15.74 GB
More
memory, measured
962 GB/s

What was run

llama-bench -m Qwen3.8-27B-UD-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1

Speed at context

128 tokens
llama.cpp50.0 tok/s of 61.2
512 tokens
llama.cpp50.4 tok/s of 61.1
The pink tick is the roofline at that context.

Where the time goes

Now, a token
19.85 ms
Roofline floor, a token
16.36 ms
Gap to the floor
3.49 ms
Kernels a token
497

Launch heavy: 1914 kernel launches per token

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up IQ4_XS3.30 / 3.01 ms 91%
ssm_alpha_beta Q8_00.18 / 0.03 ms 15%
ffn_gate_up Q4_K1.60 / 1.46 ms 91%
ffn_gate_up Q5_K1.91 / 1.78 ms 93%
attention0.16 / 0.04 ms 22%
ssm_out Q5_K0.80 / 0.74 ms 92%
attn_qkv Q4_K0.94 / 0.89 ms 94%
ffn_down IQ4_XS1.23 / 1.18 ms 96%
ffn_down Q5_K1.45 / 1.40 ms 97%
ffn_gate_up Q3_K0.28 / 0.24 ms 85%
attn_gate Q4_K0.48 / 0.44 ms 93%
ffn_down Q4_K0.55 / 0.52 ms 96%
attn_qkv Q5_K0.47 / 0.45 ms 95%
attn_gate Q5_K0.29 / 0.27 ms 93%
attn_kv Q6_K0.07 / 0.05 ms 73%
attn_gate IQ4_XS0.23 / 0.21 ms 92%
attn_kv Q5_K0.06 / 0.04 ms 71%
ffn_gate_up IQ4_NL0.17 / 0.16 ms 92%
ffn_down IQ3_S0.13 / 0.12 ms 90%
attn_kv Q8_00.06 / 0.05 ms 80%
attn_qkv IQ4_XS0.18 / 0.17 ms 94%
attn_qo Q4_K0.23 / 0.22 ms 95%
lm_head Q6_K1.10 / 1.08 ms 99%
ssm_out Q6_K0.14 / 0.13 ms 93%
attn_qo Q6_K0.12 / 0.11 ms 93%
ssm_out Q4_K0.10 / 0.09 ms 92%
ssm_out IQ4_XS0.09 / 0.09 ms 92%
ffn_down IQ4_NL0.16 / 0.16 ms 96%
ffn_down Q3_K0.05 / 0.04 ms 87%
ffn_down Q6_K0.08 / 0.08 ms 95%
ffn_gate_up IQ3_S0.04 / 0.04 ms 92%
ffn_gate_up Q6_K0.08 / 0.08 ms 97%
attn_qkv IQ4_NL0.03 / 0.03 ms 94%
attn_kv Q4_K0.00 / 0.00 ms 66%
norm0.68 ms · no floor
quantize0.49 ms · no floor
copy0.48 ms · no floor
attn qo0.43 ms · no floor
weight kernels, unattributed0.24 ms · no floor
7 other small kernels0.74 ms · no floor
Per kernel (8): ms now / ms at the roofline, launches a token
mul_mat_vec_q · Q5_K6.09 ms · no floor ×131
mul_mat_vec_q · IQ4_XS5.90 ms · no floor ×117
mul_mat_vec_q · Q4_K4.49 / 3.63 ms ×103
mul_mat_vec_q · Q6_K1.76 / 1.53 ms ×24
mul_mat_vec_q · Q8_01.01 / 0.07 ms ×104
mul_mat_vec_q · IQ4_NL0.40 / 0.34 ms ×7
mul_mat_vec_q · Q3_K0.34 / 0.28 ms ×7
mul_mat_vec_q · IQ3_S0.20 / 0.16 ms ×4

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run rtx4090-in-model

Tested on

GPU
RTX 4090 · 24 GB
System RAM
189 GB
CPU threads
96
Memory, measured
962 GB/s
Temp, max
67 °C
Power
89 W mean of 450 W
SM clock
2626 MHz, throttled
Driver
595.58.03
Measured
2026-11-09 19:21 UTC · context 512 tokens · measured, in-model (nsys): 33 of 39 weight roles timed

Qwen3.8-27B on other GPUs