boltbeam.fyi

Qwen3.8-27B UD-Q4_K_M on RTX 3090

GGUF IQ4_XS+Q5_K

The tested file was 16.46 GB. The repo's current file is 16.56 GB: it was re-uploaded after the run.

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 17:51 UTCnewer release at the time: 0.9.0

41.5 tok/s

73% of the roofline57.0 tok/s possible

now, a token
24.12 ms
floor
17.53 ms
read, a token
15.74 GB
More
memory, measured
898 GB/s

What was run

llama-bench -m Qwen3.8-27B-UD-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1

Speed at context

128 tokens
llama.cpp41.4 tok/s of 57.1
512 tokens
llama.cpp41.5 tok/s of 57.0
The pink tick is the roofline at that context.

Where the time goes

Now, a token
24.12 ms
Roofline floor, a token
17.53 ms
Gap to the floor
6.59 ms
Kernels a token
497

Launch heavy: 1914 kernel launches per token

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up Q3_K0.56 / 0.26 ms 46%
ffn_gate_up IQ4_XS3.50 / 3.22 ms 92%
ssm_alpha_beta Q8_00.28 / 0.03 ms 10%
attention0.23 / 0.04 ms 17%
attn_gate Q4_K0.65 / 0.47 ms 73%
ffn_gate_up Q4_K1.72 / 1.56 ms 91%
ffn_down Q5_K1.64 / 1.50 ms 92%
ffn_gate_up Q5_K2.05 / 1.91 ms 94%
attn_qkv Q4_K1.08 / 0.95 ms 88%
ssm_out Q5_K0.90 / 0.80 ms 88%
ffn_down IQ4_XS1.37 / 1.27 ms 92%
attn_gate Q5_K0.38 / 0.29 ms 77%
attn_gate IQ4_XS0.31 / 0.22 ms 72%
ffn_down IQ3_S0.21 / 0.13 ms 61%
ffn_down Q4_K0.62 / 0.56 ms 90%
lm_head Q6_K1.22 / 1.16 ms 96%
attn_qkv Q5_K0.53 / 0.48 ms 91%
ffn_down Q3_K0.09 / 0.04 ms 49%
attn_kv Q6_K0.10 / 0.06 ms 57%
attn_qo Q6_K0.15 / 0.12 ms 77%
attn_kv Q5_K0.08 / 0.04 ms 58%
ffn_gate_up IQ3_S0.07 / 0.04 ms 58%
attn_qo Q4_K0.26 / 0.24 ms 90%
attn_qkv IQ4_XS0.21 / 0.19 ms 89%
ssm_out Q6_K0.16 / 0.14 ms 88%
attn_kv Q8_00.07 / 0.05 ms 71%
ssm_out Q4_K0.12 / 0.10 ms 85%
ssm_out IQ4_XS0.11 / 0.09 ms 85%
ffn_down IQ4_NL0.18 / 0.17 ms 92%
ffn_down Q6_K0.09 / 0.08 ms 88%
ffn_gate_up IQ4_NL0.18 / 0.17 ms 94%
ffn_gate_up Q6_K0.09 / 0.08 ms 92%
attn_qkv IQ4_NL0.04 / 0.03 ms 89%
attn_kv Q4_K0.01 / 0.00 ms 52%
norm1.11 ms · no floor
quantize0.81 ms · no floor
copy0.71 ms · no floor
attn qo0.48 ms · no floor
kernel gaps, at most0.44 ms · no floor
7 other small kernels1.33 ms · no floor
Per kernel (8): ms now / ms at the roofline, launches a token
mul_mat_vec_q · Q5_K6.75 ms · no floor ×131
mul_mat_vec_q · IQ4_XS6.58 ms · no floor ×117
mul_mat_vec_q · Q4_K4.98 / 3.89 ms ×103
mul_mat_vec_q · Q6_K1.92 / 1.64 ms ×24
mul_mat_vec_q · Q8_01.33 / 0.08 ms ×104
mul_mat_vec_q · Q3_K0.57 / 0.30 ms ×7
mul_mat_vec_q · IQ4_NL0.45 / 0.37 ms ×7
mul_mat_vec_q · IQ3_S0.27 / 0.17 ms ×4

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run rtx3090-qwen3.8-27b-q4km-llama

Tested on

GPU
RTX 3090 · 24 GB
System RAM
252 GB
CPU threads
80
Memory, measured
898 GB/s
Temp, max
60 °C
Power
151 W mean of 280 W
SM clock
1847 MHz, throttled
Driver
595.71.05
Measured
2026-11-09 17:51 UTC · context 512 tokens · measured, in-model (nsys): 33 of 39 weight roles timed

Qwen3.8-27B on other GPUs