boltbeam.fyi

Qwen3-14B Q4_K_M on RTX 4090

GGUF Q4_K+Q6_K

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 19:39 UTC

94.4 tok/s

85% of the roofline111.2 tok/s possible

now, a token
10.59 ms
floor
8.99 ms
read, a token
8.65 GB
More
memory, measured
962 GB/s

What was run

llama-bench -m Qwen3-14B-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1

Speed at context

128 tokens
llama.cpp94.4 tok/s of 112.0
512 tokens
llama.cpp94.4 tok/s of 111.2
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
10.59 ms
Roofline floor, a token
8.99 ms
Gap to the floor
1.60 ms
Kernels a token
281

Launch heavy: 766 kernel launches per token

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

attention0.30 / 0.09 ms 31%
attn_qo Q4_K1.37 / 1.23 ms 90%
ffn_gate_up Q4_K4.31 / 4.17 ms 97%
attn_kv Q4_K0.29 / 0.18 ms 65%
ffn_down Q6_K1.59 / 1.52 ms 96%
ffn_down Q4_K1.09 / 1.04 ms 96%
attn_kv Q6_K0.12 / 0.09 ms 72%
lm_head Q6_K0.67 / 0.66 ms 99%
norm0.43 ms · no floor
quantize0.26 ms · no floor
kernel gaps, at most0.13 ms · no floor
KV cache write0.05 ms · no floor
copy0.00 ms · no floor
elementwise0.00 ms · no floor
Per kernel (2): ms now / ms at the roofline, launches a token
mul_mat_vec_q · Q4_K8.60 / 6.63 ms ×240
mul_mat_vec_q · Q6_K2.60 / 2.27 ms ×41

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run rtx4090-qwen3-14b-q4km-llama

Tested on

GPU
RTX 4090 · 24 GB
System RAM
189 GB
CPU threads
96
Memory, measured
962 GB/s
Temp, max
64 °C
Power
87 W mean of 450 W
SM clock
2640 MHz, throttled
Driver
595.58.03
Measured
2026-11-09 19:39 UTC · context 512 tokens · measured, in-model (nsys): 7 of 7 weight roles timed

Qwen3-14B on other GPUs