boltbeam.fyi

Qwen3-14B Q4_K_M on L40S

GGUF Q4_K+Q6_K

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 19:52 UTCnewer release at the time: 0.9.0

74.1 tok/s

84% of the roofline88.1 tok/s possible

now, a token
13.50 ms
floor
11.35 ms
read, a token
8.65 GB
More
memory, measured
762 GB/s

What was run

llama-bench -m Qwen3-14B-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1

Speed at context

128 tokens
llama.cpp74.8 tok/s of 88.7
512 tokens
llama.cpp74.1 tok/s of 88.1
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
13.50 ms
Roofline floor, a token
11.35 ms
Gap to the floor
2.14 ms
Kernels a token
281

Launch heavy: 766 kernel launches per token

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up Q4_K5.57 / 5.27 ms 95%
attention0.34 / 0.11 ms 34%
ffn_down Q6_K2.14 / 1.92 ms 90%
attn_qo Q4_K1.73 / 1.55 ms 90%
ffn_down Q4_K1.45 / 1.32 ms 91%
attn_kv Q4_K0.34 / 0.23 ms 68%
attn_kv Q6_K0.15 / 0.11 ms 74%
lm_head Q6_K0.88 / 0.84 ms 96%
norm0.45 ms · no floor
quantize0.28 ms · no floor
kernel gaps, at most0.12 ms · no floor
KV cache write0.05 ms · no floor
copy0.00 ms · no floor
elementwise0.00 ms · no floor
Per kernel (2): ms now / ms at the roofline, launches a token
mul_mat_vec_q · Q4_K10.52 / 8.37 ms ×240
mul_mat_vec_q · Q6_K3.30 / 2.87 ms ×41

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run l40s-qwen3-14b-q4km-llama

Tested on

GPU
L40S · 45 GB
System RAM
2016 GB
CPU threads
192
Memory, measured
762 GB/s
Temp, max
52 °C
Power
121 W mean of 350 W
SM clock
2496 MHz, throttled
Driver
570.211.01
Measured
2026-11-09 19:52 UTC · context 512 tokens · measured, in-model (nsys): 7 of 7 weight roles timed

Qwen3-14B on other GPUs