boltbeam.fyi

Local LLM speed, measured on real GPUs.

Pick a GPU, a model and a quant. Every number sits against its roofline: the fastest that GPU could ever run it.

Speed-ups

Qwen3.8-Flash-Next GSQ-RCO IQ1_M on RTX 5090

GGUF GSQ-RCO IQ1_M, 2 shards, 58.4 GB

Engines

llama.cpp

v1.0.0 · b1b770d9 ↗measured 2026-11-16 to 2026-11-17

77.2 tok/s

14% of the roofline537.1 tok/s possible

now, a step
44.53 ms
floor
6.40 ms
read, a step
5.76 GB
More
memory, measured
1707 GB/s
tokens a step
3.44 (MTP draft)
prefill per turn
2.61 s
time to first token
2.71 s
whole session
485 s
stock: -c 131072, q8_0 KV, MTP draft head, every other setting default

Speed at context

<32k
llama.cpp96.0 tok/s of 544.2
32-64k
llama.cpp86.0 tok/s of 551.7
64-96k
llama.cpp74.2 tok/s of 545.7
96-128k
llama.cpp61.3 tok/s of 510.2
Decode tok/s by context depth at the start of each turn, over a 59-turn session that grows to 129k tokens. The pink tick is the roofline.

Where the time goes

Now, a step
44.53 ms
Roofline floor, a step
6.40 ms
Gap to the floor
38.13 ms
Kernels a step
–

Floor shown is the model's bytes a step read from GPU memory. Stock keeps more experts in system RAM, so its own floor is higher.

Per role ms a step. floor (bytes ÷ measured memory speed) above the floor

This run timed the whole step only, so there is no per-role split.

No kernel capture for this run.

Run flash-next-128k-stock

Tested on

GPU
RTX 5090 · 32 GB
System RAM
31 GB
CPU threads
16
Memory, measured
1707 GB/s
Temp, max
see notes
Power
see notes
SM clock
–
Driver
595.99.02
Measured
2026-11-16 to 2026-11-17 · context 131072 tokens · 59-turn session to 129.3k tokens, 3 paired runs, temperature 0
  • GPU power capped at 450 W, reset before every run
  • start gate: GPU at or under 70 °C, board under 58 °C
  • CPU: Ryzen 7 5800X · RAM: 32 GB DDR4-2667 · PCIe 4.0 x16 (B550)
  • MoE model larger than VRAM: 16 layers of experts in system RAM
  • roofline: 5.76 GB a decode step (3.38 tokens) / 1,707 GB/s measured = 6.4 ms

Qwen3.8-Flash-Next on other GPUs