Qwen3.8-Flash-Next GSQ-RCO IQ3_S on 2× Radeon R9700
GGUF GSQ-RCO IQ3_S, 2 shards, 54.8 + 28.8 GB
Engines
llama.cpp
master · 5193ef1d ↗measured 2026-11-13
39.4 tok/s
No roofline yet: written record
More
- memory, measured
- –
- range, short
- 34.8 to 40.4 tok/s
- ~11k context (build 7182ef7)
- 33.8 to 35.2 tok/s
- prompt, 10k tokens, -ub 2048
- 2,354 tok/s
all experts on the two GPUs (offload plan from boltbeam tune --measure), -ub 2048; -fa, -ctk and -t at defaultsllama.cpp + MTP k=3
master · 5193ef1d ↗measured 2026-11-13
72.5 tok/s
No roofline yet: written record
More
- memory, measured
- –
- range, short
- 53.0 to 89.5 tok/s
- tokens accepted a step
- 2.83
- MTP k=2 / k=4
- 61.1 / 64.0 tok/s
MTP draft 3 tokens a step. Long context refused: the model and the 4.1 GB head leave 1 to 2 GB a cardSpeed at context
short
Where the time goes
From written records: the raw run files are on the R9700 machine, so there is no roofline or kernel breakdown yet.
Tested on
- GPU
- 2× Radeon R9700 · 32 GB each
- System RAM
- 128 GB
- CPU threads
- –
- Memory, measured
- –
- Temp, max
- see notes
- Power
- see notes
- SM clock
- –
- Driver
- –
- Measured
- 2026-11-13 · written record: median 39.4 tok/s (34.8 to 40.4), short context
- Two Radeon AI PRO R9700 (gfx1201), 32 GB each, 128 GB of system RAM
- Source: BoltBeam docs/models-tested.md (2026-11-13) and docs/evidence/mtp-27b-rtx5090-r9700-20261027 (r9700/sources.md). Written records: the raw run files stay on the R9700 machine.
Qwen3.8-Flash-Next on other GPUs
RTX 5090llama.cpp + mods · GSQ-RCO IQ1_M226.1 tok/s · 43%H200llama.cpp · UD-Q4_K_XL100.4 tok/s · 14%2× Radeon R9700llama.cpp + MTP k=3 · GSQ-RCO IQ3_S72.5 tok/s
The fastest setup measured on each GPU, one user. The percentage is that GPU's own roofline.