Qwen3.8-27B speed on every GPU
15 measured runs across 9 GPUs, fastest first.
| GPU | Quant | Engine | tok/s | Roofline | Share |
|---|---|---|---|---|---|
| H200 | W4A16-AWQ | vLLM | 100.8 | 299.1 | 34% |
| 2× Radeon R9700 | Quark W4A16 | vLLM + MTP k=2 | 92.3 | – | – |
| H100 80GB | W4A16-AWQ | vLLM | 87.8 | 211.9 | 41% |
| RTX 5090 | W4A16-AWQ | vLLM | 87.5 | 109.7 | 80% |
| H100 80GB | UD-Q4_K_M | llama.cpp | 83.6 | 205.7 | 41% |
| RTX 5090 | UD-Q4_K_M | llama.cpp | 81.9 | 108.7 | 75% |
| H200 NVL | UD-Q4_K_M | llama.cpp | 78.6 | 286.7 | 27% |
| 2× RTX 3090 | W4A16-AWQ | vLLM | 68.6 | 90.6 | 76% |
| 2× Radeon R9700 | Quark W4A16 | vLLM | 56.1 | – | – |
| RTX 4090 | W4A16-AWQ | vLLM | 54.4 | 61.7 | 88% |
| RTX 4090 | UD-Q4_K_M | llama.cpp | 50.4 | 61.1 | 82% |
| RTX 3090 | W4A16-AWQ | vLLM | 46.2 | 57.6 | 80% |
| L40S | W4A16-AWQ | vLLM | 43.6 | 48.9 | 89% |
| RTX 3090 | UD-Q4_K_M | llama.cpp | 41.5 | 57.0 | 73% |
| L40S | UD-Q4_K_M | llama.cpp | 40.0 | 48.4 | 83% |