H200 for local LLMs
3 measured runs. 140 GB of VRAM; memory measured at 4662 GB/s, which sets the roofline for every model below.
| Model | Quant | Engine | tok/s | Roofline | Share |
|---|---|---|---|---|---|
| Qwen3-14B | AWQ | vLLM | 166.9 | 547.3 | 30% |
| Qwen3.8-27B | W4A16-AWQ | vLLM | 100.8 | 299.1 | 34% |
| Qwen3.8-Flash-Next | UD-Q4_K_XL | llama.cpp | 100.4 | 709.1 | 14% |
Share = measured tok/s over the roofline: the bytes a token reads over this GPU's measured memory speed. Open a quant for the test machine and the per-role and per-kernel breakdown.