2× RTX 3090 for local LLMs
3 measured runs. 24 GB of VRAM; memory measured at 896 GB/s, which sets the roofline for every model below.
| Model | Quant | Engine | tok/s | Roofline | Share |
|---|---|---|---|---|---|
| Llama-3.3-70B-Instruct | AWQ | vLLM | 35.2 | 42.1 | 84% |
| Llama-3.3-70B-Instruct | Q4_K_M | llama.cpp | 19.2 | 21.3 | 90% |
| Qwen3.8-27B | W4A16-AWQ | vLLM | 68.6 | 90.6 | 76% |
Share = measured tok/s over the roofline: the bytes a token reads over this GPU's measured memory speed. Open a quant for the test machine and the per-role and per-kernel breakdown.