boltbeam.fyi

Llama-3.3-70B-Instruct AWQ on 2× RTX 3090

awq INT4_G128_ZP+F16

Engines

vLLM

0.31.0 ↗measured 2026-11-09 02:12 UTC

35.2 tok/s

84% of the roofline42.1 tok/s possible

now, a token
28.42 ms
floor
23.75 ms
read, a token
21.15 GB

Speed at context

128 tokens
vLLM35.3 tok/s of 42.4
512 tokens
vLLM35.2 tok/s of 42.1
The pink tick is the roofline at that context.

Where the time goes

Now, a token
28.42 ms
Roofline floor, a token
23.75 ms
Gap to the floor
4.68 ms
Kernels a token
–

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

This run timed the whole step only, so there is no per-role split.

No kernel capture for this run.

Run 3090x2-vllm-70b-awq-row

Tested on

GPU
2× RTX 3090 · 24 GB each
System RAM
63 GB
CPU threads
8
Memory, measured
896 GB/s
Temp, max
see notes
Power
see notes
SM clock
–
Driver
595.112.07
Measured
2026-11-09 02:12 UTC · context 512 tokens · measured, in-model (nsys): the whole step only, no per-role time

Llama-3.3-70B-Instruct on other GPUs

Llama-3.3-70B-Instruct has only been measured on the 2× RTX 3090 so far.