boltbeam.fyi

Qwen3-14B AWQ on RTX 5090

awq INT4_G128_ZP+BF16

Engines

vLLM

0.31.0 ↗measured 2026-11-09 20:41 UTCnewer release at the time: 0.33.0

151.5 tok/s

76% of the roofline200.8 tok/s possible

now, a token
6.60 ms
floor
4.98 ms
read, a token
8.52 GB

Speed at context

128 tokens
vLLM154.0 tok/s
512 tokens
vLLM151.5 tok/s of 200.8
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
6.60 ms
Roofline floor, a token
4.98 ms
Gap to the floor
1.62 ms
Kernels a token
460

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

attention0.26 / 0.05 ms 20%
ffn_gate_up INT4_G128_ZP2.37 / 2.17 ms 91%
attn_kv + attn_qo INT4_G128_ZP0.60 / 0.45 ms 74%
attn_qo INT4_G128_ZP0.46 / 0.32 ms 70%
ffn_down INT4_G128_ZP1.21 / 1.08 ms 90%
lm_head F160.99 / 0.91 ms 92%
triton_red_fused_fused_add_rms_norm_20.13 ms · no floor
kernel gaps0.13 ms · no floor
flash::flash_fwd_splitkv_combine_kernel0.12 ms · no floor
triton_red_fused_fused_add_rms_norm_00.11 ms · no floor
triton_red_fused_30.08 ms · no floor
19 other small kernels0.14 ms · no floor
Per kernel (26): ms now / ms at the roofline, launches a token
Marlin4.64 / 4.02 ms ×160
gemm · lm_head F160.99 / 0.91 ms ×1
flash_fwd_splitkv_kernel0.26 / 0.05 ms ×40
triton_red_fused_fused_add_rms_norm_20.13 ms · no floor ×40
flash::flash_fwd_splitkv_combine_kernel0.12 ms · no floor ×40
triton_red_fused_fused_add_rms_norm_00.11 ms · no floor ×40
triton_red_fused_30.08 ms · no floor ×39
vllm::reshape_and_cache_flash_kernel0.06 ms · no floor ×40
triton_poi_fused_mul_silu_slice_10.03 ms · no floor ×40
_gumbel_sample_kernel0.01 ms · no floor ×1
at::native::index_elementwise_kernel0.01 ms · no floor ×3
triton_red_fused__to_copy_embedding_rms…0.00 ms · no floor ×1
_bias_kernel0.00 ms · no floor ×1
at::native::reduce_kernel0.00 ms · no floor ×1
_post_update_kernel0.00 ms · no floor ×1
_gather_block_tables_kernel0.00 ms · no floor ×1
at::native::unrolled_elementwise_kernel0.00 ms · no floor ×1
_combine_sampled_and_draft_tokens_kernel0.00 ms · no floor ×1
triton_red_fused_10.00 ms · no floor ×1
_compute_slot_mappings_kernel0.00 ms · no floor ×1
_get_num_sampled_and_rejected_kernel0.00 ms · no floor ×1
at::native::vectorized_elementwise_kern…0.00 ms · no floor ×3
at::native::_scatter_gather_elementwise…0.00 ms · no floor ×1
_prepare_pos_seq_lens_kernel0.00 ms · no floor ×1
<unnamed>::elementwise_kernel_with_index0.00 ms · no floor ×1
_apply_write_kernel0.00 ms · no floor ×0

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run rtx5090-14b-vllm-b1

Tested on

GPU
RTX 5090 · 32 GB
System RAM
31 GB
CPU threads
16
Memory, measured
1710 GB/s
Temp, max
see notes
Power
see notes
SM clock
–
Driver
595.99.02
Measured
2026-11-09 20:41 UTC · context 512 tokens · measured, in-model (nsys): split by role from the engine's own labels, 5 weight roles

Qwen3-14B on other GPUs