boltbeam.fyi

Qwen3-14B AWQ on H200

awq INT4_G128_ZP+BF16

Engines

vLLM

0.31.0 ↗measured 2026-11-09 22:57 UTC

166.9 tok/s

30% of the roofline547.3 tok/s possible

now, a token
5.99 ms
floor
1.83 ms
read, a token
8.52 GB

Speed at context

128 tokens
vLLM169.5 tok/s
512 tokens
vLLM166.9 tok/s of 547.3
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
5.99 ms
Roofline floor, a token
1.83 ms
Gap to the floor
4.16 ms
Kernels a token
462

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up INT4_G128_ZP1.74 / 0.80 ms 46%
ffn_down INT4_G128_ZP1.15 / 0.40 ms 35%
attn_kv + attn_qo INT4_G128_ZP0.62 / 0.16 ms 26%
attn_qo INT4_G128_ZP0.56 / 0.12 ms 21%
attention0.40 / 0.02 ms 5%
lm_head F160.37 / 0.33 ms 91%
kernel gaps, at most0.55 ms · no floor
triton_red_fused_fused_add_rms_norm_mac…0.24 ms · no floor
cutlass::device_kernel0.12 ms · no floor
vllm::reshape_and_cache_flash_kernel0.08 ms · no floor
triton_red_fused_30.06 ms · no floor
21 other small kernels0.10 ms · no floor
Per kernel (26): ms now / ms at the roofline, launches a token
cutlass::device_kernel4.59 / 1.49 ms ×240
nvjet_sm90_hsh_384x8_64x4_2x1_v_bz_TNT0.37 / 0.33 ms ×1
triton_red_fused_fused_add_rms_norm_mac…0.24 ms · no floor ×79
vllm::reshape_and_cache_flash_kernel0.08 ms · no floor ×40
triton_red_fused_30.06 ms · no floor ×39
triton_poi_fused_machete_mm_mul_silu_sl…0.06 ms · no floor ×40
_gumbel_sample_kernel0.01 ms · no floor ×1
at::native::index_elementwise_kernel0.01 ms · no floor ×3
_bias_kernel0.01 ms · no floor ×1
at::native::unrolled_elementwise_kernel0.00 ms · no floor ×2
at::native::reduce_kernel0.00 ms · no floor ×1
triton_red_fused__to_copy_embedding_mac…0.00 ms · no floor ×1
_post_update_kernel0.00 ms · no floor ×1
_gather_block_tables_kernel0.00 ms · no floor ×1
triton_red_fused_fused_add_rms_norm_20.00 ms · no floor ×1
triton_red_fused_10.00 ms · no floor ×1
_combine_sampled_and_draft_tokens_kernel0.00 ms · no floor ×1
_compute_slot_mappings_kernel0.00 ms · no floor ×1
_get_num_sampled_and_rejected_kernel0.00 ms · no floor ×1
at::native::vectorized_elementwise_kern…0.00 ms · no floor ×3
flash::prepare_varlen_num_blocks_kernel0.00 ms · no floor ×1
at::native::_scatter_gather_elementwise…0.00 ms · no floor ×1
_prepare_pos_seq_lens_kernel0.00 ms · no floor ×1
<unnamed>::elementwise_kernel_with_index0.00 ms · no floor ×1
_apply_write_kernel0.00 ms · no floor ×0
concurrent kernel overlap (subtract)-0.03 / -0.03 ms ×0

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run h200-qwen3-14b-awq-vllm-k8

Tested on

GPU
H200 · 140 GB
System RAM
1511 GB
CPU threads
256
Memory, measured
4662 GB/s
Temp, max
see notes
Power
see notes
SM clock
–
Driver
595.91.07
Measured
2026-11-09 22:57 UTC · context 512 tokens · measured, in-model (nsys): split by role from the engine's own labels, 5 weight roles

Qwen3-14B on other GPUs