boltbeam.fyi

Qwen3.8-27B W4A16-AWQ on H200

safetensors INT4_G128_ZP+BF16

Engines

vLLM

0.31.0 ↗measured 2026-11-09 23:02 UTC

100.8 tok/s

34% of the roofline299.1 tok/s possible

now, a token
9.92 ms
floor
3.34 ms
read, a token
15.59 GB

Speed at context

128 tokens
vLLM100.9 tok/s
512 tokens
vLLM100.8 tok/s of 299.1
The pink tick is the roofline at that context.

Where the time goes

Now, a token
9.92 ms
Roofline floor, a token
3.34 ms
Gap to the floor
6.57 ms
Kernels a token
840

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up INT4_G128_ZP2.94 / 1.27 ms 43%
ffn_down INT4_G128_ZP1.84 / 0.64 ms 35%
attn_gate + attn_qkv INT4_G128_ZP1.12 / 0.45 ms 40%
ssm_out INT4_G128_ZP0.73 / 0.17 ms 23%
recurrent0.38 / 0.07 ms 17%
attn_kv + attn_qo INT4_G128_ZP0.37 / 0.13 ms 35%
ssm_alpha_beta BF160.23 / 0.01 ms 4%
attn_qo INT4_G128_ZP0.25 / 0.06 ms 23%
attention0.14 / 0.01 ms 6%
lm_head BF160.62 / 0.55 ms 88%
kernel gaps, at most0.43 ms · no floor
triton_red_fused__to_copy_add_fused_add…0.38 ms · no floor
triton_poi_fused_machete_mm_mul_silu_sl…0.10 ms · no floor
layer_norm_fwd_kernel0.09 ms · no floor
cublasLt::splitKreduce_kernel0.07 ms · no floor
26 other small kernels0.24 ms · no floor
Per kernel (33): ms now / ms at the roofline, launches a token
cutlass::device_kernel7.42 / 2.72 ms ×288
nvjet_sm90_tst_512x8_64x3_2x1_v_bz_TNT0.62 / 0.55 ms ×1
triton_red_fused__to_copy_add_fused_add…0.38 ms · no floor ×128
_causal_conv1d_update_kernel + fused_recurrent_gated_delta_rule_packed_decode_kernel0.38 / 0.07 ms ×96
nvjet_sm90_tst_64x8_64x16_2x1_v_bz_splitK_TNT0.23 / 0.01 ms ×48
triton_poi_fused_machete_mm_mul_silu_sl…0.10 ms · no floor ×64
layer_norm_fwd_kernel0.09 ms · no floor ×48
cublasLt::splitKreduce_kernel0.07 ms · no floor ×48
_fused_qk_rmsnorm_rope_gate_kernel0.04 ms · no floor ×16
vllm::reshape_and_cache_flash_kernel0.03 ms · no floor ×16
triton_poi_fused_zeros_30.02 ms · no floor ×32
_gumbel_sample_kernel0.02 ms · no floor ×1
triton_poi_fused_machete_mm_mul_sigmoid…0.02 ms · no floor ×16
triton_poi_fused_zeros_40.01 ms · no floor ×15
at::native::index_elementwise_kernel0.01 ms · no floor ×3
_bias_kernel0.01 ms · no floor ×1
at::native::unrolled_elementwise_kernel0.00 ms · no floor ×2
triton_red_fused__to_copy_add_embedding…0.00 ms · no floor ×1
at::native::reduce_kernel0.00 ms · no floor ×1
_gather_block_tables_kernel0.00 ms · no floor ×1
_post_update_kernel0.00 ms · no floor ×1
_prepare_rope_positions_kernel0.00 ms · no floor ×1
_combine_sampled_and_draft_tokens_kernel0.00 ms · no floor ×1
_get_num_sampled_and_rejected_kernel0.00 ms · no floor ×1
_compute_slot_mappings_kernel0.00 ms · no floor ×1
at::native::vectorized_elementwise_kern…0.00 ms · no floor ×3
flash::prepare_varlen_num_blocks_kernel0.00 ms · no floor ×1
at::native::_scatter_gather_elementwise…0.00 ms · no floor ×1
_prepare_pos_seq_lens_kernel0.00 ms · no floor ×1
_scatter_num_accepted_kernel0.00 ms · no floor ×1

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run h200-qwen3.8-27b-awq-vllm-k8

Tested on

GPU
H200 · 140 GB
System RAM
1511 GB
CPU threads
256
Memory, measured
4662 GB/s
Temp, max
see notes
Power
see notes
SM clock
–
Driver
595.91.07
Measured
2026-11-09 23:02 UTC · context 512 tokens · measured, in-model (nsys): split by role from the engine's own labels, 9 weight roles

Qwen3.8-27B on other GPUs