What a megakernel changes
The 135M route combines decoder work into a cooperative launch instead of dispatching the full layer stack through many kernels. Prefill and unsupported batch compositions use the regular path. vLLM still owns logits processing and sampling. The 360M work adds its own dimensions, weight layout, numerical gates and serving integration. A kernel specialized for SmolLM2 does not become a Qwen kernel merely because Qwen is also small. Qwen3-0.6B stays on stock vLLM in these presets.Follow the dispatch boundary
The serving path must distinguish eligible decode from prefill, mixed batches and unsupported shapes. Unsupported work falls back to native execution. Weight refresh must invalidate or refresh cached packed representations, including any graph-captured references. The acceptance work checks final hidden states and every layer’s KV values, not only the top token. Matching a greedy token can hide errors in close logits. Conversely, one BF16 ULP can fail an exact gate even if the generated token stays the same.The experiments that shaped the route
Lower shared memory does not automatically increase occupancy. Registers can remain the limiting resource. A lower-level PTX instruction also does not guarantee a faster composed decoder.