After an optimizer update, the trainer and vLLM hold different versions of the policy. Refreshing inference is a correctness boundary before it is a speed optimization.

Filesystem transport

The sender writes an inference-compatible snapshot. For LoRA, that is a PEFT-shaped adapter and its configuration. For full parameters, the filesystem transport can stream smaller shards instead of gathering one large CPU copy. The receiver waits for the finished marker and loads the completed snapshot. This limits staging memory, but filesystem publication still has handshake, serialization and loading costs. A measured publish timer is not necessarily a pure disk-write timer; it can include outstanding GPU completion and coordination.

CUDA IPC transport

The small full-parameter Llama path can share CUDA storage between local processes. Its sender streams through a fixed 64 MB slab and waits for acknowledgement before reusing it. The receiver snapshots tensors that vLLM may defer while assembling fused projections. Otherwise a reused slab could overwrite a tensor before its consumer loads it. The documented support is deliberately narrow: one shared GPU, a compatible HF-layout Llama, an FP32 trainer and BF16 inference, matching allocator settings, and no LoRA. Multi-GPU and arbitrary quantized serving are outside that contract.

What was proved

The archived production comparison matched all 182 unique inference parameter hashes and deterministic token IDs/logprobs against filesystem refresh. A guarded 135M RL smoke completed ten updates and applied policy v10, with all 11 update requests returning HTTP 200. That establishes a working transfer path for the measured configuration. It does not establish a whole-RL speedup. Removing checkpoint payload writes cannot remove generation, trainer work or startup time.

Checkpoint versus broadcast

A broadcast refreshes the live policy. A DCP training checkpoint carries model, optimizer, scheduler and progress state. An HF export is a portable full model or adapter. These have different purposes and sizes. The installed uploader preserves final inference/new-training weights; it does not promise exact optimizer-state resume after local checkpoint cleanup. Filesystem sender/receiver · CUDA IPC implementation Continue with the kernel path.