Kimi-K3
serving recipe.

Floors are the minimum the vLLM recipe publishes across the whole replica, from vLLM's own runs. The control plane compares them with its own sizing from the model config.

• Tensor Parallel (default)Fits inside one node's interconnect. single_node_tp
• Multi-Node Tensor ParallelSpans nodes. multi_node_tp
• Multi-Node Tensor + Expert ParallelSpans nodes. multi_node_tep
• Multi-Node TP + Pipeline ParallelSpans nodes. multi_node_tp_pp
• Multi-Node Data + Expert ParallelSpans nodes. multi_node_dep
• Prefill/Decode DisaggregationSpans nodes. pd_cluster
• Multi-Node TP + Data ParallelSpans nodes. multi_node_tp_dp

• B200 · 8 GPUs
• B300 · 8 GPUs
• GB200 NVL4 · 4 GPUs
• GB300 NVL4 · 4 GPUs
• H200 · 8 GPUs
• MI355X · 8 GPUs
• Ascend 910C · 16 GPUs

Wingsmith runs models through NVIDIA Dynamo, so it serves on NVIDIA GPUs. Other vendors are listed as the recipe states them.

vllm serve moonshotai/Kimi-K3 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3 \
  --reasoning-parser kimi_k3

Features

• Tool callingOn by defaultEnable automatic tool choice with the Kimi K3 tool-call parser.
• ReasoningOn by defaultParse Kimi K3 reasoning output separately from the final answer.
• Speculative decodingOpt-inUse DSpark speculative decoding.
• Text-only modeOpt-inSkip the vision encoder for text-only workloads. Mutually exclusive with encoder_parallel.
• Decode context parallelismOpt-inDecode context parallelism (DCP): shard the decode KV cache across the tensor-parallel ranks for decode-heavy long-context serving. The DCP size must divide the tensor-parallel size, so the args below assume the TP8 single-node layout. Pairs the TOKENSPEED_MLA decode backend with TRTLLM_RAGGED MLA prefill under FP8 KV.

1. Register the weights

   In the console, or through the API. Resolving a slug proposes an artifact and writes nothing.

   curl "http://localhost:8000/v1/artifacts/resolve?source=hf:moonshotai/Kimi-K3"

2. Read the draft

   The draft carries this recipe as upstream evidence: the tested flags, the precision and memory floor, the parallelism layouts, and the hardware vLLM verified, with links back to the recipe. Nothing from it is applied without you, and the draft warns when the plane's own sizing and the published floor differ by a quarter or more.

3. Create the workload

   Choose the engine, device shape and scale. The plane picks the clusters, and each cluster's operator renders the deployment as a Dynamo graph. The one recipe default it applies, --max-num-seqs, is shown above.

4. Give it an endpoint

   One model name for clients, routed by the gateway to deployments on every cluster serving it.

Run the control plane locally →

• How much GPU memory does Kimi-K3 need?The vLLM recipe publishes a minimum of 1650 GB across the whole replica for its NVFP4 variant; MXFP4 1680 GB, W4A8 (Ascend) 1711 GB. That is vLLM's own measurement, not a per-GPU figure.
• Which parallelism layouts does the Kimi-K3 recipe support?Tensor Parallel, Multi-Node Tensor Parallel, Multi-Node Tensor + Expert Parallel, Multi-Node TP + Pipeline Parallel, Multi-Node Data + Expert Parallel, Prefill/Decode Disaggregation and Multi-Node TP + Data Parallel. The recipe's default is Tensor Parallel.
• Can Wingsmith serve Kimi-K3?Wingsmith serves models through NVIDIA Dynamo on clusters with NVIDIA GPUs. Register moonshotai/Kimi-K3 and the plane attaches this vLLM recipe to the draft as evidence: its tested flags, precision and memory floor, for you to review rather than applied on your behalf.
• Does NVIDIA publish a Dynamo recipe for Kimi-K3?Yes: 6 deployment entries for vLLM, aggregated and disaggregated. When a workload is placed, the plane defaults --max-num-seqs to the value the aggregated recipe publishes for that card, and names it on the draft.

From the vLLM recipe for moonshotai/Kimi-K3 ↗ (snapshot at 4bcc189) and NVIDIA Dynamo's recipes at 165d854, Apache 2.0.