NVIDIA Nemotron 3.5 Lightning
serving recipe.

Floors are the minimum the vLLM recipe publishes across the whole replica, from vLLM's own runs. The control plane compares them with its own sizing from the model config.

• Tensor ParallelFits inside one node's interconnect. single_node_tp
• Tensor + Expert ParallelFits inside one node's interconnect. single_node_tep

• DGX Spark (GB10) · 1 GPU
• DGX Station (GB300) · 1 GPU
• H100 · 8 GPUs
• RTX Pro 6000 · 1 GPU

Wingsmith runs models through NVIDIA Dynamo, so it serves on NVIDIA GPUs. Other vendors are listed as the recipe states them.

vllm serve nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 \
  --mamba-backend flashinfer \
  --mamba-cache-mode align \
  --enable-prefix-caching \
  --max-num-batched-tokens 16384 \
  --kv-cache-dtype fp8 \
  --moe-backend marlin \
  --reasoning-parser nemotron_v3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --mamba-ssm-cache-dtype float16 \
  --enable-mamba-cache-stochastic-rounding \
  --mamba-cache-philox-rounds 5

Features

• ReasoningOn by defaultNemotron v3 reasoning parser — per-request reasoning toggle with a configurable token budget
• Tool callingOn by defaultQwen3 XML tool-call parser with automatic tool choice
• Fast SSM cacheOn by defaultFP16 Mamba SSM cache with stochastic rounding — faster decode than the FP32 default; the Philox rounds control the rounding stream
• Speculative decodingOpt-inSpeculative decoding — built-in MTP heads, the DSpark hybrid speculator, or the DFlash diffusion drafter Modes: DSpark, MTP, DFlash.

1. Register the weights

   In the console, or through the API. Resolving a slug proposes an artifact and writes nothing.

   curl "http://localhost:8000/v1/artifacts/resolve?source=hf:nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16"

2. Read the draft

   The draft carries this recipe as upstream evidence: the tested flags, the precision and memory floor, the parallelism layouts, and the hardware vLLM verified, with links back to the recipe. Nothing from it is applied without you, and the draft warns when the plane's own sizing and the published floor differ by a quarter or more.

3. Create the workload

   Choose the engine, device shape and scale. The plane picks the clusters, and each cluster's operator renders the deployment as a Dynamo graph. The one recipe default it applies, --max-num-seqs, is shown above.

4. Give it an endpoint

   One model name for clients, routed by the gateway to deployments on every cluster serving it.

Run the control plane locally →

• How much GPU memory does NVIDIA Nemotron 3.5 Lightning need?The vLLM recipe publishes a minimum of 18 GB across the whole replica for its NVFP4 variant; BF16 72 GB. That is vLLM's own measurement, not a per-GPU figure.
• Which parallelism layouts does the NVIDIA Nemotron 3.5 Lightning recipe support?Tensor Parallel and Tensor + Expert Parallel.
• Can Wingsmith serve NVIDIA Nemotron 3.5 Lightning?Wingsmith serves models through NVIDIA Dynamo on clusters with NVIDIA GPUs. Register nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 and the plane attaches this vLLM recipe to the draft as evidence: its tested flags, precision and memory floor, for you to review rather than applied on your behalf.
• Does NVIDIA publish a Dynamo recipe for NVIDIA Nemotron 3.5 Lightning?Yes: 33 deployment entries for vLLM and TensorRT-LLM, aggregated and disaggregated. When a workload is placed, the plane defaults --max-num-seqs to the value the aggregated recipe publishes for that card, and names it on the draft.

From the vLLM recipe for nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 ↗ (snapshot at 4bcc189) and NVIDIA Dynamo's recipes at 165d854, Apache 2.0.