Nemotron 3.5 Bolt 74B-A9B (24GB)

NVIDIA · Released: 2026-10-15 · Last verified 2026-10-16

Vision-language MoE compressed from Nemotron 3.5 Super VL (120B-A12B) to 74B with 9B active, shipped with experts at native 2-bit so it fits in about 24 GB. 256K context. Built for local agents in llama.cpp.

Recommended quants

Official 24GB GGUF (22.0 GB, experts IQ2_XXS)
nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF
Add mmproj file for images. NVIDIA's own recipes target 28 GB / 32 GB+ GPUs, so 24 GB cards are tight once context is added.
LiLiCorr speculative draft
nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr-GGUF
Used as a spec-decoding draft in NVIDIA's recipe.
Unsloth UD-Q2_K_XL 28.8 / UD-Q4_K_XL 50.2 / Q8_0 78.9 GB
unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF
Higher-precision options from the BF16 weights.

Chat template Works

Reasoning toggled via chat template (enable_thinking=True/False), inherited from Super 3.5 VL.

Fix: Pass enable_thinking=false for direct answers.

Sampling settings

Text (official)	`temp 1.0, top_p 0.95`

Runtimes

vLLM	Unverified
llama.cpp	Works	Official target runtime; support added in llama.cpp PR #30520 (Oct 15). Update your build.
Ollama	Unverified	Library page nemotron-3.5-bolt exists.
LM Studio	Unverified
MLX	Unverified

Tool calling in agent tools

Claude Code (local endpoint)	Unverified
OpenCode	Unverified
Cline	Unverified
Aider	Unverified

Known issues

Sources checked

Spotted something wrong or a new fix? Corrections are reviewed before publishing. (Submission form coming soon.)