NVIDIA Deploys DeepSeek-V4.1-Flash NVFP4: 552B MoE Quantized

(SAN FRANCISCO —) NVIDIA has officially released the DeepSeek-V4.1-Flash-1017-NVFP4 checkpoint, marking a major milestone in sub-byte neural network quantization and high-performance inference engineering. Built upon the Causal Encoder-Decoder Mixture-of-Experts (MoE) architecture featuring Compressed Sparse Attention 2, the model scales across a massive 552 billion total parameter footprint while dynamically routing only 8 billion parameters during prefill and 16 billion during decode phases.

Designed for rigorous production environments including AI agents, tool-calling pipelines, multimodal RAG, and terminal-based reasoning workflows, the checkpoint delivers high-throughput execution on NVIDIA Blackwell architecture. By leveraging Model Optimizer v0.48.0rc0, NVIDIA has compressed routed expert weights and activations into NVFP4 W4A4 format with a group size of 16, while mapping memory lookup structures to MXFP4 Engram tables. This technical dispatch provides a comprehensive systems-level breakdown of the architecture, quantization topology, hardware prerequisites, and empirical benchmark evaluations.

Technical Architecture & Engineering Breakdown

The DeepSeek-V4.1-Flash-1017-NVFP4 model utilizes the DeepseekV41ForCausalLM transformer backbone, integrating a Causal Encoder-Decoder MoE structure optimized with Compressed Sparse Attention 2. Beyond its dynamic routing mechanism—which activates 8B parameters during prefill and 16B during generation—the architecture incorporates a massive 196B Engram conditional memory component that acts as a structured retrieval and context expansion engine.

Quantization of this scale requires careful handling of precision boundaries across tensor operations. Using NVIDIA Model Optimizer v0.48.0rc0, the engineering team applied NVFP4 W4A4 quantization with a group size of 16 specifically to the routed expert weights and activation tensors. Meanwhile, the 196B Engram lookup tables were mapped to MXFP4 precision. Other network components retain their source precision—including MXFP8 where applicable—ensuring numerical stability throughout multi-modal inference.

Conversion of expert weights preserved dequantized numerical representations, whereas Engram table conversion involved lossy quantization. This architectural choice achieves a compact checkpoint size of approximately 400 GiB distributed across 48 safetensors shards, making multi-GPU cluster placement significantly more efficient.

Deployment, VRAM Footprint & Inference Optimization

Deploying the DeepSeek-V4.1-Flash NVFP4 checkpoint involves strict hardware and software prerequisites. The model requires NVIDIA Blackwell infrastructure (validated on four NVIDIA GB300 GPUs running Linux) and cannot be loaded using stock vLLM container images. Inference execution mandates a custom vLLM build compiled with MXFP4 Engram support, as standard configurations fail to resolve Engram lookup tables. SGLang has not yet been validated for this specific checkpoint.

Calibration for the quantization process utilized 1,024 samples drawn from CNN/DailyMail and selected Nemotron-Post-Training-Dataset-v2 subsets. The model natively supports multimodal inputs (text and RGB images) and features an extended context window capable of handling up to 1 million tokens, supporting advanced tool-calling flags within vLLM deployment pipelines.

Benchmark Evaluation & Rigorous Comparison

Rigorous empirical evaluation confirms that sub-byte NVFP4 quantization maintains competitive performance parity against the higher-precision MXFP4 source baseline without introducing catastrophic degradation. Across a battery of scientific reasoning, coding, long-context recall, and agentic benchmarks, the NVFP4 checkpoint matches or exceeds baseline metrics.