Anthropic Open-Sources FlashProve for Provable MoE Inference

(SAN FRANCISCO —) Anthropic’s Frontier Red Team, alongside academic and industry collaborators from EPFL, MIT, and StarkWare, has introduced and open-sourced FlashProve, a novel protocol and systems prototype designed to solve one of the most persistent verification challenges in cloud-hosted artificial intelligence: confirming that a specific model actually generated a given response. Released under an Apache 2.0 license, FlashProve addresses the opaque nature of API inference endpoints where enterprise buyers and independent evaluators are forced to trust that providers are serving requested frontier weights rather than substituting cheaper, smaller substitutes.

Operating at the intersection of applied cryptography and high-performance machine learning systems, FlashProve generates mathematically rigorous proofs of correct execution. By utilizing a hybrid architecture featuring an optimized C++ runtime for day-to-day operations and a Lean-verified reference implementation to guarantee protocol soundness, completeness, and zero-knowledge properties, the project sets a new baseline for verifiable AI inference. Tested against the 400 billion parameter Llama 4 Maverick Mixture-of-Experts (MoE) architecture, FlashProve demonstrates that cryptographic accountability for large language models is no longer strictly theoretical.

Technical Architecture & Engineering Breakdown

Commercial API infrastructure operates on implicit trust. When enterprises route sensitive workloads to a commercial endpoint, they possess no direct mechanism to verify whether the inference engine executed a top-tier frontier model or a heavily quantized, smaller substitute. Similarly, independent safety evaluators testing model vulnerabilities lack the cryptographic tools to prove that an API provider hasn't hot-swapped weights behind the proxy. FlashProve resolves this trust deficit by producing verifiable cryptographic proof linked directly to the weight state and execution path of the model.

The engineering stack within the open-source repository comprises three distinct runtime executables: an optimized prover, a high-speed C++ verifier for production pipelines, and a formal verifier written in the Lean proof assistant. Anthropic utilized Lean to formally state and verify the mathematical soundness, completeness, and zero-knowledge properties of the protocol. While the Lean verifier operates too slowly for real-time validation, it acts as a rigid reference standard, ensuring that the C++ verifier strictly adheres to the proven theorem boundaries. An interactive visualizer is packaged within the codebase to help auditors navigate discrepancies between the Lean protocol definitions and the runtime C++ implementation.

Mixture-of-Experts (MoE) Optimization Strategies

Applying zero-knowledge and succinct proof architectures to massive language models typically encounters insurmountable memory and compute walls. FlashProve circumvents this bottleneck by tailoring its proving mechanism specifically to Mixture-of-Experts (MoE) architectures, such as the 400-billion-parameter Llama 4 Maverick. Because MoE models route tokens through a sparse subset of expert networks per layer—activating roughly 17 billion parameters out of 400 billion per token—the prover selectively touches active weight matrices rather than serializing the entire global tensor state.

During the offline phase, which takes approximately 39 minutes on a single H100 GPU, the prover commits to the model weights once. This single commitment covers approximately 16 million subsequent inference requests, yielding a compact 6.5 MB commitment file that clients can store. During online inference, the prover streams weights through in layer-sized segments, requiring just a single 80GB VRAM GPU paired with roughly 290GB of host system memory to handle the streaming throughput cleanly.

Benchmark Evaluation & Rigorous Comparison

Performance overhead is the historical Achilles' heel of cryptographic proof systems in machine learning. FlashProve achieves unprecedented throughput efficiency for a zero-knowledge protocol applied to frontier-class models, moving approximately 110 tokens per second on a single H100 accelerator paired with 24 CPU cores.

For a standard 1,024-token response, the generated proof weighs just 18.78 megabytes. Verification of this proof completes in roughly 17 seconds utilizing 24 CPU threads, or approximately 21 seconds when constrained to a single thread. When benchmarked against unverified baseline inference running via vLLM with FP8 quantization across eight H100 nodes, FlashProve’s provable inference incurred an overhead factor of roughly 72x per GPU across standard runs, peaking at approximately 94x under heavy concurrent load.

Deployment, VRAM Footprint & Inference Optimization

Deploying FlashProve requires careful management of host-to-device memory pipelines. Because the prover does not need to store the entire 400B+ parameter matrix of Llama 4 Maverick concurrently within GPU VRAM, it leverages layer-by-layer weight streaming. An 80GB VRAM accelerator acts as the primary compute engine while approximately 290GB of host system memory holds the staged weight segments before they are piped into the execution kernel.

The offline commitment step—requiring 39 minutes of compute per model variant—must be executed prior to serving provable traffic. However, because this commitment remains valid across roughly 16 million sequential user interactions, the amortized infrastructure penalty for enterprise operators is negligible. Clients interacting with the provable endpoint simply download a lightweight 6.5 MB weight commitment file once, followed by a minor ~18MB proof payload accompanying each response token stream.

# Clone the Anthropic FlashProve repository
git clone https://github.com/anthropic/flashprove.git
cd flashprove

# Initialize submodules and build C++ verifier and prover binaries
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j24

# Download Llama 4 Maverick MoE weights from Hugging Face under Meta license
python3 scripts/fetch_weights.py --model-id "meta-llama/Llama-4-Maverick-MoE"

# Execute offline weight commitment phase (~39 mins on H100)
./prover --commit-weights --weights-path ./weights/llama4_maverick --out commitment.dat

# Run provable inference test with 1024-token prompt context
./prover --infer --commitment commitment.dat --prompt "Analyze cryptographic verification." --proof-out proof.bin

# Verify cryptographic proof using the high-speed C++ verifier
./verifier --proof proof.bin --commitment commitment.dat --prompt "Analyze cryptographic verification."