Mistral Large 4 goes open-weight: what it means for teams that want to self-host

In late October 2026, Mistral released the weights of Mistral Large 4 ("Le Chonk"), the model it first opened as a public preview on 6 October (announcement). It is a French company's flagship model, and you can now run it on your own infrastructure. Here is what an engineering team should check before getting excited.

The numbers, from the model card

According to the Hugging Face model card:

• A Mixture-of-Experts model with about 1 trillion total parameters and 49B active per token (52B including embeddings and output layers). The docs give 1.05T total.
• 128 routed experts, 4 active per token, plus 1 shared expert.
• A 1M-token context window; text and image in, text out.
• Function calling, JSON output and strict JSON schema enforcement, with three reasoning efforts (low, medium, high).
• Trained on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters in Europe.

Mistral positions it around coding and defensive cybersecurity, and says it refuses malicious requests.

The license: read the Model-as-a-Service clause

The weights ship under the Mistral Ecosystem Open-Weight License, a modified MIT license (license text). Use, modification, fine-tuning and commercial use are allowed, with one condition: if you run a Model-as-a-Service business, meaning you give third parties access to the model for inference or post-training with meaningful control over inputs, parameters or training data, you need a separate commercial license from Mistral.

The license explicitly excludes internal use, end-user products where the model is embedded in specific features, and simply relaying requests to models hosted by others. For most companies building an internal tool or a product feature, that is good news. If you plan to resell raw API access, talk to Mistral first.

Self-host or call the API?

Mistral's own API lists $0.70 per million input tokens, $0.07 cached and $2.10 output (source). Self-hosting is supported with vLLM or SGLang, and there is an NVFP4 checkpoint aimed at Blackwell GPUs.

My rule of thumb as a DevOps engineer:

1. Start on the API to validate quality on your real tasks.
2. Self-host when a requirement forces it: data that cannot leave your network, a security team that wants full control, or volumes where GPU cost beats per-token pricing.
3. Budget for the hardware honestly. Even with only ~49B parameters active per token, all ~1T must be loaded, so this is multi-GPU territory, not a laptop model. Benchmark on your own workload before committing.
4. Treat it like any production service: version pinning, eval sets, monitoring, and a rollback path.

Self-hosting: VRAM, nodes, inference servers, monitoring and cost

Figures below come from the Mistral open-weight announcement, the Hugging Face model card and the Mistral docs. Those pages do not publish a fixed “needs X GB VRAM” table. What follows is the weight floor from the verified parameter count; KV cache and runtime overhead sit on top.

Where the memory goes

This is a Mixture-of-Experts model: about 1.05 trillion total parameters, with about 52B active per token (the model card states 49B active, 52B including embeddings and output layers). Sparse activation does not mean sparse residency — every expert’s weights must stay loaded, or a routed expert is simply missing. The docs also list a separate 1.6B vision encoder.

Weight footprint (parameters × bytes per parameter only; no KV, activations or fragmentation):

Formula: weight_bytes ≈ num_params × bytes_per_param. The model card ships an NVFP4 checkpoint aimed at Blackwell GPUs, with a “smaller memory footprint, faster inference and same performance”, but no absolute GB figure. The NVFP4 row is a 4-bit estimate; packing metadata adds more — measure on your box.

You still need headroom for the KV cache and concurrency. Cache grows with context length and in-flight requests; the model card says to lower --max-model-len and/or --max-num-seqs when memory is tight. A full 1M-token window at high concurrency is a large add-on on top of the weight floor; production setups usually cap context first, then raise concurrency.

What class of GPU / node

A single consumer GPU will not hold this. The model card’s serving examples use --tensor-parallel-size 8 (SGLang: --tp-size 8), i.e. at least one 8-GPU node (or an equivalent multi-GPU topology) to shard the weights. The NVFP4 checkpoint targets Blackwell; on non-Blackwell hardware use the non-NVFP4 path from the model card and validate the weight floor on your own cards.

Ask three sizing questions: do eight (or more) GPUs’ VRAM sum clearly above the weight floor; is the interconnect good enough for tensor parallel; can host RAM and disk pull the checkpoint in one go. Cloud list prices and per-SKU tokens/s are not in the sources above — benchmark on your workload.

Which inference server

Only what the model card / announcement state:

• vLLM (recommended): official image vllm/vllm-openai:mistral-large-4; example flags include the Mistral tool and reasoning parsers, --gpu-memory-utilization 0.85, and optional DSpark speculative decoding (model card reports up to about 1.9× throughput vs the target alone; hardware- and workload-dependent).
• SGLang: image lmsysorg/sglang:mistral-large-4; example --tp-size 8, optional --speculative-algorithm DSPARK.
• Also listed: transformers (multi-GPU torchrun), llama.cpp; the announcement also mentions an NVIDIA NIM container (serving with vLLM) and TensorRT LLM for high-throughput serving.

Practical path: stand up the model card’s vLLM recipe first, measure TTFT and tokens/s on real prompts, then decide on DSpark and NVFP4.

Monitoring and rate limiting

Treat it like any production service:

• Capacity: per-GPU utilization, used VRAM, OOM events, and (if you can split them) weight vs KV share.
• Latency / throughput: TTFT, tokens/s per request, queue depth, reject/queue rate under load.
• Correctness / change control: pin checkpoint revision and image digest; audit changes to sampling (temperature, top_p, reasoning_effort).
• Rate limiting: at the gateway — token bucket or QPS per API key, with a concurrency cap aligned to --max-num-seqs (model card examples are on the order of low tens; SGLang’s example pins 64). Return 429 before the GPUs swap or OOM. Client-side retry policy is out of scope here.

Cost reasoning

Mistral API pricing from the open-weight announcement: $0.70 / M input, $0.07 / M cached input, $2.10 / M output.

There is no official self-host $/token. Compare with:

monthly_self_host ≈ node_rent (or GPU_hours × unit_price) + storage + egress + ops time
monthly_api ≈ (input_tokens × 0.70 + cached_input × 0.07 + output_tokens × 2.10) / 1e6

Measure a week of real tokens on your task mix, then plug in. Self-host when data cannot leave the network, compliance needs full control, or API spend stably exceeds fully loaded self-host cost. License note: a Model-as-a-Service offering to third parties needs a separate commercial license from Mistral; internal use is exempt (LICENSE).

Sources

• https://mistral.ai/news/mistral-large-4-oss/
• https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B
• https://docs.mistral.ai/models/mistral-large-4
• https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B/raw/main/LICENSE

Wiring it into a Python backend

Whether you self-host or call Mistral's API, keep the application code identical. vLLM exposes an OpenAI-compatible /v1/chat/completions endpoint, so a single client with a configurable base_url can switch between your own cluster and the hosted API with an environment variable, not a code change.

Treat the model as a slow, fallible dependency.

• Set explicit connect and read timeouts. A long completion can take tens of seconds; an unbounded request ties up a worker.
• Retry only what is safe to retry: timeouts, 429 and 5xx, with exponential backoff and jitter, and honour Retry-After when it is present. Never retry a 400; fix the request.
• Put a concurrency limit in your own code (a semaphore or a task queue such as Celery or RQ) that matches what the gateway allows, so a traffic spike queues instead of failing.

Never trust the output blindly.

• When you need structured data, ask for JSON and validate it with Pydantic. If validation fails, retry once with the error message, then fall back or flag the item for a human.
• Keep facts out of the model's hands where you can: fetch them from your database and let the model format or summarise them, not invent them.

Make it observable and testable.

• Log the model name, latency, token counts and a request ID for every call; that is what you will need when costs or quality drift.
• In tests, mock the client and assert on your validation and fallback paths. A model upgrade should be a config change you can roll back.

Why it matters in Europe

An open-weight frontier-class model trained in Europe gives European companies another option for keeping sensitive workloads under their own control. Whether it is the right choice depends on your workload, your team and your budget, and that is exactly the kind of evaluation worth doing carefully.

Need help evaluating or deploying an open-weight model on your infrastructure? Get in touch.