The Best Open-Source LLMs You Can Actually Self-Host (2026)

The self-hosting landscape flipped in the first half of 2026, and most of the advice still circulating online hasn't caught up. Google released Gemma 4 under Apache 2.0 — dropping the custom Gemma Terms (and their remote-restriction clause) that shackled every previous generation. DeepSeek shipped V4 under plain MIT, weights included. Qwen kept its open line on Apache 2.0 — but its newest flagship, Qwen3.7, is API-only, so Qwen3.6 is now the self-hosting ceiling. And Meta has shipped nothing new with open weights since Llama 4 in April 2025, which itself sits behind a gated download and a community license with real strings attached.

The licensing story this year is one of divergence: permissive licenses got more so, restrictive ones stayed put, and the gap between "open weights" and "open source" widened. Qwen3.6-27B or Gemma 4 on consumer hardware; DeepSeek V4-Flash if you have datacenter-class memory. All three are permissively licensed. The rest of this guide is why, with the license fine print that actually matters. (And if you'd rather not run your own, see our hosted-assistant comparison.) The other half of that decision is arithmetic: our LLM Price Comparison shows what the hosted equivalents charge per request, which is the number self-hosting has to beat.

Downloads don't lie

First, a look at what people are actually pulling from Hugging Face. The main caveat is that 30-day download counts strongly favor older, established checkpoints baked into thousands of pipelines and tutorials. Qwen2.5-7B-Instruct dwarfing everything doesn't mean it's the best model you can run in July 2026 — it means it's the default that half the ecosystem standardised on. The newer Qwen3.5/3.6 and DeepSeek V4 checkpoints are where the capability is; the chart is a measure of installed base, not quality.

The one new name in that list is DeepSeek V4-Pro, a model released in late April that's already out-pulling its own predecessor. That is the 2026 story in miniature.

Qwen — the Apache 2.0 ceiling is 3.6

Alibaba's open line moved fast this year: Qwen3.5 shipped between February and March 2026 in sizes from 0.8B up to a 397B-A17B mixture-of-experts, followed by Qwen3.6 (a 35B-A3B sparse MoE and a dense 27B) in April. The official repo is unambiguous: "All our open-weight models are licensed under Apache 2.0." Context runs to 262,144 tokens, and the family's strengths per its own documentation are multilingual coverage (201 languages and dialects claimed), agentic workloads, and vision/OCR on the multimodal variants.

The caveat is that the open line now trails the closed one. Qwen3.5-Plus, Qwen3.5-Omni and the May/June flagship Qwen3.7 are proprietary, API-only, with no published weights. If you're self-hosting, Qwen3.6 is the ceiling, and that's worth knowing before you benchmark against Alibaba's marketing numbers.

Llama — still open on paper, gated in practice

Meta's latest open-weight generation remains Llama 4 (April 2025): Scout (17B active, 109B total, natively multimodal, a claimed 10M-token context) and Maverick (17B active, ~400B total). Nothing newer has shipped open in 2026, and the previewed Behemoth never publicly released. Per the Scout model card, Meta claims it "can fit within a single H100 GPU with on-the-fly int4 quantization" — an impressive piece of engineering.

The license is where readers get burned. The Llama 4 Community License is not Apache and not MIT: products over 700 million monthly active users need a separate commercial deal, distributors must display "Built with Llama," derivative model names must start with "Llama," and an Acceptable Use Policy is incorporated by reference. Downloads are gated — Hugging Face requires your legal name, date of birth and organisation, and approval isn't automatic. For dense models, Llama 3.1/3.3 (8B/70B) remain the community's stable picks.

DeepSeek — MIT, all the way up to 1.6T

DeepSeek's V4 family arrived 24 April 2026 with the cleanest license story in the frontier class: the V4-Flash model card states the repository and the model weights are MIT-licensed. V4-Pro is 1.6T total / 49B active; V4-Flash is 284B total / 13B active, both with 1M-token context and switchable thinking modes. The official announcement leads with agentic coding and reasoning, and the Flash card posts LiveCodeBench 91.6 and a 3052 Codeforces rating in max-thinking mode.

The caveat is physics, not law. An MIT license on a 1.6T model is a technicality for most self-hosters; V4-Pro requires datacenter hardware regardless of license, and even V4-Flash at 4-bit needs roughly 150GB+ of memory. Also note DeepSeek-R2 never shipped; reasoning folded into V4's thinking modes, and anything claiming otherwise is rumour.

Mistral — mostly Apache, with one asterisk

Mistral consolidated its lineup this cycle. Mistral Large 3 (December 2025) is a 675B-A41B sparse MoE with image understanding under plain Apache 2.0, alongside the edge-focused Ministral 3 series (3B/8B/14B, also Apache 2.0, 40+ languages). A Mistral Small 4 followed in early 2026, likewise Apache 2.0. Mistral's stated strengths are European multilingual conversation and, on the newer models, agentic coding with configurable reasoning effort.

The asterisk is Mistral Medium 3.5 (April 2026, dense, 256K context): its official model card lists a "modified MIT license" — and "modified" is doing unspecified work. Read the actual LICENSE file in the repo before any commercial deployment; don't assume vanilla MIT.

Phi — small, MIT, and honest about its limits

Microsoft's current open line is still the Phi-4 family — no Phi-5 has officially shipped despite what third-party blogs claim. Phi-4 proper is a 14B dense model under MIT with 16K context, and its model-card numbers are the reason it punches above its weight class: MMLU 84.8, MATH 80.4, HumanEval 82.6. Phi-4-mini (3.8B, 128K context), multimodal and reasoning variants round out the family.

Microsoft's own model card states the main caveat: Phi-4 is "primarily optimized for English," and its multilingual performance is "substantially reduced." For ASEAN-language workloads, Qwen or Gemma are better bets.

Gemma — the license conversion of the year

Gemma 4 (2 April 2026) is the release that changed the calculus: Google's announcement puts it under "a commercially permissive Apache 2.0 license" — a clean break from Gemma 1–3's custom Terms of Use, which (per the still-live terms) reserve Google's right to restrict usage "remotely or otherwise." Sizes span E2B and E4B (effective 2.3B/4.5B — note those are effective, not total, counts), a 26B MoE, a 31B dense, and a 12B added in June. All process images and video natively; the small ones add audio input, run offline on phones, Raspberry Pi and Jetson Orin Nano per Google's own claim, and Google cites 140+ languages.

One caveat: the Apache grant applies only to Gemma 4. Gemma 2/3/3n stay under the old restrictive terms, remote-restriction clause included — if you standardised on Gemma 3, that clause still applies to you.

The license table

This is the money table. The "License" column shows the license you actually operate under, pulled from official documents, not marketing labels.

One row that deserves a sentence: OpenAI's gpt-oss models (August 2025) are genuinely Apache 2.0 and strong text-only reasoners, but ship with a separate usage policy layered on top — permissive, not unconditional. Read both documents.

What you can actually run

A rule of thumb we'll state as editorial guidance, not vendor fact: a 4-bit quantised model file weighs roughly 0.55–0.6 GB per billion total parameters, and the file plus context cache has to fit in VRAM (or unified memory, at reduced speed). MoE models spend less compute per token but still need the whole model in memory — sparsity buys speed, not RAM.

Tooling is no longer the bottleneck. Qwen3.5/3.6 has official Ollama, llama.cpp, vLLM, SGLang and MLX support; Gemma 4 and Phi-4 ship across Ollama and vLLM; DeepSeek V4 has quantised llama.cpp builds plus official vLLM/SGLang Docker images; Llama 4 and gpt-oss are on Ollama and vLLM. One trap worth flagging: Ollama's featured :cloud tags (e.g. deepseek-v4-flash:cloud) run on hosted infrastructure — that is not self-hosting. Pull the local tags and check the true download size before committing.

What this guide doesn't cover

Honest limits. We haven't covered fine-tuning — licenses interact with derivative models in ways this roundup only gestures at (Llama's naming rules being the sharpest example). We haven't measured quantisation quality loss — our hardware figures assume 4-bit files, and how much capability q4 costs varies by model and task. And we've said nothing about inference-server operations: batching, KV-cache management, and multi-GPU serving are their own discipline. We also stayed within the six major Western-visible families plus gpt-oss; China's other open-weight labs deserve their own roundup. The clearest omission is Moonshot AI's Kimi K-series — K2.6 (April 2026) is a 1-trillion-parameter, 32B-active MoE with 256K context, and Moonshot publishes the weights on Hugging Face under a modified MIT license (read the actual LICENSE file, same caution as Mistral Medium). It is a genuine frontier open-weight model; we left it out for the same reason V4-Pro barely qualifies above — at roughly 595GB for the K2.5 weights alone, self-hosting Kimi is a datacenter exercise, not a workstation one. All versions, license terms and hardware claims were verified in July 2026 against official model cards and license documents; licenses change, so re-read the actual text before you ship anything commercial.