How Big a "Small" Model Really Is

"Runs on a laptop" is the promise of a small open model, and the number usually attached is its parameter count: 7B, 8B, 3.8B. But a parameter count isn't a file size. It's a count of weights, and the space they occupy depends on how precisely each one is stored.

We measured what the repositories actually publish.

What one usable copy weighs

Mistral 7B, the model most often described as the one that runs locally, is 14.5 GB of weights. That is before anything else the model needs in order to actually produce a token.

The arithmetic that turns parameters into bytes

Each weight is stored at some precision, and the precision decides the size:

• float32 — 4 bytes per parameter
• float16 — 2 bytes per parameter, the usual full-precision download
• 8-bit — 1 byte
• 4-bit — roughly half a byte

So the floor for a 7B model is 14 GB at float16, 7 GB at 8-bit, and around 3.5 GB at 4-bit. Our measured 14.50 GB for Mistral works out at 2.01 bytes per parameter — float16, exactly as the arithmetic predicts. Phi-3 mini comes in at 2.01 as well.

The story is different for older, smaller models. BERT base measures 4.83 bytes per parameter, MiniLM 4.11, Whisper small 3.97 — all float32. Models from that era were published at full precision and never re-released in half, which is why a 110-million-parameter model still costs half a gigabyte.

This is the practical use of the arithmetic: you can predict the download before you start it, and you can tell which precision a repository is really shipping by dividing.

The duplication nobody mentions

Look at the last two columns again. For most of these models, the repository is far larger than one usable copy of the weights.

Repositories publish the same weights several times over — safetensors alongside PyTorch .bin, sharded alongside a single consolidated file, sometimes ONNX and TensorFlow variants too. Every format is there so that whichever framework you use finds what it expects.

The consequence is that a naive git clone of the MiniLM repository downloads nearly ten times what you need. Use the library rather than cloning, or fetch only the specific files your framework loads. You'll save yourself most of the download.

Phi-3 mini, at 1.00×, is the counter-example: its repository publishes one format and nothing else.

Weights are a floor, not the memory bill

This is where a plan meets reality.

The figures above are what the weights occupy. Running a model also needs:

• The KV cache, which stores attention state for the context so far. It grows with context length and with batch size, and at long contexts it can rival the weights.
• Activations during the forward pass.
• Framework overhead — the runtime, the tokenizer, whatever else is resident.

So a machine with exactly 16 GB of memory does not run a 14.5 GB model. It runs out. The practical rule of thumb is to leave plenty of headroom above the weight size and to treat any memory figure quoted without a context length as incomplete.

Quantisation is what makes a 7B model laptop-sized. At 4-bit the same weights are around 3.6 GB, which changes what is possible on ordinary hardware. It also changes output quality, by an amount that depends on the model and the task and which this measurement says nothing about.

Working out the download before you start it

The arithmetic runs in both directions, which makes it a practical tool.

Given a parameter count and a precision, you know the size: multiply. A 13B model at float16 is about 26 GB, at 8-bit about 13 GB, at 4-bit about 6.5 GB. If a repository is offering you a 4 GB file for a 13B model, it is quantised below 4-bit and the trade-off is quality.

Given a file size and a parameter count, you know the precision: divide. This is the more useful direction, because repositories do not always label clearly. A file that works out at 1.1 bytes per parameter is 8-bit whatever the filename says.

Two practical consequences. Check the number before you start a download on a metered connection — the difference between formats in the same repository can be several gigabytes. And match the format to your runtime before you fetch anything: GGUF for llama.cpp-family tools, safetensors for the Python stack, ONNX for ONNX Runtime. Downloading the whole repository to find out which one you needed is how the duplication figures above turn into your bandwidth bill.

One model we could not measure

Llama 3.1 8B is gated: reading the file index requires authentication. We have left it out of every precision figure above rather than estimate it.

That decision matters because the alternative was worse. Our first run fell back to the largest single file when the index could not be read, and reported 0.62 bytes per parameter — "4-bit or smaller" — for a model published at float16. A gated repository is unmeasured, not small, and a harness that quietly substitutes a partial view produces a confident wrong answer rather than an honest gap.

Two companion measurements sit either side of this one. What you are trusting when you download a model asks what those same repositories tell you about where the weights came from — 99% ship a model card, and one in five declares what it was trained on. And your AI API call leaves the country measures the alternative: from Singapore, a hosted model answers in about a quarter of a second, which is the latency you are buying back by running 14.5 GB locally.

What this does and does not tell you

Parameter counts are sourced from each model's published documentation, not measured here. Everything else — file sizes, formats, the bytes-per-parameter arithmetic — comes from the Hub's own file listing.

Five models is a sample chosen to span the range the phrase "small model" is used across, from a 22-million-parameter embedder to a 7-billion-parameter chat model. It is not a survey.

A note on the method: An earlier version of this measurement grouped files by extension and reported Mistral 7B at 4.03 bytes per parameter — float32, for a model everyone knows is float16 — because the repository ships a consolidated file and a sharded set, one copy of the weights stored twice under the same extension. The control we run on every execution caught it: a known float16 repository has to land near 2 bytes per parameter, and when it does not, the arithmetic is wrong rather than the model. It now resolves one coherent set from the shard index, which is the repository's own statement of which files form a model.

Figures are from 19 August 2026; the script is in our repository.