How to Generate Embeddings Locally

If you followed our guide to running Qdrant in Docker you now have a vector database and nothing to put in it. Vectors have to come from somewhere, and that somewhere is an embedding model.

You can run one on a laptop. The model used here is 46 MB.

What an embedding actually is

An embedding model turns a piece of text into a list of numbers — a position in space. Texts that mean similar things land near each other, whether or not they share any words. That last clause is the whole point, and this guide sets out to demonstrate it rather than assert it.

Run the model

Ollama is the least ceremonious way to serve an embedding model locally:

docker run -d --name ollama -p 11434:11434 ollama/ollama:latest
docker exec ollama ollama pull all-minilm

One number worth knowing before you start: the Ollama image is 2.78 GB and the model is 46 MB. The runtime is about sixty times the size of the thing it runs. That is fine on a laptop and worth thinking about on a small VPS or in a CI image.

The model is all-minilm: 23M parameters, BERT family, and it produces 384 numbers per text. Ollama reported version 0.32.15 on this run.

Generate one

curl http://localhost:11434/api/embed \
  -d '{"model":"all-minilm","input":"the invoice is overdue"}'

You get back a list of 384 floats. The dimension count is critical: it must exactly match the size you gave your Qdrant collection, and if it does not, the insert fails rather than the collection creation. Because different models produce different dimension counts, swapping models later means rebuilding the collection from scratch.

What it costs in time

Cold, the first call took 1077 ms, of which 783 ms was loading the model into memory. That load happens once; the model then stays resident.

Beyond that, we are not going to give you per-call timings, and the reason is worth more than the numbers would have been. Five identical warm calls on this machine ranged over a factor of 7.1, and five identical batch calls over a factor of 11.5 — from 37 ms to 423 ms for the same work. A median computed across that is a number, not a measurement.

A naive reading would suggest that embedding five strings is faster than embedding one. That is not true, and would be an embarrassing thing to teach. Docker Desktop on a shared laptop is simply not a timing instrument. If throughput matters to you, measure it on the hardware you will deploy on, with the batch sizes you will actually use.

The part that justifies the whole exercise

Five short documents, embedded and loaded into Qdrant:

the invoice is overdue
payment has not arrived from the client
deploy the service to production
the server restarted overnight
annual leave policy for new staff

Now search for something a person would actually type:

we have not been paid

Strip the stopwords and that query has exactly one content word: paid. None of the five documents contains it. A keyword search — LIKE '%paid%', or a naive full-text index — returns zero results.

The vector search returns this:

Both money documents come out on top. The second one, the invoice is overdue, shares not a single word with the query — not even a stopword — and the model still placed it second because "invoice", "overdue" and "paid" occupy nearby regions of its space.

Below the two money documents the ranking degrades sensibly: the HR document, then the two for operations, with "deploy the service to production" last at 0.0363.

Read the scores honestly

The top score is 0.4959, not 0.95. Short strings and a 23M-parameter model do not produce confident-looking numbers, and there is no threshold above which a result is "correct".

What matters is the ordering and the gap: 0.50 and 0.37 for the two relevant documents against 0.22 for the next one. In a real system you tune a cut-off against your own data, and you will find that the cut-off moves when you change model, document length or language.

Five strings is also an illustration, not an evaluation. It shows the mechanism works; it says nothing about how the model behaves on ten thousand support tickets, which is the only test that would matter to you.

What to do next

• Match the dimensions. 384 here. Create the collection with the same number or the insert fails.
• Embed the query with the same model as the documents. Two different models put the same sentence in two unrelated places, and the failure is silent — you get results, they are just wrong.
• Store the original text as payload. A vector is not reversible. If you do not keep the text alongside it, a search returns coordinates and nothing you can show a user.
• Try a larger model only when quality plateaus. The 46 MB all-minilm is a reasonable starting point; bigger models are worth their cost only once you can measure the difference on your own corpus.
• Keep keyword search too. Vectors are bad at exact identifiers like invoice numbers and product codes. A robust system runs both kinds of search and merges the results.

What is measured here

Everything above was executed on 21 August 2026 against real containers — Ollama 0.32.15 and Qdrant 1.19.0 — not transcribed from documentation. The script that produces every version, size, dimension and score is in our repository and re-runs the whole sequence.

The cold-start figure is a single observation and a large, robust one. The warm and batch timings are deliberately not published as comparable numbers, because their spread on this machine was 7.1× and 11.5× across identical calls; the script records that spread and refuses the comparison rather than reporting a median that would imply something false.

The retrieval demonstration is five documents and one query. The keyword baseline is computed the same way in the script — content words after stopword removal — so "zero keyword matches" is a calculation you can check rather than a claim you have to take on trust.