Tokens and Context Windows, Explained From Zero

You are being charged in a unit nobody explained to you. Every AI provider prices in "tokens", every model has a "context window", and both are quoted as though everyone already knows what they mean. Most people do not, including plenty who use these tools daily and pay the bills.

This is the explanation, with no jargon and no assumed background. If you already know what a token is, you want the guide on what an LLM actually costs instead. This one starts from zero.

A token is a chunk of text. That is all.

Before a model can read anything, it chops the text into pieces. Those pieces are tokens. The model never sees letters or words the way you do — it sees a stream of these chunks.

Here is a real sentence, chopped exactly as a current model would chop it:

"The cat sat on the mat."
   7 tokens:  [The] [ cat] [ sat] [ on] [ the] [ mat] [.]

Two things to notice, because both surprise people.

The spaces are inside the tokens. It is [ cat], not [cat] — the space before the word belongs to it. That is why a full stop gets its own token: nothing precedes it but the letter t.

Seven tokens for six words. In ordinary English a token works out at roughly four-fifths of a word. The rule of thumb you will see quoted everywhere is ¾, or "100 tokens ≈ 75 words"; measured across a thousand words of this site's own English copy it came out at 0.81 words per token, so ¾ is a slightly pessimistic version of a decent guess. Either way it is a guess for English prose only, and a bad one for everything else — which is the rest of this guide.

Why is it called a "token"?

The word is borrowed, not invented for AI. It comes from computing, where it is decades older than any language model.

When a programming language is compiled, the first step is tokenisation: a stream of characters is split into the smallest units the machine will actually work with — a keyword, a number, a bracket. Those units have been called tokens since the 1960s. Language models do the same job on ordinary prose, so they inherited the same word.

The everyday meaning fits neatly too. A bus token or a casino chip is not the thing itself; it is a countable stand-in for it, handed over one at a time. That is exactly what these are — the text is not stored as text, but as a sequence of numbered stand-ins, each one an entry in that fixed list. When you are billed "per token", you are being billed per stand-in handed over.

Why not just count words?

Because the chopping is not done by meaning. It is done by how often a chunk appeared in the material the model learned from. Common chunks get to be a single token. Rare ones get broken into smaller pieces.

Watch what that does to ordinary words:

"unbelievable"     3 tokens:  [un] [bel] [ievable]
"strawberry"       3 tokens:  [st] [raw] [berry]
"tokenisation"     2 tokens:  [token] [isation]

Nobody would divide "unbelievable" as un-bel-ievable. It is not how the word is built. But bel and ievable happen to be chunks that earned a slot, so that is the split. The model is not parsing language here; it is looking things up in a fixed list. The list used above is called o200k_base, and the name is the size — roughly 200,000 chunks, where its predecessor cl100k_base held about half that.

This is also why models were, for years, notoriously bad at questions like "how many R's are in strawberry". The model never saw s-t-r-a-w-b-e-r-r-y. It saw [st][raw][berry]. Corrected 3 August 2026: this passage used to say models are famously bad at that question, with no source. We later ran it as a controlled test and both models we could test scored six out of six. The mechanism above is real and unchanged — a model genuinely does not see the letters — but current models evidently work around it, and asserting a live weakness we had not measured was our error. See what AI is actually good and bad at.

Why your city costs more than London

Here is where it stops being trivia. The list of chunks was built from text that was overwhelmingly English and overwhelmingly Western. Whatever appeared often is cheap. Whatever did not, is not.

London is one token. Kuala Lumpur is three. Johor Bahru is four, and it does not even split at the space — it goes [Joh][or][ Bah][ru], cutting both words in half.

Singapore gets a single token because the word appears constantly in English-language text. That is the whole mechanism, visible in one table: not importance, not size, just how often the chunk turned up.

Now the same thing in seven writing systems

Latin letters are the easy case. Here is one phrase — thank you — written properly in seven languages, and what each costs:

Read the top row again. Ten Japanese characters cost one token. ありがとうございます is a single chunk, cheaper than the two-word English "thank you", because that exact polite phrase appears so relentlessly in Japanese text that it earned its own slot. Length has nothing to do with it. Frequency is everything.

Then look at the two Chinese rows. Same greeting, same meaning, one written in Simplified and one in Traditional. Simplified gets [谢谢] as a single chunk — the whole word. Traditional gets [謝][謝][你], every character standing alone, because 謝 never appeared often enough to be worth a slot of its own.

What happens when a character is too rare

It gets worse than one-token-per-character. Watch a full sentence in both scripts:

Simplified   "公司审查了季度账目"   7 tokens
             [公司][审][查][了][季度][账][目]

Traditional  "公司審查了季度帳目"   8 tokens
             [公司][?][?][查][了][季度][帳][目]

Those question marks are not a display fault in this page. The character 審 has no token of its own, so it is cut into pieces of its own byte encoding — fragments that are not valid characters at all and cannot be printed. It takes two of them to reassemble one 審.

Push further into genuinely rare characters and the pattern holds: 龜鬱籲, three characters, comes to six tokens — two byte-fragments each, none of them printable on their own.

So a Taiwan or Hong Kong operation writing the same words as a mainland one pays more, not through any policy, but because the chopping list was built from text where Simplified forms dominated. We measured that gap properly across a parallel corpus and it runs to about 21%; the numbers are in the cost guide, along with the same figure for Malay, Thai, Japanese and Korean.

Does ChatGPT count the same as Claude? No.

This is the question that catches people out, and the answer is blunt: there is no standard. Every model family has its own chopping list, built separately, and the same sentence produces different counts on different models.

It is not even stable within one company over time. Both columns below are OpenAI tokenisers — the older one used by GPT-4, and the current one:

Same company, same text, one generation apart — and the Thai greeting more than halved. English barely moved, because it was already well served. Everything else got substantially cheaper, which is what happens when a vocabulary is rebuilt with more of the world's writing in it.

So: a token count is only meaningful alongside the model it was measured on. "This document is 4,000 tokens" is an incomplete sentence.

There is a second, practical wrinkle. OpenAI publishes its tokenisers; Anthropic and Google do not. Anyone can compute an exact GPT count offline — that is what the examples in this guide are. Nobody outside those companies can compute an exact Claude or Gemini count, so every tool that offers you one, including ours, is estimating and should say so. If a tool gives you a confident exact figure for Claude, it is guessing and not telling you.

Input tokens and output tokens

Every provider quotes two prices, and this is the difference between them.

Output almost always costs more per token than input — commonly three to five times more. The reason is mechanical rather than commercial: the model reads your input in one pass, but it produces its answer one token at a time, each one requiring a fresh pass over everything written so far. Reading is cheap. Writing is slow and expensive.

The practical consequence is the opposite of most people's instinct. Sending a long document and asking for a short answer is cheap. Sending a short question and asking for a long essay is expensive. Length of the reply is the lever, not length of the question.

A rough shape, using a mid-priced model at $3 per million input and $15 per million output:

The first sends 750 times more text and costs about the same. That is the input/output split doing its work.

Where does the price actually come from?

Reasonable next question: is a token price tied to something physical — electricity, hardware, data centre space — or is it a number somebody chose?

Both, and the split is worth understanding.

What you are really renting is a slice of a GPU's time

A model does not run "in the cloud" in any vague sense. It runs on a specific graphics processor with the model's weights loaded into its memory, and that chip is occupied while your request is served. Everything else follows from that.

The dominant cost is not electricity. It is the machine itself, spread over its useful life. A data-centre GPU costs tens of thousands of dollars and is written off over a few years, and that amortisation typically exceeds the power bill. Cloud rates for an H100-class chip have settled around $3 an hour in 2026, which already bundles the hardware, the power, the cooling and the building.

From there the arithmetic is simple. If a server sustains 2,000 output tokens per second, one hour produces 7.2 million tokens, and $3.20 of rent spread across them is about 44 cents per million tokens. Push throughput to 8,000 tokens per second by serving many users at once and the same rent falls to about 11 cents. That is the whole reason batching matters: the cost is time-on-chip, so the more requests share a pass, the less each one carries.

Why output really does cost more

Earlier I said output is priced three to five times higher than input. The reason is physical rather than commercial.

Reading your input is one pass — the whole thing can be processed in parallel. Writing the reply cannot: each token depends on the one before it, so the model runs again for every single word it produces. And on each of those runs the bottleneck is not arithmetic but memory bandwidth. The chip must haul the model's weights from memory to do anything at all, and it finishes the sums long before the memory can feed it more. During generation an expensive GPU spends much of its time waiting.

So input is bulk work done once, and output is a sequence of individually slow steps. The price gap reflects a real difference in machine time.

The clearest proof that price tracks cost

Look at what happens when the physical work disappears. Providers charge dramatically less for cached input — text they have already processed and can reuse:

Nothing about the text changed. What changed is that the work was already done, and the price collapsed by two orders of magnitude. Batch tiers behave the same way — send work that can wait, let the provider fit it into idle capacity, pay roughly half. Prices that move like this are tracking something real.

But it is not cost-plus, and you should not assume it is

Three things break the tidy story.

Training is not in the meter. Building the model cost enormous sums that have nothing to do with your particular request. That has to be recovered somewhere, and it is not attributable per token, so it is spread across the price by judgement rather than measurement.

Comparable models differ by more than their costs do. Models of broadly similar capability are priced across a range far wider than any plausible difference in what they cost to run. That gap is strategy, not physics.

The industry as a whole is not covering its costs. Prices today are set in the middle of an enormous capital build-out, and a price can sit below cost for years when someone has decided market position matters more than margin. That is a competitive posture, and it is a whole argument of its own.

The practical upshot for you: the physical floor is real and falling — inference has become 10 to 100 times cheaper since 2023 — but today's price is a business decision layered on top of that floor. Budget on published rates, not on what you think the electricity costs, and re-check the rates often, because the gap between the two moves.

Three different things you can actually buy

"Using AI" covers three quite different purchases, and they get muddled constantly. They sit in a stack, each one built on the one below.

GPU as a service: renting the machine

You rent a processor by the hour and do everything else yourself — load a model onto it, handle requests, keep it fed. The meter runs whether the chip is working or sitting idle, which is the entire catch.

People assume this must be cheaper, because you have cut out a middleman. Usually it is not. Here is a rented H100-class GPU at roughly $3.20 an hour, priced per million tokens at different levels of use:

To beat a cheap open-weight API at around $0.28 per million, you would need to sustain roughly 3,200 tokens per second, around the clock, forever. Not at your busy hour — as a daily average, nights and weekends included. And if the machine is only genuinely busy a tenth of the time, every figure above multiplies by ten.

That is the honest arithmetic, and for most businesses it points one way: renting a GPU costs more than buying tokens, because you cannot fill it. The providers can, because they are filling it with everybody's work at once — and that is what the middle layer is.

The inference provider, and why "token factory"

These companies take open-weight models — Qwen, DeepSeek, Llama and the rest — run them at scale, and sell you the output by the token. You never see a machine. Their whole business is the thing you cannot do: packing thousands of unrelated requests through the same hardware so that no capacity is wasted.

The important structural fact is that the model is not theirs. Qwen is Alibaba's, published openly, so several providers host the identical model and compete on price, speed and location. If one raises prices you move, and the model you are using does not change. That is a commodity market, and it behaves like one.

The model vendors are a different business

Anthropic, OpenAI and Google sell tokens too, and the transaction looks the same from your side. It is not the same. Their models are closed, so there is exactly one place on earth to buy Claude, and exactly one to buy GPT. No second source, no price shopping, no switching provider while keeping the model.

This is the practical difference behind the price gap. It is not only that open-weight models are cheaper to run — it is that several sellers are competing to sell you the very same one, while a closed model has a single seller setting a single price. When you commit to a closed model you are accepting that, and it is a reasonable trade if the model is genuinely better for your work. It is just worth doing knowingly.

Why the building is now called a factory

Against that backdrop, the industry's rebranding makes sense. When operators describe themselves as token factories, it is not marketing whimsy — it is a fairly precise description of what changed, and the strongest possible answer to whether token prices are tied to physical things.

The framing came out of NVIDIA and was everywhere by 2026. A traditional data centre is a place that stores and serves data — its output is hard to quantify, so it was measured by what went in: floor space, racks, teraflops. An AI factory is described instead as a plant with a product. Electricity and data go in one end; tokens come out the other, continuously, and get sold by the million.

Once you accept that framing, the industry's accounting follows from it. The metric that matters becomes tokens per second per megawatt, and the revenue model reduces to something a factory manager would recognise:

revenue  =  tokens per watt  ×  available gigawatts

Which is to say: how efficiently you convert power into tokens, multiplied by how much power you can actually get. Note what has quietly become the binding constraint. Not chips, not engineers — electricity. A factory that cannot secure another gigawatt cannot produce more, however many processors it can buy.

This is also why the hardware generations are now marketed in these terms. NVIDIA's current rack-scale systems are pitched at roughly 50 times more tokens per megawatt than the Hopper generation they replace, and about 35 times lower cost per token. Whatever you make of the vendor's own numbers, the unit of comparison is the point: the industry has stopped selling capability and started selling throughput per unit of power.

Jensen Huang's framing for it was the end of the "training era" and the beginning of the "inference era". Training built the models. Inference is the ongoing business of running them — and that business is measured in tokens, which is why your bill is too.

Read it with one eyebrow raised

The formula is elegant and it is published by the company that sells both terms in it. "Revenue = tokens per watt × gigawatts" makes the case that the constraint on your business is how much NVIDIA hardware you can power, which is a convenient conclusion for NVIDIA. The underlying physics is sound; the framing is not neutral.

What it does tell you, reliably, is where the industry believes its costs live. When an entire sector reorganises its accounting around tokens per watt, the answer to "is a token price tied to electricity and hardware?" is: yes, that is now the explicit unit of production. The link is no longer implicit — someone has written it as a formula and put it on a slide.

And it closes a small circle. The word "token" started as compiler jargon for a chunk of text. It became the unit you are billed in. It is now the unit an industry manufactures, measured against megawatts. Same word, three jobs, one of which involves a power station.

The context window: a desk, not a memory

A model's context window is the total amount of text it can hold in view at one moment. It is quoted in tokens — 128,000, 200,000, a million.

The useful picture is a desk. Everything the model can consider has to be laid out on that desk at once: your instructions, the document you pasted, the whole conversation so far, and the space where it will write its answer. When the desk is full, something has to come off — and what comes off is usually the oldest part of the conversation.

This is not memory. Nothing is stored between conversations. Start a new chat and the desk is bare. The model does not remember you; it only ever sees what is on the desk right now.

How to work out whether it fits

Add up everything that has to be on the desk together:

system instructions      ~500 tokens
the document you pasted  ~90,000 tokens
conversation so far      ~3,000 tokens
your new question        ~50 tokens
──────────────────────────────────────
already on the desk       93,550 tokens
model's window           128,000 tokens
room left for the answer  34,450 tokens

That is the whole calculation. The reason it matters is the last line: the answer has to fit too. If you paste a 125,000-token document into a 128,000-token window, you have left room for about two pages of reply, and the model will either produce something truncated or refuse. People routinely fill the desk and then wonder why the response is cut short.

A rough conversion, for English: 128,000 tokens is around 96,000 words, or a 350-page book. For Malay or Chinese, expect meaningfully less — the same desk holds fewer pages, because each page costs more tokens.

The part almost nobody is told

Models are stateless. They do not remember the previous message. So how does a chatbot follow a conversation?

It re-sends the whole thing. Every single turn.

When you type your fifth message, the application quietly sends messages one through four back along with it, so the model can see what was said. On turn ten, it sends all nine previous exchanges again. You are paying for that history every time.

turn 1  sends    500 tokens
turn 2  sends  1,000 tokens   (turn 1 again, plus turn 2)
turn 3  sends  1,500 tokens
…
turn 10 sends  5,000 tokens
──────────────────────────────────
total input   27,500 tokens, not 5,000

Ten turns of 500 tokens each is 5,000 tokens of typing and 27,500 tokens of billing — five and a half times more. It is also why long conversations get slower and start forgetting the beginning: the desk fills up, and the earliest papers get pushed off the edge.

The fix is unglamorous and effective. Start a fresh conversation when you change subject. Do not continue a hundred-turn thread out of habit.

Try it on your own text

None of this has to be taken on trust. Paste your own writing into our token counter and it will give you the exact count for OpenAI models and a labelled estimate for the others. If you want to see the chopping rather than just count it, the tokenization visualizer shows every boundary, which is the fastest way to understand why your product names and your language cost what they do.

Then, when you want to know what it all adds up to on a real month's traffic, that is the next guide.