Tokens: the unit everything else is priced in

TL;DR: A model never reads your text. It reads tokens, and tokens are the unit your bill, your latency, your context limit, and a surprising number of your architecture decisions are all denominated in. Learn to estimate in tokens now and the next six modules stop feeling abstract.

Where you are. You have just come from a map of the whole stack. This module opens the box: what a language model actually does with the text you hand it. We start with tokens because everything later in this course is measured in them. Nothing here requires machine learning background. If you can read a function signature, you can follow it.

The one idea

You send a string. The model does not see a string.

A tokenizer chops your text into subword pieces and turns each into an integer. The model only ever sees that sequence of integers. It has no access to your spaces, your capital letters as such, or your word boundaries beyond what the chopping preserved.

That sounds like an implementation detail. It is not, because the token is the unit of account for the entire job. Every meaningful budget you will manage is priced in tokens:

• Money. Providers bill per token in and per token out, at different rates. Your cost per request is a token count, not a word count.
• Time. Output tokens are generated one at a time, so response latency scales with how many tokens you ask for. Input is processed far faster than output is produced, which is why a long prompt with a short answer often feels quicker than a short prompt with a long answer.
• Space. The context window is a token budget. Everything competing for it (system prompt, conversation history, retrieved documents, tool definitions, the answer itself) is spending from one pool.
• Quality. When the pool runs short, something gets dropped or summarised, and what you drop decides what the model can no longer see.

Four different problems, one currency.

Why an engineer new to this gets surprised

Three surprises catch almost everyone, and all three are the same misunderstanding wearing different clothes.

A token is not a word. For ordinary English prose the rough working figure is that a token averages around three quarters of a word. That ratio is a planning number, not a law, and it moves. Code tokenizes worse than prose because punctuation and indentation eat tokens. Rare technical terms split into several pieces. Most importantly, languages other than English routinely cost noticeably more tokens for the same meaning, because the vocabulary was fit mostly to English text. If you are building for a multilingual audience, that is a real budget line, and it is invisible until you measure it.

The conversation is not free. A chat feels stateful. It is not. Most APIs are stateless, so every turn resends the whole history. A twenty-turn conversation is not twenty small requests; it is twenty requests where the last one carries nineteen turns of history in its input. Cost per turn grows as the conversation goes on, and people discover this on an invoice.

Long context is not a solved problem just because the window is large. A large window tells you what fits. It does not tell you what you should put there, what it costs to put it there, or whether the model will use the middle of it well. "Just put everything in the context" is the most common beginner instinct and the most common thing a senior engineer talks someone out of.

The habit to build now

Estimate before you build. Not precisely: to the right order of magnitude.

The shape of the calculation is always the same, and it is worth doing on paper before writing code:

tokens per request  =  system prompt
                     + conversation history carried forward
                     + retrieved context
                     + tool definitions
                     + expected output

cost per request    =  input tokens  x input rate
                     + output tokens x output rate

monthly cost        =  cost per request x requests per month

Put your own numbers in. If the answer is a few hundred dollars, proceed. If the answer is six figures, you have just learned something important for the price of five minutes, and the fix is almost always architectural rather than a cheaper model: retrieve less, cache more, summarise history, or shorten the output.

That last point is the reason this lesson is second in the course rather than buried in a chapter on cost. Token thinking is not a billing topic. It is the constraint that shapes the design.

Where this comes back

Keep this lesson in mind, because the following are all token problems in disguise:

• Chunking (module 4) is the question of how many tokens of a document you retrieve at once, and what you sacrifice at the boundaries.
• Context budgeting for agents (module 5) is deciding what a loop is allowed to remember when it cannot remember everything.
• Caching (module 7) works because repeated input tokens can be charged differently from fresh ones.
• The latency and cost triangle (module 7) is drawn in token units on both axes.

An engineer who thinks in tokens finds those four topics obvious. An engineer who does not finds them arbitrary.

This module gives you what building requires and stops there. If you want the full academic treatment of the same ground in one sitting, this lecture is that, from a course you can trust to be careful.

WATCH

Lec 21. Language ModelsMIT OpenCourseWareBackground for this whole module rather than for this lesson. Watch it if you want the proper version of how these models are built and trained, which is deeper than anything you need here but makes the rest of the module feel less like a set of rules.

Do this before moving on

Take a real prompt of your own, or any two paragraphs of your writing, and get its token count from a tokenizer for the model family you plan to use. Then do the same for the same text translated into another language, and for a block of source code of similar length. The three numbers will not match. That gap is the lesson.

Go deeper

• Tokenization is the mechanism underneath this lesson: how byte pair encoding builds a vocabulary, why there is no unknown word, and why models miscount letters. Read it now if you want the how; this lesson only needed the what and the why.
• Context window explains the shared pool in more detail, and is the direct prerequisite for module 4.
• Multilingual models covers the non-English cost gap properly, and matters if your users are not all English speakers.
• Practice question: How do you estimate the true cost of self-hosting an LLM versus paying an API? puts the arithmetic above under interview conditions.
• Practice question: Design a cost-control and quota system for an internal LLM platform is where this thinking ends up once several teams share one bill. Save it for later; it assumes module 7.

Key takeaways

• The model reads tokens, and tokens are the shared currency of cost, latency, context, and quality.
• A token is roughly three quarters of an English word, worse for code, and worse again for most non-English text. Treat every such figure as a planning estimate.
• Stateless APIs resend history, so cost per turn grows through a conversation.
• Estimate tokens per request on paper before you build. The expensive mistakes are architectural, and paper is where you catch them.

LEARNING LAB1 of 4

Check yourself before an interviewer does. Answer from memory first.

A chat feature bills far more than expected by turn fifteen, though each user message is short. What is the most likely cause?

Output tokens are billed at a higher rate than input tokensThe full conversation history is resent as input on every turnThe tokenizer splits short messages into more tokens than expected