Embeddings: why a computer can suddenly tell that two sentences mean the same thing
Embeddings are the reason search stopped depending on matching words. This lesson builds the intuition from a problem you already have, shows what the geometry does and does not know, and sets up why retrieval works at all.
TL;DR: An embedding turns a piece of text into a list of numbers positioned so that similar meanings sit close together. That single move is what lets a computer match "my card was declined" to "payment failed" with no shared words, and it is the foundation under every retrieval system you will build later in this course.
Where you are. You know a model reads tokens, not words. What happens to those tokens next is the first idea in this course you will use directly rather than merely reason about. No maths beyond "a list of numbers" is needed.
Start from the problem
Suppose you have ten thousand support tickets and a user types "my card was declined."
Keyword search fails, and it fails in a way that is worth sitting with. The ticket you want says "payment failed at checkout." Zero words in common. You could bolt on synonyms, but you would be maintaining a synonym list forever, and it would never cover "the transaction did not go through."
The problem is that the computer is comparing spellings. What you want is to compare meanings, and meaning is not a property of the characters.
What an embedding actually is
An embedding model takes text and returns a fixed-length list of numbers. Hundreds or a couple of thousand of them, always the same length whatever you put in. Every piece of text you embed with the same model produces a list of that same length.
That list is a position. With three numbers you would have a point in a room. With a thousand you have a point in a space too large to picture, but the arithmetic that matters does not care how many there are: you can measure how far apart two points are.
The whole trick is that the model was trained so that distance corresponds to meaning. "My card was declined" and "payment failed at checkout" land near each other. "How do I reset my password" lands somewhere else entirely.
So semantic search becomes a geometry problem. Embed every ticket once and store the positions. When a query arrives, embed it and find the stored positions nearest to it. No synonym list, no shared words required.
That is the entire mechanism behind retrieval, and it is why module 4 is possible at all.
The three things beginners get wrong
An embedding is not a summary, and it is not reversible. You cannot read a list of numbers and recover the sentence. It is a position, not a compression. This surprises people who expect to be able to inspect one, and it is why debugging retrieval means comparing distances rather than reading vectors.
Two embeddings are only comparable if they came from the same model. Different models put meaning in different places, and there is no conversion between them. This has a real operational consequence you will meet again: changing your embedding model means re-embedding everything you have stored. Every position moves. Plan for that day, because it arrives.
Close does not mean correct. This is the one that costs people weeks. The geometry captures similarity, which is not the same as relevance and is definitely not the same as truth. Two sentences that flatly contradict each other are often near neighbours, because contradiction is a form of being about the same subject. "The refund was processed" and "the refund was not processed" are about as close as two sentences get.
State that last point as a rule, because it explains a whole class of retrieval bug later: an embedding knows what a piece of text is about. It does not know whether the text is right, current, or the answer to your question.
Do this before moving on
Pick any embedding model with a free tier or a small local one, and embed six short sentences: two that mean the same thing in different words, two on the same topic that contradict each other, and two unrelated ones. Then compute the distance between every pair.
Look for one specific thing: how close the contradicting pair is compared with the unrelated pair. On most models the contradiction scores as highly similar. That number is the reason retrieval systems need a reranking step, and seeing it yourself now will make module 4 much shorter.
Go deeper
- Embeddings is the reference for this lesson: how similarity is actually computed, what dimensionality buys you, and where the representation quietly fails. Read it when you want the mechanism rather than the intuition.
- Vector search is the next step outward, and covers how you find nearest neighbours among millions of stored positions without comparing against all of them.
- Embedding model selection matters the moment you build something real, because the choice is hard to reverse. Save it for module 4.
- Practice question: Explain word embeddings and how they are learned is the version of this an interviewer asks early to check you understand the representation rather than just the API.
- Practice question: How would you migrate to a new embedding model? is the re-embedding problem above, asked as a production question. Harder than it looks, and a common senior screen.
Key takeaways
- An embedding is a position in a space where distance corresponds to meaning, which is what lets search work without shared words.
- Positions from different embedding models are not comparable, so switching models means re-embedding everything you store.
- Similarity is not relevance and not truth. Contradictory sentences are usually close neighbours.
- You cannot read an embedding. Debugging means comparing distances, not inspecting numbers.
Check yourself before an interviewer does. Answer from memory first.
A search over your docs returns a passage that states the opposite of the correct answer, yet scores as highly relevant. What is happening?