Embeddings

TL;DR: An embedding model turns a piece of text into a fixed-length vector positioned so that things with similar meaning are close together and unrelated things are far apart. You then measure relatedness with cosine similarity and retrieve nearest neighbors. This is the bridge from messy unstructured content to math you can index and search, and it is the backbone of semantic search and RAG. The catch: an embedding is only as good as the model that produced it, and the right similarity metric and domain fit matter.

Meaning as geometry

The core idea is to place text in a space where distance encodes meaning. "How do I reset my password?" and "I forgot my login credentials" use different words but should sit close together; "what is the refund policy?" should sit far away. An embedding model learns that geometry from data, so you can find related content without shared keywords, which keyword search cannot do.

You compare two embeddings with cosine similarity, the cosine of the angle between the vectors, which captures direction (meaning) and ignores magnitude (length). On normalized vectors this is just a dot product, which is why vector databases optimize for it (see cosine similarity and metrics).

What embeddings unlock

Once content is embedded, a lot becomes a nearest-neighbor query:

• Semantic search and RAG retrieval (find the chunks most relevant to a question, see the RAG pipeline).
• Clustering and deduplication (group near-identical or related items).
• Recommendation and matching (similar users/items are nearby).
• Classification by similarity (nearest labeled examples).

Modern embeddings are produced by transformer encoders fine-tuned with contrastive learning (pull related pairs together, push unrelated apart, see contrastive learning), and many are multimodal (CLIP-style), placing text and images in the same space so a text query can retrieve images.

Where embeddings quietly fail

• Domain mismatch. A general embedding model can miss domain jargon (legal, medical, internal product names). Retrieval silently returns near-misses.
• Wrong metric / unnormalized vectors. Using Euclidean distance where cosine is intended, or forgetting to normalize, distorts neighbors.
• Embedding drift. If you upgrade the embedding model, old and new vectors are incomparable; you must re-embed and rebuild the index.
• Exact-term blindness. Embeddings blur exact identifiers (codes, SKUs, names), which is why production search is usually hybrid (embeddings plus keyword/BM25).

Why interviewers probe this

Embeddings are where "we have a pile of documents" becomes "we can search and reason over them," so they sit under search, RAG, recommendation, and clustering. A strong answer explains meaning-as-geometry and cosine similarity in one breath, then names the operational traps, domain fit, normalization, drift, and the need for hybrid search, that separate a working retrieval system from a demo.

Common misconceptions

• "Embeddings understand text." They place text in a useful geometry learned from data; they do not reason about it.
• "Any embedding model works anywhere." Domain fit matters; a generic model can miss specialized vocabulary.
• "Cosine and Euclidean are interchangeable." Cosine captures direction/meaning; use it (and normalize) for text embeddings.
• "You can swap embedding models freely." A new model's vectors are incomparable to the old ones; re-embed the whole corpus.

Key takeaways

• An embedding maps content to a vector so semantic similarity becomes geometric closeness.
• Cosine similarity (direction, not magnitude) is the standard comparison; vector indexes optimize for it.
• Embeddings power semantic search, RAG retrieval, clustering, and recommendation.
• Domain fit, normalization, drift, and exact-term blindness (hence hybrid search) are the real-world failure modes.