Embeddings and Cosine Similarity

TL;DR — An embedding turns a piece of text into a fixed-length list of numbers (a vector) so that texts with similar meaning land near each other in that space. You compare two embeddings with cosine similarity — the cosine of the angle between them, 1 for same-direction, 0 for unrelated — which ignores length and measures direction only. The vector's length (its dimension, e.g. 384 for all-MiniLM-L6-v2, 768/1024 for larger models) is fixed by the model, not the text. Embeddings power semantic search, clustering, and recommendation. They break on out-of-domain text, very long documents (one vector loses detail), and when you use raw cosine on models that need normalization — always match the metric and model to the task.

---

1. Simple explanation

Computers can't compare the meaning of two sentences directly — they need numbers. An embedding model reads text and emits a fixed-length vector (say 384 numbers) that acts as the text's "meaning fingerprint." The trick is that the model is trained so that sentences meaning similar things get vectors pointing in similar directions, even if they share no words. "How do I reset my password?" and "I forgot my login" end up close; "the cat sat on the mat" ends up far away.

Once text is vectors, everything meaning-related becomes geometry. "Find similar documents" becomes "find nearby vectors." "Group these reviews by topic" becomes "cluster these points." "Is this answer relevant to the question?" becomes "is the answer's vector close to the question's vector?" The measure of "close" that works best for text is cosine similarity: it looks at the angle between two vectors and ignores their length, so a short phrase and a long paragraph about the same topic still count as similar.

Analogy — describing people by a personality chart. Give everyone scores on a handful of traits (outgoing, curious, tidy...). Two people with similar charts are similar, even if you described them with different words. You'd compare charts by their shape (pattern of highs and lows), not by who has bigger numbers overall — that shape comparison is cosine similarity, and the chart is the embedding.

---

2. Diagram

   TEXT ──▶  embedding model (all-MiniLM-L6-v2) ──▶  fixed-length vector
   "spicy noodle soup"                                [0.03, -0.21, ..., 0.08]   (384 numbers)

   Compare two vectors by the ANGLE between them (cosine):

        a
        ^          cos = 1.0   (0 deg)   -> same meaning
        |     b    cos = 0.8   (~37 deg) -> related
        |    /     cos = 0.0   (90 deg)  -> unrelated
        |   /
        |  /  theta
        | /
        +--------------->

   cosine( a , b ) = (a . b) / (|a| * |b|)     # dot product over the lengths
   -> length-invariant: only DIRECTION matters, not magnitude

---

3. How it works

3.1 From text to a vector

A modern sentence embedding model is a small transformer that maps any input text to one vector of a fixed size. That size — the embedding dimension — is a property of the model, not the text: all-MiniLM-L6-v2 always returns 384 numbers, bge-base/mpnet return 768, some return 1024. More dimensions can capture more nuance but cost more memory and compute per comparison. The individual numbers are not human-readable ("dimension 57" has no name); only the geometry — which vectors are near which — carries meaning. Two coexisting kinds of vectors are worth separating: these semantic embeddings (hundreds of dimensions, learned, for meaning) versus small hand-designed feature vectors (e.g. a 6-value taste or personality profile, interpretable, for a specific attribute). Use semantic embeddings for "same meaning," feature vectors for "same measured attribute."

3.2 Comparing vectors: cosine, dot, Euclidean

  cosine(a,b)     = (a . b) / (|a| |b|)     angle only; length-invariant   -> DEFAULT for text
  dot(a,b)        = a . b                    angle AND length               -> when length encodes confidence
  euclidean(a,b)  = |a - b|                  straight-line distance         -> when magnitudes are meaningful

Cosine is the default for text because document length shouldn't make two texts "less similar." Dot product equals cosine when vectors are unit-normalized, which is why many pipelines normalize once and then use the cheaper dot product. Euclidean distance is sensitive to magnitude, so it's used when the raw scale means something (some clustering, some image work). Match the metric to how the model was trained: several models are trained with cosine in mind, and using Euclidean on them quietly degrades results.

3.3 Semantic search (nearest neighbor)

The workhorse application: embed a query, embed every item once (offline), and return the items whose vectors are closest to the query's. For a handful of items you compare against all of them (exact); for millions you use an approximate nearest-neighbor index (HNSW, IVF) inside a vector database so search stays fast. The quality ceiling is set by the embedding model — a good index over weak embeddings still returns weak matches.

3.4 Sentence vs document embeddings

A single vector summarizes a short passage well. For a long document, cramming everything into one vector blurs detail ("what is this 40-page report about?" is answerable; "what did page 31 say about refunds?" is not). The standard fix is chunking: split the document into passages, embed each, and retrieve at the passage level — the backbone of retrieval-augmented systems. Averaging word vectors is the crude, older way to get a sentence vector; trained sentence models (SBERT and successors) beat it because they are optimized directly for sentence-level similarity.

Boundary condition. Embeddings are only as good as the data the model saw. On out-of-domain text (legal jargon, a low-resource language, code) a general model's "nearby" can be wrong. Very long inputs get truncated at the model's token limit — text past the limit is silently ignored. And raw cosine on an un-normalized, anisotropic model (vectors clustered into a narrow cone) can make everything look similar. Section 8 lists the fixes.

---

4. The math

4.1 Cosine similarity

For vectors a and b of the same dimension d:

  a . b        = sum_{k=1..d} a_k * b_k            (dot product)
  |a|          = sqrt( sum_k a_k^2 )               (length / L2 norm)
  cosine(a,b)  = (a . b) / (|a| * |b|)             in [-1, 1]

For the non-negative embeddings common in practice, cosine lands in [0, 1]. If you L2-normalize every vector first (a_hat = a / |a|), then cosine(a,b) = a_hat . b_hat — the dot product of unit vectors — which is why normalized-then-dot is the fast path.

4.2 Worked example

Take five toy vectors over five made-up features. king = [0.9,0.1,0.8,0.1,0.0], man = [0.2,0.0,0.9,0.1,0.0]. Their dot product is 0.9*0.2 + 0.1*0 + 0.8*0.9 + 0.1*0.1 + 0 = 0.18 + 0.72 + 0.01 = 0.91. Lengths: |king| = sqrt(0.81+0.01+0.64+0.01) ≈ 1.208, |man| = sqrt(0.04+0.81+0.01) ≈ 0.933. So cosine ≈ 0.91 / (1.208*0.933) ≈ 0.809. Now apple = [0,0,0,0,1] shares no active feature with king, so their dot product is 0 and cosine is 0.000 — unrelated. Ranking the others by cosine to king gives man (0.809) > queen (0.580) > woman (0.290) > apple (0.000). The code below reproduces these exact numbers.

---

5. Real code

import numpy as np

# Toy 'embeddings': tiny hand-made vectors over 5 imaginary features.
vocab = ["king", "queen", "man", "woman", "apple"]
emb = {
    "king":  np.array([0.9, 0.1, 0.8, 0.1, 0.0]),
    "queen": np.array([0.9, 0.9, 0.1, 0.8, 0.0]),
    "man":   np.array([0.2, 0.0, 0.9, 0.1, 0.0]),
    "woman": np.array([0.2, 0.8, 0.1, 0.9, 0.0]),
    "apple": np.array([0.0, 0.0, 0.0, 0.0, 1.0]),
}

def cosine(a, b):
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))

print("cosine similarity (1=same direction, 0=unrelated):")
for w in ["queen", "man", "woman", "apple"]:
    print(f"  king vs {w:<6} = {cosine(emb['king'], emb[w]): .3f}")

# Nearest-neighbour search: which word is closest to a query vector?
query = emb["king"]
ranked = sorted((w for w in vocab if w != "king"),
                key=lambda w: cosine(query, emb[w]), reverse=True)
print("\nnearest to 'king':", ranked)

assert ranked[0] == "man"                       # closest by our toy features
assert cosine(emb["apple"], emb["king"]) < 0.1  # unrelated
print("asserts passed")

# Output:
#   cosine similarity (1=same direction, 0=unrelated):
#     king vs queen  =  0.580
#     king vs man    =  0.809
#     king vs woman  =  0.290
#     king vs apple  =  0.000
#
#   nearest to 'king': ['man', 'queen', 'woman', 'apple']
#   asserts passed

In production you would replace the hand-made vectors with a real model — e.g. SentenceTransformer("all-MiniLM-L6-v2").encode(texts) returns 384-dim vectors — and the exact same cosine / nearest-neighbor logic applies unchanged. Only the source of the vectors changes.

---

6. Real-world example

A support team built semantic search over its help center by embedding every article with a general-purpose model and returning the top cosine matches for a user's question. In testing it looked great. In production, agents complained that unrelated articles kept surfacing near the top. The cause was two-fold: the help center was full of domain-specific billing terms the general model had barely seen (so "chargeback" and "refund" sat closer than they should), and the model's embeddings were anisotropic — nearly everything scored 0.6–0.8 cosine, so the ranking had little room to separate good from bad. Two cheap changes fixed most of it: L2-normalize and switch to a retrieval-tuned model (a BGE-family model with the recommended query prefix), and add a lightweight reranker on the top 20 candidates. Neither change touched the search plumbing — the vectors and the metric improved, and the relevance followed. The recurring lesson: an embedding pipeline's ceiling is the embedding model and metric, not the index — when results are weak, look there first.

---

7. Interview questions companies actually ask

Q1. What is an embedding and why is its dimension fixed? It is a learned, fixed-length vector representation of text whose geometry encodes meaning — similar texts get similar-direction vectors. The dimension (384, 768, 1024...) is a property of the model architecture, not the input, so every text from a given model yields the same-length vector, which is what lets you compare and index them uniformly.

Q2. Why cosine similarity instead of Euclidean distance for text? Because text length shouldn't change topical similarity. Cosine compares direction and ignores magnitude, so a one-line question and a long answer about the same topic still match. Euclidean is magnitude-sensitive and is preferred only when the raw scale carries meaning. Also, many text models are trained with cosine as the target metric.

Q3. How are cosine and dot product related? For unit-normalized vectors they are identical: cosine(a,b) = â · b̂. That's why pipelines often L2-normalize once and then use the cheaper dot product. On un-normalized vectors, dot product mixes in magnitude, which can distort ranking.

Q4. When does averaging word embeddings fail for sentences? It ignores word order and drowns rare, meaning-carrying words in common ones ("not good" averages close to "good"). Sentence-trained models (SBERT and successors) are optimized directly for sentence similarity and handle these cases far better, which is why averaging is now a baseline, not a default.

Q5. Your semantic search returns mediocre results — where do you look first? At the embedding model and metric, not the index. Check for domain mismatch (jargon the model never saw), missing normalization, anisotropy (everything scores similarly high), input truncation past the token limit, and whether the model expects a query/document prefix. Swapping to a retrieval-tuned model plus a reranker usually helps more than index tuning.

Q6. How do you embed a long document? Don't force it into one vector — chunk it into passages, embed each, and retrieve at the passage level; optionally keep a coarse document-level vector for routing. One vector for a long document blurs page-level detail and truncates past the token limit.

Q7. 384 vs 768 dimensions — how do you choose? Higher dimensions can capture more nuance and usually rank slightly better, at higher memory and per-query cost. For latency- or memory-constrained systems a 384-dim model (like MiniLM) is a strong, cheap default; move up only if evaluation shows the quality gain is worth the cost. Benchmark on your own data rather than trusting a leaderboard blindly.

---

8. When to use / tradeoffs

Reach for embeddings when:

• You need "same meaning," not "same words" — semantic search, dedup, clustering, retrieval, recommendation.
• You can precompute item vectors once and compare many queries against them.
• Approximate matching is acceptable (embeddings rank by similarity, they don't prove equivalence).

Do NOT rely on them when:

Honest limits. An embedding is a lossy summary: things it wasn't trained to distinguish will collapse together, and you can't tell from the vector alone what it lost. Cosine scores are relative, not absolute — 0.72 means "more similar than 0.60," not "72% relevant," so hard thresholds transfer poorly across models and domains. General models underperform on specialized text, and the only reliable way to pick a model or dimension is to evaluate on your data, not a public leaderboard. Finally, semantic similarity is not truth: two sentences can be near in embedding space and still contradict each other, so retrieval must be paired with checking, not trusted blindly.

---

9. Summary + related articles

• An embedding is a fixed-length vector whose direction encodes a text's meaning; the dimension (384, 768, ...) is set by the model.
• Compare embeddings with cosine similarity (angle only, length-invariant); it equals the dot product once vectors are L2-normalized.
• Embeddings turn meaning tasks into geometry: semantic search = nearest neighbor, clustering = grouping points.
• Chunk long documents; averaging word vectors is a weak baseline versus trained sentence models.
• Boundary: results are only as good as the model on your domain — watch for out-of-domain text, truncation, anisotropy, and missing normalization; cosine is a relative similarity, not a calibrated or truthful score.

Related:

• Vector Databases: Dimensions, Footprint, and the Migration Nobody Plans For — indexing embeddings for fast approximate nearest-neighbor search.
• Search Systems — embeddings inside a full retrieval + ranking stack.
• Fair Aggregation: Balancing Utility and Fairness — using cosine similarity as a per-person satisfaction score to aggregate.

Resources

• Reimers, N., Gurevych, I. (2019). "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks." EMNLP — https://arxiv.org/abs/1908.10084 (verified arXiv id).
• Cer, D. et al. (2018). "Universal Sentence Encoder." — https://arxiv.org/abs/1803.11175 (verified arXiv id).
• Muennighoff, N. et al. (2023). "MTEB: Massive Text Embedding Benchmark." — the standard leaderboard for comparing embedding models on your task type. (arXiv; verify id before citing.)
• Embedding Projector (visualization): https://projector.tensorflow.org/