Why Cosine Similarity Powers Semantic Search

Keyword retrieval scores documents on lexical overlap. It is fast, interpretable, and blind in a specific way: "How do I reset my password?" and "Steps to recover account access" share almost no terms and nearly all of their meaning. Term-based scoring has no mechanism to see that, because its only evidence is which strings coincide.

Semantic search replaces string identity with a representation. Each piece of text is mapped to a point in a continuous space, chosen so that texts meaning similar things land in similar places. Comparison then becomes arithmetic on coordinates rather than matching on tokens, and the question "do these mean the same thing?" is answered by a proxy: "are these two vectors aligned?"

That substitution is the whole subject of this paper. It proceeds in one chain:

flowchart LR
  L["Language
query, document"] --> M["Embedding model
trained encoder"]
  M --> V["Vector representation
fixed dimension d"]
  V --> G["Semantic relations become
geometric relations"]
  G --> C["Cosine similarity
directional alignment"]
  C --> S["Relevance score"]
  S --> R["Ranking"]
  R --> I["Vector index
scalable retrieval"]
  style M fill:#e8f1ff,stroke:#3975c6
  style C fill:#e8f7f1,stroke:#157a5a,stroke-width:2px
  style G fill:#fff7e6,stroke:#9a6500

2.1 Four things that are not the same

The word "embedding" is used for at least four distinct objects. Conflating them is the single most common source of confusion about why cosine sometimes works and sometimes does not.

Only the last is what a vector database holds. It is an activation, not a parameter: it exists only because a specific text was passed through the model.

2.2 The pipeline

flowchart TB
  T["Raw text"] --> TK["Tokenizer
subword units"]
  TK --> LU["Static token vectors
lookup — context-free"]
  LU --> EN["Transformer layers
attention + feed-forward"]
  EN --> HS["Hidden states
one vector per token"]
  HS --> PO["Pooling
mean / CLS / last-token"]
  PO --> NM["L2 normalization
optional but common"]
  NM --> EM["Stored embedding
d numbers"]
  style EN fill:#e8f1ff,stroke:#3975c6
  style PO fill:#fff7e6,stroke:#9a6500
  style EM fill:#e8f7f1,stroke:#157a5a,stroke-width:2px

2.3 Contextualization is what makes the geometry possible

The lookup step assigns one vector per token regardless of context, so bank in "river bank" and "savings bank" begin identically. The transformer layers fix this: each token's representation is recomputed as a function of the other tokens it is permitted to attend to. By the upper layers, the two bank tokens have been pulled apart.

2.4 Pooling, and why the choice is not cosmetic

Pooling must turn \(n\) vectors into one. Mean pooling averages all token states; CLS pooling reads a designated position; last-token pooling reads the final position — the natural choice for a causal model, since it is the only position that has seen the whole input.

Each rule discards different information, and none is universally correct. What matters is consistency with training: a model fine-tuned with mean pooling in the loss will produce a usable space under mean pooling and a degraded one under CLS pooling. The pooling rule is part of the model's contract, not a post-hoc convenience.

2.5 The misconception that costs the most

2.6 In one sentence

An embedding is a pooled activation of a trained encoder, of a dimension fixed in advance, whose usefulness for angular comparison is a consequence of the objective that shaped it.

3.1 Two independent pieces of information

A vector \(a=(a_1,\dots,a_d)\) carries exactly two separable things: how long it is and which way it points. Formally, \(a = \lVert a\rVert \cdot \hat a\), where

The decomposition is unique for any \(a\neq 0\). Every similarity function is, in effect, a choice about how much weight to give each part.

3.2 The inner product, and what it already knows about angle

The dot product is a sum of coordinate-wise products:

It is large when the two vectors agree in sign and magnitude coordinate by coordinate, and negative when they systematically disagree. It also carries angular information already — but entangled with length, since scaling \(a\) by \(5\) scales \(a\cdot b\) by \(5\) without rotating anything. Disentangling the two is precisely what cosine does.

3.3 Euclidean distance is magnitude-sensitive

Take \(A=(2,3,5)\) and \(B=(4,6,10)=2A\). These point in exactly the same direction — the angle between them is \(0\). Yet

Raw Euclidean distance calls them far apart purely because one is twice as long. If the application treats \(A\) and \(2A\) as meaning the same thing, this is the wrong comparison.

4.1 The definition, and where it comes from

For nonzero \(a,b\in\mathbb{R}^d\),

This is not a definition invented for search; it is forced by the law of cosines. For the triangle with sides \(a\), \(b\) and \(a-b\):

So the division by two lengths is not a normalization applied to the dot product as an afterthought. It is what the dot product was always missing in order to be an angle. Assumption: \(a\neq 0\) and \(b\neq 0\). The zero vector has no direction and cosine is undefined for it — an edge case worth handling explicitly in code, since a zero-length embedding silently produces \(0/0\).

4.2 Why the value is bounded in \([-1,+1]\)

The bound is not empirical. It is the Cauchy–Schwarz inequality, which states that for any \(a,b\in\mathbb{R}^d\),

This gives the three landmark values a precise geometric reading:

4.3 The interactive figure: drag the vectors

Everything above is two-dimensional and visible. Drag either arrowhead. Watch which readouts change and which refuse to. The green dashed arrow is the projection of \(a\) onto \(b\) — the geometric object the dot product actually measures.

4.4 Normalization: the same comparison, pre-paid

If every vector is scaled to unit length before storage, then \(\lVert \hat a\rVert=\lVert \hat b\rVert=1\) and

Cosine similarity on raw vectors is identical to the inner product on normalized vectors. The division has not vanished; it has been paid once at indexing time instead of once per comparison at query time. This identity is the reason §7 works at all.

flowchart LR
  subgraph RAW["Raw space — length varies freely"]
    A1["a = (4,1)
‖a‖ = 4.12"]
    B1["b = (−1,3)
‖b‖ = 3.16"]
  end
  RAW -->|"divide each by its norm"| SPH
  subgraph SPH["Unit sphere — only direction survives"]
    A2["â
‖â‖ = 1"]
    B2["b̂
‖b̂‖ = 1"]
  end
  SPH --> DP["Inner product â · b̂
= cos θ, no division"]
  style SPH fill:#e8f7f1,stroke:#157a5a
  style DP fill:#fff7e6,stroke:#9a6500,stroke-width:2px

4.5 Ranking by cosine and by Euclidean distance are the same ranking

On normalized vectors, expand the squared distance:

So \(\lVert \hat a-\hat b\rVert=\sqrt{2-2\cos\theta}\), a strictly decreasing function of \(\cos\theta\) on \([-1,1]\). Sorting ascending by Euclidean distance and sorting descending by cosine therefore produce identical orderings — not similar ones, identical.

4.6 Cosine distance is not a metric

It is tempting to treat \(1-\cos\theta\) as a distance. It is non-negative and zero exactly when vectors are parallel, but it violates the triangle inequality. Take \(x=(1,0)\), \(y=(\tfrac{1}{\sqrt2},\tfrac{1}{\sqrt2})\), \(z=(0,1)\):

Since \(1 > 0.586\), the inequality fails. The angular distance \(\theta/\pi\) is a true metric (here \(0.5 \le 0.25+0.25\), holding with equality), and some systems use it for that reason. This matters when an algorithm's correctness depends on metric axioms; for pure top-\(k\) ranking it does not, since ranking needs only monotonicity.

5.1 The question stated honestly

Section 4 established what \(\cos\theta\) is: a bounded, scale-invariant measure of directional alignment, derived from the law of cosines. Every result there holds for arbitrary vectors — stock prices, pixel intensities, random noise. None of it mentions language, and none of it could, because cosine similarity is a function of two arrays of numbers and nothing else.

So why should the angle between two embedding vectors have anything to do with meaning?

5.2 What the training objective actually arranges

A representation space in which angle tracks meaning does not arise by accident. It arises because an objective function penalised the alternative. The widely used family is contrastive: given an anchor text \(q\), a semantically related positive \(p\), and negatives \(n_i\), the loss rewards high similarity for \((q,p)\) and low similarity for \((q,n_i)\). In the common InfoNCE form, with similarity function \(s\) and temperature \(\tau\):

The decisive detail is that \(s\) is typically cosine similarity itself. Gradient descent is therefore directly rewarded for arranging directions so that cosine reads high on related pairs and low on unrelated ones. The geometry is not a byproduct of training — it is the thing trained.

5.3 When cosine is not meaningful: the scaling degrees of freedom

There is a precise and important failure case, and it is the strongest argument for §5.2's rule. Steck, Ekanadham and Kallus (2024) analyse linear matrix-factorization models and show the following. Suppose embedding matrices \(A\) and \(B\) are learned such that reconstruction depends only on the product \(AB^\top\). Then for any invertible diagonal matrix \(D\), the pair \((AD,\,BD^{-1})\) is an equally optimal solution, because

The inner product is invariant under this transformation. Cosine similarity is not: \(D\) rescales individual coordinates, which changes both norms and the angle between rows. So whenever the training objective constrains only \(AB^\top\), the per-coordinate scaling is unidentified, and cosine similarity computed on the resulting embeddings is arbitrary — a function of an optimisation accident rather than of the data.

5.4 Anisotropy: the space is not uniformly occupied

One more model-dependent property shapes how cosine scores read. Ethayarajh (2019) measured the geometry of contextual representations in ELMo, BERT and GPT-2 and found them highly anisotropic: rather than spreading over the sphere, vectors occupy a narrow cone, so even randomly chosen unrelated words exhibit substantial positive cosine similarity. In the upper layers of GPT-2 the effect was most extreme, with average similarity between random words approaching \(1\). To measure genuine contextuality the study had to subtract this baseline.

Two consequences, stated carefully:

• Absolute cosine values are not interpretable across models. A score of \(0.8\) means one thing in a near-isotropic space and something quite different in a narrow cone. Only relative ordering within one space is reliable without calibration.
• Anisotropy is a property of a specific model and layer, not a universal constant. It was measured in named models under stated conditions. Modern retrieval-tuned encoders are frequently trained in ways that reduce it, and quoting a fixed numeric threshold as though it applied to all embeddings would be unsupported.

5.5 What random-vector geometry does and does not explain

There is a genuine mathematical fact about high dimension. For two vectors drawn uniformly at random from the unit sphere in \(\mathbb{R}^d\), the cosine of the angle between them has mean \(0\) and variance \(1/d\), so its standard deviation is \(1/\sqrt{d}\). At \(d=768\) that is \(1/\sqrt{768}\approx 0.036\): random directions are nearly orthogonal, and sharply concentrated about orthogonality.

5.6 Small angular gaps do not imply weak semantic distinctions

Converting two cosine scores to angles is straightforward. For \(0.92\) and \(0.88\):

a gap of roughly \(5.3^\circ\).

6.1 The whole mechanism

Retrieval requires an ordering over documents, and an ordering requires a single comparable number per document. Cosine supplies exactly that, and its properties are what make the ordering well-defined:

1. Encode each document \(D_j\) once, offline: \(v_j = \text{pool}(\text{encode}(D_j))\), normalized.
2. At query time encode the query with the same model: \(u = \text{pool}(\text{encode}(Q))\), normalized.
3. Score every document: \(s_j = u\cdot v_j = \cos\theta_j\).
4. Sort descending and return the top \(k\).

flowchart LR
  Q["Query text"] --> QE["Encoder"] --> QV["u — query vector"]
  D["Document corpus"] --> DE["Same encoder
offline, once"] --> DV["v₁ … v_N — stored"]
  QV --> S["cos(u, v_j)
one scalar per document"]
  DV --> S
  S --> R["Sort descending"] --> K["Top-k results"]
  style S fill:#fff7e6,stroke:#9a6500,stroke-width:2px
  style K fill:#e8f7f1,stroke:#157a5a

6.2 Which properties of cosine the ranking actually relies on

6.3 The ranking is relative, not absolute

Cosine turns a learned representation into a relative relevance signal. It reliably says that document \(A\) is more aligned with the query than document \(B\). It does not, without calibration, say that \(A\) is relevant — the same score may indicate a good match in one model's space and a mediocre one in another's (§5.4). Threshold-based filtering therefore requires per-model calibration against labelled data; top-\(k\) ranking does not.

7.1 The identity that cheapens everything

Section 4.4 established \(\hat a\cdot\hat b=\cos\theta\). Normalize once at indexing time and every subsequent comparison is a bare inner product: \(d\) multiplications and \(d-1\) additions, no square roots, no division. This is the single most consequential practical fact about cosine similarity — it is a dot product in disguise, and dot products are what hardware and BLAS libraries are built to do in bulk.

Exhaustive scoring of \(N\) documents of dimension \(d\) costs \(\Theta(Nd)\) arithmetic operations per query — linear in corpus size. At large \(N\) that is the thing to avoid, which is what ANN indexes are for.

7.2 How vector databases actually expose cosine

7.3 ANN indexing, in the role it belongs in

Exhaustive search is exact but linear in \(N\). Approximate nearest-neighbour indexes trade a controllable amount of recall for a large reduction in the number of comparisons. HNSW — a multi-layer proximity graph in which search descends from sparse upper layers to dense lower ones, greedily following whichever neighbour improves the similarity score — is among the most widely deployed. Malkov and Yashunin report search cost scaling logarithmically with corpus size under their experimental conditions.

flowchart TB
  EP["Entry point
sparse top layer"] --> L2["Upper layers
long-range hops, coarse approach"]
  L2 --> L1["Lower layers
short hops, local refinement"]
  L1 --> CAND["Candidate pool
size controlled by ef"]
  CAND --> OUT["Top-k by cosine"]
  style EP fill:#e8f1ff,stroke:#3975c6
  style CAND fill:#fff7e6,stroke:#9a6500
  style OUT fill:#e8f7f1,stroke:#157a5a,stroke-width:2px

Two parameters express the tradeoff. \(M\) governs graph connectivity at build time; \(\mathit{ef}\) governs how many candidates the search keeps in flight. Raising either increases recall and cost. The right values are found by measuring recall against exhaustive search on representative data — not by quoting defaults.

8.1 The architectural constraint

In the §6.1 design, \(v_j\) is computed before any query exists. This is precisely what makes precomputation and ANN indexing possible — and it imposes a hard constraint:

8.2 What follows from it

• A fixed-capacity bottleneck. A passage containing several distinct claims must be represented by one direction. Aspects of it that no training signal emphasised may not be recoverable, whatever metric is applied afterwards.
• No token-level interaction at scoring time. Fine-grained correspondence between a query phrase and a specific document clause is not available to a single dot product between pooled vectors.
• Query-dependent emphasis is impossible. If relevance hinges on a minor detail of a long passage, the pooled vector may have given that detail little weight.

8.3 Negation, stated carefully

8.4 The cross-encoder: dropping the independence property

If the constraint is independent encoding, the remedy is joint encoding. A cross-encoder takes the concatenated pair \((Q, D_j)\) as a single input and emits a relevance score directly. Attention spans both texts, so every document token is read in the presence of the query.

flowchart TB
  subgraph BI["Bi-encoder — independent encoding"]
    BQ["Query"] --> BQE["Encoder"] --> BQV["u"]
    BD["Document"] --> BDE["Encoder
offline"] --> BDV["v"]
    BQV --> BS["cos(u,v)"]
    BDV --> BS
  end
  subgraph CE["Cross-encoder — joint encoding"]
    CQ["Query + Document
concatenated"] --> CM["Transformer
attention spans both"] --> CS["Relevance score"]
  end
  BS --> N1["Precomputable · ANN-indexable · O(d) per comparison"]
  CS --> N2["Not precomputable · one forward pass per pair"]
  style BS fill:#e8f1ff,stroke:#3975c6
  style CS fill:#f4eefc,stroke:#7b52b9
  style N1 fill:#e8f7f1,stroke:#157a5a
  style N2 fill:#fff0f0,stroke:#a93636

The cost asymmetry is severe and is the reason the architectures coexist. Reimers and Gurevych (2019), introducing Sentence-BERT, report that finding the most similar pair among 10,000 sentences with BERT used as a cross-encoder takes roughly 65 hours on their hardware, because all \(\binom{10000}{2}\approx 5\times10^{7}\) pairs must be scored individually. The same task with SBERT embeddings takes about 5 seconds to encode, after which cosine comparison is near-instantaneous. That contrast is the entire economic case for embedding-based retrieval.

8.5 Reranking as an extension, not a replacement

Combining the two is straightforward and follows directly from their cost structures. Stage one uses cosine over an ANN index to reduce \(N\) documents to \(k\) candidates at a cost that does not grow with \(k\). Stage two applies the cross-encoder to those \(k\) pairs at a cost linear in \(k\) and independent of \(N\).

flowchart LR
  N["Corpus — N documents"] --> S1["Stage 1 · cosine + ANN
cheap, recall-oriented"]
  S1 --> C["k candidates
k ≪ N"]
  C --> S2["Stage 2 · cross-encoder
expensive, precision-oriented"]
  S2 --> F["Reordered top results"]
  style S1 fill:#e8f1ff,stroke:#3975c6
  style S2 fill:#f4eefc,stroke:#7b52b9
  style F fill:#e8f7f1,stroke:#157a5a,stroke-width:2px

8.6 Late interaction: the middle of the spectrum

The two architectures are endpoints of a spectrum, and it is worth seeing the middle. Late interaction approaches such as ColBERT store a vector per token rather than one per document, then compute similarity by matching query tokens against document tokens at query time. This recovers some token-level interaction while keeping document encoding query-independent and therefore precomputable. The cost is storage and a more complex retrieval procedure.

8.7 Downstream: retrieval-augmented generation

One application deserves a short mention because it is where retrieval quality becomes visible. In RAG, semantic retrieval selects the context supplied to a generative model. The retrieval stage determines what the generator is given; context that was never retrieved cannot inform the output. Weak retrieval therefore constrains achievable generation quality.

9.1 The alternatives

9.2 Why cosine is nonetheless so common

Not because it is superior in general, but because several desirable properties coincide in it:

And decisively: it is the function most embedding models for retrieval are trained with (§5.2). Using at inference the similarity the objective optimised is not a stylistic preference but a correctness consideration — §5.3 shows what can go wrong otherwise. Where magnitude carries genuine learned information, the unnormalized inner product may be the better choice, and normalizing would discard signal.

Cosine similarity occupies a precise position in semantic search, and most confusion about it comes from assigning it a different one. It is not a semantic operator. It is not an approximation to understanding. It is a bounded, scale-invariant measurement of angular alignment, derived in §4 from the law of cosines and carrying no knowledge of language whatsoever.

What makes it indispensable is what sits beneath it. An embedding model, trained under an objective that rewards similarity structure, produces a space in which semantic relationships have become geometric ones. In that space — and only in that space — measuring an angle recovers something about meaning. The metric contributes rigour and cheapness; the model contributes the semantics.

Each layer depends on the one above and cannot substitute for it. A better metric cannot repair a poorly trained space. A larger index cannot create relevance that the representation failed to encode. A reranker cannot retrieve what the first stage never surfaced. And cosine similarity — the single division that this entire architecture rests on — works not because it is clever, but because a model was trained to make it informative.