Why cosine similarity and Euclidean distance can disagree

Suppose a vector search returns one result when you use cosine similarity and another when you use Euclidean distance. That can happen even when both calculations are correct. Cosine similarity compares the directions of vectors, while Euclidean distance measures how far apart their endpoints are. A vector can point almost the same way as your query and still end a long way from it.

Normalizing the vectors changes this comparison. Once every nonzero vector has length one, maximizing cosine similarity and minimizing Euclidean distance give the same neighbor ranking. The scores remain different numbers, however, so a threshold from one metric cannot be copied unchanged into the other.

A two-dimensional example makes both facts visible. You can inspect every coordinate before considering the higher-dimensional vectors used in an embedding search.

Two candidates, two different winners

Take the query $q = (1, 0)$ and two candidates, $A = (4, 1)$ and $B = (1, 1)$. Candidate A points mostly to the right, much like the query. Candidate B points diagonally upward but ends closer to the query's endpoint.

Cosine similarity divides the dot product by the product of the vector lengths:

$$\operatorname{cos}(q,v) = \frac{q \cdot v}{\lVert q \rVert\lVert v \rVert}.$$

This is the normalized dot product used by scikit-learn. A larger score means a smaller angle between the vectors. For our query, whose length is one, the calculations are

$$\begin{aligned} \operatorname{cos}(q,A) &= \frac{4}{\sqrt{17}} \approx 0.9701,\\ \operatorname{cos}(q,B) &= \frac{1}{\sqrt{2}} \approx 0.7071. \end{aligned}$$

Cosine similarity therefore selects A. Its greater length does not penalize it, because the denominator removes the effect of length.

Euclidean distance uses the coordinate differences instead. In two dimensions, $d(q,v) = \sqrt{(q_1-v_1)^2+(q_2-v_2)^2}$. The same definition extends by summing over all coordinates, as in scikit-learn's Euclidean distance documentation. Here it gives

$$\begin{aligned} d(q,A) &= \sqrt{(1-4)^2+(0-1)^2} = \sqrt{10} \approx 3.1623,\\ d(q,B) &= \sqrt{(1-1)^2+(0-1)^2} = 1. \end{aligned}$$

Distance is better when it is smaller, so this calculation selects B.

Neither result tells us, by itself, which candidate is more useful. That depends on what the coordinates represent and what we want “similar” to mean. The example establishes why the metrics disagree; it is not evidence that one metric retrieves better documents.

Move each vector onto the unit circle

To normalize a nonzero vector, divide it by its length. The direction stays the same, while the length becomes one. Our query already has unit length. The candidates become

$$\begin{aligned} \hat A &= \left(\frac{4}{\sqrt{17}},\frac{1}{\sqrt{17}}\right) \approx (0.9701, 0.2425),\\ \hat B &= \left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}}\right) \approx (0.7071, 0.7071). \end{aligned}$$

A hat marks a normalized vector here; it does not denote a prediction as it sometimes does elsewhere in machine learning.

Figure link ↗SVG ↓PNG ↓

The cosine scores have not changed because the directions have not changed. The Euclidean distances have: A is now about $0.2444$ from the query, and B is about $0.7654$ away. A wins under both metrics.

The diagram explains the reversal. Normalizing A pulls its endpoint much closer to the query. All three endpoints now lie on the same circle, so the smaller angle also corresponds to the shorter straight-line segment between endpoints.

Why the rankings agree after normalization

Expand the squared distance between two unit vectors $\hat q$ and $\hat v$:

$$\begin{aligned} \lVert \hat q-\hat v \rVert^2 &= \lVert \hat q \rVert^2 + \lVert \hat v \rVert^2 - 2\hat q \cdot \hat v\\ &= 2 - 2\operatorname{cos}(q,v). \end{aligned}$$

The first two terms are each one because both vectors have unit length. Their dot product equals their cosine similarity. As that similarity increases, the squared distance decreases; taking the square root preserves the distance ordering. Equal cosine scores also give equal distances.

This is the relationship documented in the Faiss guide to metrics and distances. The guide uses it to connect cosine search with normalized inner-product and L2 search. For the displayed identity, normalize both the stored vectors and the query.

The equivalence concerns exact scores and their ordering. An approximate search implementation can still miss a neighbor, and floating-point rounding can affect nearly tied results. Normalization does not promise that two independently configured indexes will return identical lists.

Similarity, cosine distance, and squared distance have different scales

You may encounter three related quantities in an API: cosine similarity, cosine distance defined as $1-\operatorname{cos}(q,v)$, and squared Euclidean distance. On unit vectors, the last is twice the cosine distance. Ordinary Euclidean distance is the square root of that quantity.

For example, a cosine similarity of $0.8$ corresponds to cosine distance $0.2$, squared Euclidean distance $0.4$, and Euclidean distance about $0.6325$ after unit normalization. These numbers describe the same comparison, but applying a cutoff of $0.8$ to each would select different results. Faiss specifically reports squared distance for its L2 metric, which matters when interpreting a returned score.

There is also a boundary case before any of this works: a zero vector has no direction. The mathematical cosine formula divides by zero for such a vector. Check how your chosen library handles it and decide what an empty or zero representation should mean in your application, rather than treating its returned score as an ordinary angular comparison.

Check the numbers without a vector database

This Python example uses only the standard library and prints both the raw and normalized distances. It rejects zero vectors explicitly so that an undefined cosine cannot silently enter the results.

from math import sqrt

def dot(a, b):
    if len(a) != len(b):
        raise ValueError("Vectors must have the same dimension")
    return sum(x * y for x, y in zip(a, b))

def normalize(v):
    length = sqrt(dot(v, v))
    if length == 0:
        raise ValueError("A zero vector has no direction")
    return tuple(x / length for x in v)

def distance(a, b):
    if len(a) != len(b):
        raise ValueError("Vectors must have the same dimension")
    return sqrt(sum((x - y) ** 2 for x, y in zip(a, b)))

q = (1.0, 0.0)
for name, v in [("A", (4.0, 1.0)), ("B", (1.0, 1.0))]:
    uq, uv = normalize(q), normalize(v)
    similarity = dot(uq, uv)
    raw_distance = distance(q, v)
    unit_distance = distance(uq, uv)
    print(f"{name}: cosine={similarity:.4f}, "
          f"raw={raw_distance:.4f}, unit={unit_distance:.4f}")

The output is A: cosine=0.9701, raw=3.1623, unit=0.2444 and B: cosine=0.7071, raw=1.0000, unit=0.7654.

For an embedding search, a practical next step is to inspect the model's documentation and whether its outputs are already normalized. Then test the intended configuration against queries with known relevant results. If you are considering removing vector length, ask what information that length carries in your representation. The raw example above shows exactly how discarding it can change the result.

For a closer look at length's role in similarity scores, our dot product and cosine similarity article keeps the direction fixed and scales one vector. That isolates the part of the comparison that normalization removes.