What Are Embeddings

An embedding is a list of numbers (a vector) that represents a piece of text, an image, or any item so that similar things get similar vectors. That lets you compute "how alike are these two things" with simple math, which is the foundation of semantic search, recommendations, and RAG.

How Do Embeddings Capture Similarity?

An embedding model is trained so that items appearing in similar contexts land close together in a high-dimensional space — typically 384 to 3072 dimensions. "How do I reset my password" and "I forgot my login" share almost no words, but their embeddings sit close together because they mean the same thing.

What Is Cosine Similarity?

Cosine similarity measures the angle between two vectors: 1 means pointing the same direction (very similar), 0 means unrelated, and it ignores vector length so long and short texts compare fairly.

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Toy 3-dim example (real embeddings have hundreds of dims)
password_reset = np.array([0.8, 0.1, 0.2])
forgot_login   = np.array([0.7, 0.2, 0.3])
pizza_recipe   = np.array([0.1, 0.9, 0.1])

cosine_similarity(password_reset, forgot_login)  # 0.98 -> same topic
cosine_similarity(password_reset, pizza_recipe)  # 0.25 -> unrelated

What Are Embeddings Used For?

Anywhere "find similar things" is the core operation, embeddings are the workhorse.

• Semantic search: embed the query and every document, return the closest documents — matches meaning, not keywords.
• Recommendations: "users who liked this" becomes "items whose vectors are near this one."
• RAG: retrieve the most relevant chunks to feed an LLM as context.
• Clustering and deduplication: group nearby vectors with k-means to find topics or near-duplicate records.

How Do You Get Embeddings in Practice?

Call an embedding API or run an open-source model locally; either way you get one vector per input, which you store in a vector database or a plain numpy array for small datasets.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")  # free, runs locally
docs = ["Reset your password in Settings", "Best pizza dough recipe"]
vectors = model.encode(docs)   # shape: (2, 384)

query = model.encode("forgot my login")
scores = vectors @ query / (np.linalg.norm(vectors, axis=1) * np.linalg.norm(query))
# scores[0] >> scores[1] -> the password doc wins

Common Pitfalls

• Mixing models: vectors from different embedding models live in different spaces and cannot be compared. Re-embed everything when you switch models.
• Embedding huge texts whole: long documents blur into a vague average vector. Split them into chunks first.
• Assuming similarity = relevance: "refund policy" and "no refunds allowed" embed close together. Similarity finds topic, not stance.
• Skipping normalization: many libraries expect unit-length vectors before dot-product search; check your model's docs.