Vector Embeddings Explained: A Practical Introduction for Engineers

You have heard that embeddings are at the center of modern AI search. You have seen the word "vector database" in every AI architecture diagram. But if someone asked you to explain what a vector embedding actually is and why it enables semantic search, the explanation might feel slippery.

This post gives you a concrete, engineer-friendly explanation of vector embeddings — what they are, how they are produced, and how they connect to RAG systems and semantic search in practice.

What a Vector Embedding Actually Is

A vector embedding is a list of numbers that represents the meaning of a piece of text. Not the characters, not the tokens — the meaning.

Take the sentence "the server failed to connect to the database." An embedding model maps that sentence to a list of floating-point numbers — say, 768 or 1536 numbers, depending on the model. Every sentence gets mapped to a point in that high-dimensional space. Sentences with similar meaning end up close to each other in that space. Sentences with unrelated meaning end up far apart.

The critical point: "server connection error" and "database host unreachable" will produce embeddings that are geometrically close, even though they share almost no words. That is the property that makes semantic search work and that makes BM25 keyword search fail at the same task.

How Embedding Models Work

Embedding models are transformer neural networks, trained on large corpora of text, that have learned to compress the meaning of text into a fixed-size vector. The training objective — often contrastive learning — pushes similar text pairs to have similar embeddings and dissimilar pairs to have distant embeddings.

The output dimension (768, 1024, 1536, 3072) is a hyperparameter of the model. Larger dimensions generally capture more nuance but cost more to store and compare. Models like OpenAI's text-embedding-3-small use 1536 dimensions. Sentence-transformers models like all-MiniLM-L6-v2 use 384 dimensions and run efficiently on CPU.

Every embedding model has a maximum input length — a context window for the embedding, separate from any LLM context window. Many popular models cap at 512 tokens. If your text is longer than the model's context window, the input is either truncated or must be chunked before embedding.

How Similarity Is Measured

Once you have embeddings, you compare them using a distance or similarity metric:

• Cosine similarity: Measures the angle between two vectors, ignoring magnitude. The standard metric for text embeddings. A cosine similarity of 1.0 means identical direction; 0.0 means orthogonal (unrelated); -1.0 means opposite.
• Dot product: Proportional to cosine similarity when vectors are normalized. Often used as a slightly cheaper approximation.
• Euclidean distance (L2): Measures straight-line distance in the vector space. Less common for text but used in some vision embedding applications.

Most text embedding models are designed for cosine similarity. Normalize your vectors and use dot product for speed when your vector count is large enough to matter.

What a Vector Index Does

Comparing a query embedding to every embedding in your corpus one-by-one is an exact nearest neighbor search. At 1 million vectors it is slow. At 100 million it is unusable.

Vector indexes implement Approximate Nearest Neighbor (ANN) search. The most common algorithm is HNSW (Hierarchical Navigable Small World), which builds a layered graph over your vectors that allows fast traversal to approximate neighbors. You trade a small amount of recall accuracy for orders-of-magnitude improvements in query speed.

This is what vector databases — Pinecone, Qdrant, Weaviate, pgvector — provide: an efficient ANN index plus APIs for inserting, querying, and filtering vectors.

How Vector Embeddings Enable RAG

Retrieval-Augmented Generation uses embeddings at indexing time and query time:

At indexing time, you chunk your documents, embed each chunk, and store the embedding alongside metadata in a vector database.

At query time, you embed the user's question using the same embedding model, query the vector database for the top-K closest chunk embeddings, retrieve the corresponding text chunks, and pass them to an LLM as context for generating the answer.

The embedding model is the bridge between the user's question and the document corpus.