# Hybrid Search in RAG: How BM25 and Dense Vector Retrieval Work Together > A deep dive into hybrid search for RAG — how BM25 and dense vector retrieval complement each other, how rank fusion works, and when to use hybrid over pure vector search. - **Published:** 2025-11-07 - **Author:** Clixo - **Reading time:** 5 min read - **Tags:** hybrid-search, bm25, vector-search, rag, retrieval - **Canonical URL:** https://clixo.sh/blog/hybrid-search-bm25-dense-vector-rag Dense vector search is powerful, but it has a quiet failure mode: it treats all queries as semantic paraphrase problems. Ask for "Kubernetes node affinity configuration" and it retrieves conceptually related content rather than documents that contain the exact term. If your users ask about product codes, error messages, API method names, or proper nouns, pure vector search will routinely surface wrong documents while appearing to work. Hybrid search — combining BM25 keyword retrieval with dense vector retrieval — is the architectural decision that separates retrieval systems that hold up under production load from those that look good in demos and fail in practice. ## What Hybrid Search for RAG Actually Means Hybrid search is not a single technique; it is a retrieval architecture that runs two retrieval passes in parallel and merges the results. The first pass is lexical. BM25 is a term-frequency–based ranking algorithm that scores documents by how often query terms appear in each document, adjusted for document length. It is exact: if a document contains the token `ERR_CONNECTION_TIMED_OUT`, BM25 will find it. Dense search may not, depending on how well that string is represented in embedding space. The second pass is semantic. Dense vector search embeds the query using a neural model and retrieves documents whose embeddings are closest in the vector space. This captures conceptual similarity, synonyms, and paraphrases that BM25 misses entirely. The merged result set is then re-ranked using a fusion algorithm that combines scores from both passes into a single ranked list. ```mermaid flowchart LR Q["User query"] --> BM["BM25 keyword search"] Q --> VEC["Dense vector search"] BM --> RRF["RRF rank fusion"] VEC --> RRF RRF --> RR["Cross-encoder reranker"] RR --> R["Top results for LLM"] ``` ## How Reciprocal Rank Fusion Works The standard merging approach is Reciprocal Rank Fusion (RRF). The idea is straightforward: instead of combining raw scores (which are on incomparable scales between BM25 and cosine similarity), each retriever assigns a rank to each document. The fused score for a document is the sum of `1 / (rank + k)` across both retrievers, where `k` is a smoothing constant (typically 60). Documents that rank well in both retrievers bubble to the top. Documents that only appear in one retriever still get credit. The result is a list that is more robust than either retriever alone. Score normalization is an alternative: normalize BM25 and cosine similarity scores to the same range, then take a weighted sum. This requires tuning the weight parameter per corpus and per query distribution, which makes RRF preferable as a starting point — it has no trainable parameters. ## When Hybrid Search Outperforms Pure Vector Search Hybrid search consistently outperforms pure dense retrieval in these scenarios: - **Exact identifiers**: Error codes, SKUs, version numbers, API endpoints, ticker symbols. BM25 finds these reliably; vector search often fails to distinguish `v2.3.1` from `v2.3.2`. - **Rare technical terms**: Terms that appear infrequently in embedding model training data may not have well-calibrated vector representations. - **Short queries**: A two-word query gives dense models little semantic signal. BM25 performs proportionally better on short inputs. - **Multi-domain corpora**: When your index covers both technical documentation and conversational support content, BM25 anchors retrieval to the specific domain a user is querying. Conversely, pure dense retrieval outperforms BM25 when: - Users ask questions in different words than the documents use - Documents are in a different language than the query - The corpus has heavy jargon that BM25 would misweight In practice, hybrid almost always matches or beats either retriever alone, which makes it the sensible default. ## Implementing Hybrid Search in Practice Most modern vector databases expose hybrid search natively. In Weaviate, you set the `alpha` parameter to blend BM25 and vector search. In Qdrant, you run sparse and dense searches separately and fuse the results. In Elasticsearch and OpenSearch, the `knn` query can be combined with BM25 in a `bool` query with weighted scoring. The operational steps: 1. **Index sparse vectors alongside dense vectors.** For BM25, this means your database needs to maintain an inverted index of tokens. For Pinecone and Qdrant, sparse vectors are explicitly stored alongside dense vectors. For Weaviate, the BM25 index is built automatically. 2. **Choose a fusion strategy.** RRF for most cases. Score normalization if you have enough query volume to tune the alpha hyperparameter against an evaluation set. 3. **Add a reranker.** Hybrid search improves recall. A cross-encoder reranker improves precision. The combination — hybrid retrieval plus reranking — is the current production-quality baseline for RAG retrieval. ## Sparse Embeddings vs. BM25 A newer variant of hybrid search uses learned sparse embeddings (SPLADE, SPLADE++) instead of traditional BM25. SPLADE expands queries and documents with semantically related terms from the vocabulary of a transformer model, producing sparse vectors that combine the recall of keyword search with some of the semantic awareness of dense models. SPLADE tends to outperform raw BM25, especially on technical domains. The trade-off is inference cost at index time — you need to run a transformer model over every document to produce the sparse representation. For most teams, standard BM25 hybrid search is the right starting point. Move to learned sparse embeddings when you have measured a specific BM25 weakness and have the inference budget. ## The Practical Recommendation Do not deploy a production RAG system on pure dense retrieval if your users will query by exact terms, identifiers, or technical strings. Hybrid search is not an optimization — it is the baseline retrieval architecture that handles the full range of real user queries. Start with RRF fusion, add a reranker, and measure Recall@5 and MRR against your labeled evaluation set. The numbers will make the argument for hybrid search more clearly than any benchmark paper. If you are building a search or RAG system and want it architected correctly from the start, [talk to the Clixo team](https://clixo.sh/#contact). We design and ship retrieval systems for product teams. --- Clixo · 1141 W Bryn Mawr Ave, Itasca, IL 60143, US · [hello@clixo.sh](mailto:hello@clixo.sh) [Start a build](https://clixo.sh/#contact) · [All services](https://clixo.sh/services) · [Agent guide (llms.txt)](https://clixo.sh/llms.txt)