Hybrid Search: BM25 + Dense Retrieval (2026) — Reciprocal Rank Fusion + Reranking

The BEIR benchmark (Thakur et al., 2021, arxiv.org/abs/2104.08663) is the standard evaluation suite for information retrieval. It spans 18 datasets across diverse domains — biomedical (TREC-COVID, NFCorpus), legal (FiQA), code (CodeSearchNet), news (TREC-NEWS), and more. The metric is NDCG@10 (normalized discounted cumulative gain at 10 results): higher is better, range 0-1, meaningful differences are >0.01.

On BEIR, hybrid BM25 + dense retrieval outperforms either approach alone by 5-12 NDCG@10 points on most datasets. The intuition: TREC-COVID queries like 'what is the effect of COVID-19 on hemoglobin levels' benefit from both BM25 (exact term 'hemoglobin') and dense retrieval (semantic similarity to COVID pathophysiology). FiQA financial queries benefit from BM25 on exact ticker symbols and dense retrieval on conceptual questions.

Published results on BEIR from Formal et al. (2021, SPLADE paper) and subsequent work show that SPLADE (a sparse learned model that approximates BM25 with neural expansion) + dense retrieval achieves state-of-the-art on 12 of 18 BEIR datasets. In production systems without SPLADE infrastructure, classical BM25 + dense hybrid with RRF recovers most of the gain — typically within 1-3 NDCG@10 points of SPLADE hybrid at a fraction of the operational complexity.

Practical evidence from production teams (Cohere blog, Pinecone blog, Elastic blog — all 2024-2026): hybrid consistently outperforms dense-only on enterprise RAG datasets, particularly for: (1) product catalogs with SKUs and model numbers; (2) technical documentation with function names and error codes; (3) legal/compliance documents with specific clause references; (4) medical records with procedure codes. Dense-only is competitive on general QA over natural language corpora without specialized terminology.

RRF is retrieval-system-agnostic. It takes ranked lists from any number of retrieval systems and computes a fused score. The only parameter is `k` (default 60), which controls how much weight is given to top-ranked vs lower-ranked results. Higher k makes the fusion smoother (less dominant top ranks); lower k amplifies top-rank differences.

```python from typing import Any def reciprocal_rank_fusion( ranked_lists: list[list[str]], k: int = 60, ) -> list[tuple[str, float]]: """ Fuse multiple ranked document ID lists using Reciprocal Rank Fusion. Args: ranked_lists: List of lists, each containing document IDs in rank order (index 0 = best match). k: Smoothing constant. Standard choice is 60 (Cormack et al., 2009). Returns: List of (doc_id, rrf_score) tuples, sorted descending by score. """ scores: dict[str, float] = {} for ranked_list in ranked_lists: for rank, doc_id in enumerate(ranked_list): scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank + 1) return sorted(scores.items(), key=lambda x: x[1], reverse=True) ```

Example usage with two ranked lists:

```python dense_results = ["doc_A", "doc_B", "doc_C", "doc_D", "doc_E"] bm25_results = ["doc_C", "doc_F", "doc_A", "doc_G", "doc_H"] fused = reciprocal_rank_fusion([dense_results, bm25_results], k=60) # fused = [ # ('doc_A', 1/61 + 1/63), # rank 0 in dense + rank 2 in BM25 # ('doc_C', 1/63 + 1/61), # rank 2 in dense + rank 0 in BM25 # ('doc_B', 1/62), # rank 1 in dense only # ('doc_F', 1/62), # rank 1 in BM25 only # ... # ] print(fused[:3]) ```

The constant `k+1` in the denominator (not `k + rank` as sometimes misquoted) is the standard from the original paper. The `+1` ensures rank 0 gets score `1/(k+1)`, not infinity at rank 0 with k=0. For k=60 and rank 0: score = 1/61 = 0.0164. For rank 50: score = 1/111 = 0.009. The top rank is worth about 1.8x the 50th rank — a soft weighting that prevents top-rank tyranny while still rewarding consensus top results.

Three retrieval systems can be fused with the same function:

```python # Example: dense + BM25 + sparse-learned (SPLADE) fused = reciprocal_rank_fusion( [dense_results, bm25_results, splade_results], k=60 ) ```

This phase builds a complete hybrid pipeline: Elasticsearch for BM25, Qdrant for dense retrieval, parallel execution, RRF fusion, and content retrieval by fused IDs.

```bash pip install elasticsearch==8.14.0 qdrant-client==1.9.0 openai==1.50.0 anthropic==0.40.0 ```

```python import os import asyncio from elasticsearch import AsyncElasticsearch from qdrant_client import AsyncQdrantClient from qdrant_client.models import Distance, VectorParams, PointStruct from openai import OpenAI oai = OpenAI(api_key=os.environ["OPENAI_API_KEY"]) es = AsyncElasticsearch( os.environ.get("ELASTICSEARCH_URL", "http://localhost:9200") ) qd = AsyncQdrantClient( url=os.environ.get("QDRANT_URL", "http://localhost:6333") ) INDEX_NAME = "rag_docs" COLLECTION = "rag_docs" EMBED_MODEL = "text-embedding-3-small" DIM = 1536 ```

Index setup for both backends:

```python async def setup_backends(): # Elasticsearch index with BM25 (default similarity) if not await es.indices.exists(index=INDEX_NAME): await es.indices.create( index=INDEX_NAME, body={ "mappings": { "properties": { "doc_id": {"type": "keyword"}, "content": {"type": "text", "analyzer": "english"}, "metadata": {"type": "object"}, } }, "settings": { "similarity": { "default": { "type": "BM25", "b": 0.75, # length normalization "k1": 1.2, # term-frequency saturation } } } } ) # Qdrant collection for dense vectors existing = [c.name for c in (await qd.get_collections()).collections] if COLLECTION not in existing: await qd.create_collection( collection_name=COLLECTION, vectors_config=VectorParams(size=DIM, distance=Distance.COSINE), ) ```

Ingestion (parallel embed + index into both backends):

```python async def ingest_chunk(chunk: dict) -> None: """ chunk: {doc_id, content, metadata} Embeds and indexes into both Elasticsearch and Qdrant. """ # Embed emb = oai.embeddings.create(model=EMBED_MODEL, input=[chunk["content"]]).data[0].embedding # Index in Elasticsearch (BM25) await es.index( index=INDEX_NAME, id=chunk["doc_id"], document={ "doc_id": chunk["doc_id"], "content": chunk["content"], "metadata": chunk.get("metadata", {}), } ) # Upsert in Qdrant (dense) # Qdrant requires integer IDs; use a hash if doc_id is a string import hashlib int_id = int(hashlib.md5(chunk["doc_id"].encode()).hexdigest(), 16) % (2**63) await qd.upsert( collection_name=COLLECTION, points=[ PointStruct( id=int_id, vector=emb, payload={"doc_id": chunk["doc_id"], "content": chunk["content"], **chunk.get("metadata", {})} ) ] ) ```

Run BM25 and dense retrieval in parallel with `asyncio.gather` to minimize latency. Fuse with RRF and fetch content for the top results.

```python async def retrieve_hybrid( query: str, top_k: int = 5, retrieve_k: int = 50, # retrieve more, fuse, then cut to top_k ) -> list[dict]: """ Parallel BM25 + dense retrieval, fused with RRF. Returns top_k results with content and metadata. """ # Embed the query for dense retrieval q_emb = oai.embeddings.create(model=EMBED_MODEL, input=[query]).data[0].embedding # Run BM25 and dense in parallel bm25_coro = es.search( index=INDEX_NAME, body={"query": {"match": {"content": query}}, "size": retrieve_k, "_source": ["doc_id"]}, ) dense_coro = qd.search( collection_name=COLLECTION, query_vector=q_emb, limit=retrieve_k, with_payload=True, ) bm25_resp, dense_resp = await asyncio.gather(bm25_coro, dense_coro) # Extract ranked ID lists bm25_ids = [hit["_source"]["doc_id"] for hit in bm25_resp["hits"]["hits"]] dense_ids = [point.payload["doc_id"] for point in dense_resp] # RRF fusion fused = reciprocal_rank_fusion([dense_ids, bm25_ids], k=60) top_ids = [doc_id for doc_id, _ in fused[:top_k]] # Fetch content for top results from Elasticsearch (single mget) mget_resp = await es.mget( index=INDEX_NAME, body={"ids": top_ids}, ) id_to_doc = { doc["_id"]: doc["_source"] for doc in mget_resp["docs"] if doc.get("found") } # Return in fused rank order results = [] for doc_id, score in fused[:top_k]: if doc_id in id_to_doc: results.append({ "doc_id": doc_id, "rrf_score": score, "content": id_to_doc[doc_id]["content"], "metadata": id_to_doc[doc_id].get("metadata", {}), }) return results ```

The `retrieve_k=50` pattern (retrieve 50 from each system, fuse, cut to 5) is standard. Retrieving only top_k=5 from each system before fusion misses documents that rank 10th in one system and 3rd in another — those fused to a high RRF score. The 50-candidate pool captures cross-system consensus effectively. Cost: one extra Qdrant query over 50 vectors instead of 5 (negligible); one ES search over 50 hits (negligible).

Latency of the parallel retrieve step on a co-located stack (app + Elasticsearch + Qdrant in the same region) is typically 5-15ms. Dominant latency source is the embedding call to OpenAI (~80-120ms for text-embedding-3-small) — consider batching queries or using a self-hosted embedding model (e.g., sentence-transformers/all-MiniLM-L6-v2 via ONNX Runtime) to eliminate network latency on embeddings if sub-50ms retrieval is required.

RRF produces a fused list based on rank position alone — it has no access to the actual query-document content at ranking time. A cross-encoder reranker sees both the query and each candidate document's full text, computing a relevance score from their interaction. Cohere rerank-v3.5 is the standard production reranker in 2026: it achieves state-of-the-art on MS-MARCO and BEIR reranking tasks and is available via a simple API call.

The standard production pattern: retrieve top-50 via RRF, rerank with Cohere to get top-10, pass top-10 to Claude for generation. The reranking call adds ~100-200ms latency and costs $0.0001 per 50-document rerank (at $2/1M search units where each document = 1 unit). At 100K daily queries with 50 candidates each: $0.0001 × 100K = $10/day. For most RAG workloads, the quality improvement justifies this.

```bash pip install cohere==5.5.0 ```

```python import cohere co = cohere.Client(api_key=os.environ["COHERE_API_KEY"]) def rerank_results( query: str, candidates: list[dict], top_n: int = 10, ) -> list[dict]: """ Rerank candidates with Cohere rerank-v3.5. candidates: list of {doc_id, content, ...} Returns top_n candidates in reranked order. """ if not candidates: return [] response = co.rerank( model="rerank-v3.5", query=query, documents=[c["content"] for c in candidates], top_n=top_n, return_documents=False, # we already have the documents ) reranked = [] for result in response.results: candidate = candidates[result.index] reranked.append({ **candidate, "rerank_score": result.relevance_score, }) return reranked async def retrieve_and_rerank( query: str, final_k: int = 5, retrieve_k: int = 50, ) -> list[dict]: """Full pipeline: hybrid retrieve → RRF fusion → Cohere rerank.""" # Step 1: hybrid retrieval (top 50 fused) candidates = await retrieve_hybrid(query, top_k=retrieve_k, retrieve_k=retrieve_k) # Step 2: Cohere rerank (top 5 from 50) return rerank_results(query, candidates, top_n=final_k) ```

The `return_documents=False` parameter saves bandwidth — Cohere's response only includes indexes and scores rather than returning the full document text again. Use `result.index` to look up the original candidate from your list.

Cohere rerank-v3.5 vs alternatives: Cohere is the easiest managed option. Alternatives: Jina reranker-v2 (open-source, self-host on a GPU instance), BGE-reranker-v2-m3 (open-source, strong BEIR scores, available via HuggingFace Inference API), Voyage rerank-2 (voyage.ai, competitive pricing). All cross-encoder rerankers follow the same API pattern: query + list of document texts → list of relevance scores.

Running separate BM25 and dense indexes adds operational complexity — two services to manage. Qdrant supports native hybrid search with sparse + dense vectors in a single collection. The sparse vector is typically SPLADE or BM25-encoded (using the `qdrant-fastembed` library), eliminating the Elasticsearch dependency entirely.

```bash pip install qdrant-client[fastembed]==1.9.0 # fastembed extra installs onnxruntime + model weights for local sparse encoding ```

```python from qdrant_client import QdrantClient from qdrant_client.models import ( Distance, VectorParams, SparseVectorParams, SparseIndexParams, PointStruct, SparseVector, NamedVector, NamedSparseVector, SearchRequest, FusionQuery, Prefetch, Fusion, ) qd = QdrantClient(url=os.environ.get("QDRANT_URL", "http://localhost:6333")) # Create collection with both dense and sparse vectors qd.create_collection( collection_name="hybrid_docs", vectors_config={ "dense": VectorParams(size=1536, distance=Distance.COSINE), }, sparse_vectors_config={ "sparse": SparseVectorParams( index=SparseIndexParams(on_disk=False) ) } ) ```

Upsert with both dense and sparse vectors using fastembed for sparse encoding:

```python from fastembed import SparseTextEmbedding sparse_model = SparseTextEmbedding(model_name="Qdrant/bm25") def encode_sparse(texts: list[str]) -> list[SparseVector]: results = [] for emb in sparse_model.embed(texts): results.append(SparseVector(indices=emb.indices.tolist(), values=emb.values.tolist())) return results def upsert_hybrid(points: list[dict]) -> None: """ points: list of {id: str, content: str, dense_vector: list[float], metadata: dict} """ sparse_vectors = encode_sparse([p["content"] for p in points]) qd.upsert( collection_name="hybrid_docs", points=[ PointStruct( id=points[i]["id"], vector={ "dense": points[i]["dense_vector"], "sparse": sparse_vectors[i], }, payload={"content": points[i]["content"], **points[i].get("metadata", {})} ) for i in range(len(points)) ] ) ```

Query with Qdrant's native RRF fusion (available in Qdrant 1.7+):

```python from fastembed import SparseTextEmbedding def retrieve_qdrant_hybrid(query: str, top_k: int = 5) -> list[dict]: """Native hybrid search in Qdrant with RRF fusion.""" # Dense query vector dense_vec = oai.embeddings.create(model=EMBED_MODEL, input=[query]).data[0].embedding # Sparse query vector sparse_vec = list(sparse_model.embed([query]))[0] qdrant_sparse = SparseVector( indices=sparse_vec.indices.tolist(), values=sparse_vec.values.tolist() ) results = qd.query_points( collection_name="hybrid_docs", prefetch=[ Prefetch(query=dense_vec, using="dense", limit=50), Prefetch(query=qdrant_sparse, using="sparse", limit=50), ], query=FusionQuery(fusion=Fusion.RRF), limit=top_k, with_payload=True, ) return [ {"doc_id": str(p.id), "score": p.score, "content": p.payload.get("content", ""), "metadata": p.payload} for p in results.points ] ```

Qdrant's native RRF (`Fusion.RRF`) is implemented at the server side — no client-side fusion code needed. The `prefetch` list defines the candidate retrieval steps; `query=FusionQuery(fusion=Fusion.RRF)` fuses them. This pattern eliminates the Elasticsearch dependency entirely: one service (Qdrant) handles both sparse BM25-like scoring and dense ANN. The operational simplicity is the main advantage over the Elasticsearch + Qdrant two-service architecture.

Weaviate supports hybrid search natively via the `hybrid` query parameter. The `alpha` parameter controls the blend: `alpha=0` is pure BM25, `alpha=1` is pure dense, `alpha=0.5` is equal weighting. Weaviate uses a weighted fusion (not RRF) by default — adjust alpha by domain. For technical documentation, try alpha=0.3 (more BM25 weight); for general QA, try alpha=0.7.

```bash pip install weaviate-client==4.6.0 ```

```python import weaviate import weaviate.classes as wvc client = weaviate.connect_to_local() # or connect_to_wcs() for Weaviate Cloud # Create collection (schema) if not client.collections.exists("RagDocs"): client.collections.create( name="RagDocs", vectorizer_config=wvc.config.Configure.Vectorizer.text2vec_openai( model="text-embedding-3-small" ), properties=[ wvc.config.Property(name="content", data_type=wvc.config.DataType.TEXT), wvc.config.Property(name="source_id", data_type=wvc.config.DataType.TEXT), ] ) collection = client.collections.get("RagDocs") ```

Hybrid search query:

```python def retrieve_weaviate_hybrid( query: str, top_k: int = 5, alpha: float = 0.5, # 0 = BM25 only, 1 = dense only ) -> list[dict]: """ Hybrid search in Weaviate with configurable alpha blend. """ results = collection.query.hybrid( query=query, alpha=alpha, limit=top_k, return_properties=["content", "source_id"], ) return [ { "doc_id": str(obj.uuid), "content": obj.properties["content"], "source_id": obj.properties.get("source_id", ""), "score": obj.metadata.score if obj.metadata else 0.0, } for obj in results.objects ] ```

Weaviate's `alpha` parameter is simpler to tune than RRF's `k` because its semantics are more intuitive (linear blend vs rank smoothing). For production, evaluate alpha at 0.3, 0.5, and 0.7 on your held-out eval set and pick the value with the highest NDCG@5 or recall@5. Most teams end up between 0.4-0.6 for general text corpora.

For teams processing billions of queries per day, Vespa is the production-scale hybrid search platform. Vespa runs BM25, dense ANN, and arbitrary ML ranking inside a single query execution — no client-side fusion, no separate services. The trade-off: Vespa has a steep learning curve and requires YAML-based application configuration.

Vespa rank profile for hybrid BM25 + dense with cross-encoder reranking:

```yaml # services.xml schema snippet rank-profile hybrid-rag { first-phase { expression { # RRF-style fusion: BM25 + HNSW inner product reciprocal_rank_fusion( fieldMatch(content).significance, closeness(field, embedding) ) } } second-phase { rerank-count: 50 expression { # Use a ONNX cross-encoder model for second-phase reranking onnxModel(reranker).score } } global-phase { rerank-count: 10 } inputs { query(embedding) tensor<float>(d[1536]) } } ```

Vespa executes all three phases server-side: first-phase BM25+dense fusion across all candidates, second-phase ONNX reranker on top-50, global-phase final cut to top-10. The ONNX cross-encoder model runs on each Vespa content node — for high query rates this is more efficient than making a separate Cohere API call per query. The operational cost: maintaining a Vespa cluster and managing ONNX model deployment. Most teams under 10M queries/day are better served by the Elasticsearch + Qdrant + Cohere stack. Vespa becomes cost-effective at high query volume where per-query API costs for a hosted reranker would dominate.

At 100K daily queries with hybrid retrieval + Cohere rerank on top-50:

``` Component Cost/query Daily (100K) Monthly ───────────────────────────────────────────────────────────── OpenAI embed (query) $0.0000003 $0.030 $0.90 Elasticsearch $0.000001 $0.100 $3.00 Qdrant $0.000002 $0.200 $6.00 Cohere rerank-v3.5 $0.0001 $10.00 $300 Claude Sonnet 4.6 $0.000012 $1.20 $36 ───────────────────────────────────────────────────────────── Total ~$0.000115 ~$11.53 ~$346 ```

Cohere reranking dominates the hybrid pipeline cost at 87% of total. For cost-sensitive workloads, consider: (1) skip reranking and use RRF alone (quality drop: 2-5 NDCG@10 points on most datasets); (2) rerank only for high-stakes queries (identify them with a fast classifier on the query); (3) use a self-hosted reranker (BGE-reranker-v2-m3 on a g4dn.xlarge AWS instance at ~$0.5/hr can handle ~20K reranks/hr, breaking even vs Cohere at >50K queries/day).

For the full workload cost model including embedding ingestion costs, vector storage costs, and generation costs, use the RAG cost per query calculator. For architecture choice between Pinecone-only, pgvector, and hybrid systems, see the RAG architecture decision tree.