Cohere Rerank vs Voyage Rerank vs BGE Rerankers (2026): The Honest RAG Pipeline Comparison

To understand why rerankers exist, you need to understand the trade-off that embedding-based retrieval makes. Embedding models (bi-encoders) encode the query and each document independently into fixed-dimension vectors. Similarity search finds the documents whose vectors are closest to the query vector. The key word is independently — the query and document are never processed together. The model has no way to reason about how well a specific document answers a specific query; it can only compare vector proximity in an abstract embedding space.

Cross-encoders are architecturally different. They take the query and a candidate document as a single concatenated input — [CLS] query [SEP] document [SEP] — and output a single relevance score. Because query and document are processed together, the model can attend to the specific words in the query when scoring the document, and attend to the specific words in the document when scoring the query. The interaction is direct. This is why cross-encoders consistently outperform bi-encoders on precision metrics: they can detect that a document uses the word 'bond' in a financial context (relevant) vs a chemical context (not relevant) based on the specific query asking about 'corporate bonds.'

The cost of this accuracy is inference time. A bi-encoder encodes a query once and then does dot-product similarity against pre-computed document vectors in milliseconds. A cross-encoder must run a forward pass for each (query, document) pair — if you have 100 candidate documents, you run 100 forward passes. This is O(N) per query, where N is the candidate count. At a 1-billion-document corpus, running a cross-encoder over all documents is computationally infeasible. This is why rerankers are always second-stage: first-pass retrieval (BM25 or embedding similarity) narrows candidates to 50-200 documents, then the cross-encoder reranks those candidates precisely.

The latency math: a typical cross-encoder forward pass on a T4 GPU takes 10-50ms depending on document length. Reranking 100 documents sequentially = 1-5 seconds, which is too slow for user-facing latency. Production deployments either (1) batch all 100 pairs into a single GPU inference call (reduces wall-clock time to 50-200ms) or (2) use the cloud API (Cohere, Voyage) which handles batching server-side. Self-hosted BGE needs batching logic in your application; cloud APIs abstract this away.

Why the quality lift is so consistent: the BEIR benchmark (Benchmarking Information Retrieval, 18 heterogeneous datasets covering news, biomedical, financial, code, legal, and general web corpora) consistently shows that reranking on top of first-stage BM25 or dense retrieval adds 3-10 nDCG@10 points. On some BEIR tasks (TREC-COVID, FiQA, SCIDOCS) the lift exceeds 10 points. The average lift across all BEIR tasks is 5-8 nDCG@10 when replacing BM25-only retrieval with BM25 + cross-encoder rerank. This is not a marginal improvement — it moves systems from 'borderline useful' to 'reliably useful' on real retrieval tasks.

Why not just use a bigger embedding model instead of a reranker? Bigger embedding models (e.g., upgrading from voyage-3-lite to voyage-3-large) add 3-6 nDCG@10 points. Adding a reranker adds another 5-10 points. They compound rather than substitute. The optimal architecture for quality-critical RAG is: strong embedding model for first-stage recall + cross-encoder reranker for second-stage precision.

Cohere rerank-v3.5 launched in late 2025 as the successor to rerank-v3, with improved multilingual performance and better handling of long documents. It is Cohere's primary go-to-market product alongside their embed-v4.0 embeddings, and its pricing model is the simplest of any reranker: $1 per 1,000 queries, where one query can rerank up to 100 documents. Whether you pass 10 documents or 100 documents per query, you pay $0.001.

The per-query pricing model is unusual in the API world and has a key implication: Cohere's reranker is highly cost-efficient when you rerank many documents per query, and less efficient if you only rerank a few. If you send 100 candidates per query, you pay $0.001/query and get 100 scored pairs. If you send 10 candidates per query (because your first-stage retrieval only returns 10 useful candidates), you still pay $0.001/query but you get only 10 scored pairs — effectively paying 10x the per-pair cost.

BEIR benchmarks: as of June 2026, Cohere rerank-v3 scores among the top cross-encoder rerankers on the MTEB leaderboard's reranking tasks. nDCG@10 on the BEIR test suite is competitive with the best open-source rerankers. Verify the exact current position at https://huggingface.co/spaces/mteb/leaderboard under the reranking category before making procurement decisions — these rankings update as new models are submitted.

Multilingual is Cohere's headline differentiator. rerank-v3.5 supports 100+ languages and, critically, cross-lingual reranking: a query in English can rerank documents in French, Spanish, Japanese, Arabic, or Korean without the query or documents being pre-translated. This is valuable for global enterprise RAG systems where the document corpus spans languages.

Integration: Cohere rerank works with any embedding source. You do not need to use Cohere embed-v4.0 as your embedding model to use Cohere rerank-v3.5 as your reranker. Cohere's SDK exposes a simple co.rerank() call that accepts the raw query string and a list of document strings — the reranker receives the text, not the vectors. Integrates with LangChain, LlamaIndex, Haystack, and raw Python.

Limitations: the $1/1k query cost becomes meaningful at high volume. 10M queries/month = $10,000. 100M queries/month = $100,000. For consumer-facing applications with high query volume, this is a significant budget item. Cloud API means latency is network-dependent and varies with Cohere's infrastructure; there is no SLA for inference latency published on the public pricing tier. For latency-sensitive applications, test empirically.

Voyage rerank-2 takes a structurally different approach to pricing. Instead of charging per query, Voyage charges per token: $0.05 per 1M reranked tokens. The rerank-2-lite variant is $0.02 per 1M tokens. The total token count for a rerank call is the sum of (query tokens + document tokens) across all pairs. If your query is 20 tokens and you rerank 100 documents averaging 200 tokens each, the token count is 20 × 100 (query repeated) + 200 × 100 (documents) = 22,000 tokens per query call.

At 22,000 tokens per query × $0.05/1M = $0.0011 per query. Roughly comparable to Cohere at $0.001 per query for this document length. But the comparison shifts dramatically with document length. For short documents (50-word snippets, ~65 tokens): 20 × 100 + 65 × 100 = 8,500 tokens × $0.05/1M = $0.000425 per query — less than half Cohere's cost. For long documents (500-word passages, ~650 tokens): 20 × 100 + 650 × 100 = 67,000 tokens × $0.05/1M = $0.00335 per query — 3.35x more expensive than Cohere.

This makes Voyage's pricing model optimal for short-document RAG (FAQ retrieval, product catalog search, short customer reviews) and uncompetitive for long-document RAG (contract retrieval, research papers, technical documentation). Know your average document length before defaulting to either model. Use our embeddings cost calculator to model total RAG pipeline cost including reranker.

rerank-2-lite at $0.02/1M tokens is the budget option with slightly reduced quality. Voyage positions rerank-2-lite for latency-sensitive applications where throughput matters more than precision — smaller model, faster inference, meaningfully lower cost. At 22,000 tokens per query, rerank-2-lite costs $0.00044 per query, competitive with BGE self-hosted at mid-volume.

BEIR benchmarks: Voyage rerank-2 scores competitively with Cohere rerank-v3.5 on English BEIR tasks. Multilingual support covers English and major European languages; cross-lingual is more limited than Cohere. Domain-specialized reranker variants (rerank-2-finance, rerank-2-law) are implicitly referenced in Voyage documentation and may be available for enterprise customers — verify with sales for production use.

Integration: Voyage exposes a similar reranking API to Cohere. The raw text is passed (not vectors), and a relevance score is returned per document. LangChain and LlamaIndex integrations available.

The BGE reranker family (BAAI General Embeddings, developed by the Beijing Academy of Artificial Intelligence) are open-source cross-encoder models available on Hugging Face under the Apache 2.0 license, free for commercial use with no API fee. The relevant models for production reranking in 2026 are: bge-reranker-v2-m3 (~568M parameters, multilingual, T4-GPU compatible), bge-reranker-v2-gemma (Gemma-2B base, ~2B parameters, stronger but requires L4/A100), and bge-reranker-large (560M parameters, English-optimized, the well-understood baseline).

The hardware requirements are concrete. bge-reranker-v2-m3 at 568M parameters fits on a T4 GPU (16 GB VRAM) and handles 500-1,000 rerank calls per second depending on document length and batch size. On GCP, a T4 instance (n1-standard-4 + T4) runs ~$0.40/hr on-demand, $0.14/hr on spot. At 24/7 on-demand: $0.40 × 24 × 30 = $288/month, call it $290. At spot: ~$100/month with occasional interruption. bge-reranker-v2-gemma at 2B parameters requires an A100 (40 GB) or L4 (24 GB) — L4 on GCP is ~$0.70/hr on-demand ($504/month), A100 is ~$2.50-3.00/hr ($1,800-2,160/month).

The throughput math at 10M queries/month: 10M queries ÷ 30 days ÷ 24 hours ÷ 3,600 seconds = ~3.9 queries/second average. bge-reranker-v2-m3 on a single T4 handles 500-1,000 QPS — more than 100x headroom on average load. Even at peak (say 10x average = 39 QPS), a single T4 handles it comfortably. The cost: $290/month vs Cohere's $10,000/month. That is a 97% cost reduction.

BEIR performance: bge-reranker-v2-m3 is competitive with Cohere rerank-v3 on English BEIR tasks — nDCG@10 scores are within 1-3 points on most individual BEIR datasets according to the MTEB leaderboard (verify at https://huggingface.co/spaces/mteb/leaderboard). On multilingual tasks, bge-reranker-v2-m3's cross-lingual training gives it competitive performance vs Cohere's multilingual claims. bge-reranker-v2-gemma improves over m3 by 2-5 nDCG@10 points on English tasks, at the cost of 4x the hardware footprint.

What you give up by self-hosting: operational overhead (deployment, monitoring, auto-scaling, CUDA driver maintenance, model updates), cold-start latency if you run on spot instances, and the engineering time to build the inference service. A minimal FastAPI wrapper around BGE reranker is roughly 50-100 lines of Python; frameworks like Hugging Face Text Embeddings Inference (TEI) make deployment substantially easier. The cloud providers (GCP, AWS, Azure) all offer pre-built inference endpoints that reduce operational burden significantly.

When BGE self-hosted beats the API economically: the break-even point varies by query volume. At 1M queries/month, Cohere costs $1,000 and BGE T4 costs $290 — BGE wins by $710. At 100k queries/month, Cohere costs $100 and BGE T4 costs $290 — Cohere wins unless you need the GPU for other things. The economic inflection point is roughly 300k-500k queries/month for a dedicated T4 instance. Below that, the API is cheaper when you account for engineering time. Above that, BGE is cheaper, and the gap grows linearly with volume.

BEIR (Benchmarking Information Retrieval) is the standard evaluation suite for information retrieval models — 18 heterogeneous datasets covering news (TREC-NEWS), biomedical (TREC-COVID, BioASQ, NFCorpus, SCIDOCS), financial (FiQA-2018), code, legal (FIQA legal), and general web (MS MARCO, Natural Questions, HotpotQA) retrieval. The standard metric is nDCG@10 (normalized Discounted Cumulative Gain at rank 10), which measures how well the top-10 results are ordered by relevance. Higher is better; the maximum is 1.0; real-world strong reranker scores are in the 0.55-0.75 range depending on the dataset.

A note on reporting: BEIR consists of 18 datasets, and models can cherry-pick the datasets on which they perform best. When evaluating a model, check the average across all BEIR datasets, not just the cherry-picked highlights. The MTEB leaderboard reports the full breakdown — use it. Verify current standings at https://huggingface.co/spaces/mteb/leaderboard under the 'Reranking' category.

What the reranking task measures in MTEB: the standard MTEB reranking setup starts from an existing first-stage retrieval result (usually BM25 or a dense retriever) and measures how well the reranker re-orders those candidates. A nDCG@10 gain of 5 points over the baseline is a meaningful improvement for user-facing RAG. A gain of 10+ points is exceptional.

Baseline context: BM25 (pure lexical retrieval, no embedding or reranking) achieves nDCG@10 averages in the 0.40-0.50 range on BEIR tasks. Strong embedding-only (no reranker) retrieval achieves 0.50-0.60. Adding a cross-encoder reranker pushes to 0.55-0.70. The reranker contribution is especially large on tasks where lexical similarity is misleading (domain jargon, paraphrase queries, cross-lingual retrieval).

Data caveats: benchmark scores reflect controlled evaluation conditions with specific document lengths and query styles. Production performance may differ. A pipeline built on MS MARCO-fine-tuned models may overfit MS MARCO-style queries. If your retrieval task is domain-specific (medical, legal, code), look for BEIR subsets that match your domain and weight those scores more heavily. Always run an empirical evaluation on your own held-out (query, relevant-doc) pairs before committing to a production reranker.

As of June 2026: all three major options (Cohere rerank-v3.5, Voyage rerank-2, bge-reranker-v2-m3/gemma) achieve strong BEIR scores and are meaningfully better than no reranking. The quality differences between them are smaller than the quality difference between having a reranker and not having one. Choose primarily on cost model and operational requirements; trust that quality is competitive across all three.

**1M queries/month scenario** (mid-scale SaaS RAG — e.g., 10,000 daily active users averaging 3-4 queries each). Cohere rerank-v3.5: $1,000/month. Voyage rerank-2 (avg 200-token documents): ~$220/month. BGE bge-reranker-v2-m3 on a T4 (on-demand): $290/month. BGE on spot: ~$100/month. Verdict: at 1M queries, Voyage token pricing wins on raw cost for typical document lengths; BGE self-hosted is competitive on cost but adds engineering overhead. Cohere is 3.5-5x more expensive for general-purpose use cases.

**10M queries/month scenario** (high-scale consumer or enterprise RAG — e.g., 100k DAU, 3-4 queries each). Cohere: $10,000/month. Voyage rerank-2 (avg 200-token docs): ~$2,200/month. BGE bge-reranker-v2-m3 on a single T4 (on-demand): $290/month (single T4 handles this load comfortably at 3.9 QPS average). BGE on spot: ~$100/month. Verdict: the cost gap at this scale is the core reason BGE self-hosting exists. BGE on-demand is 97% cheaper than Cohere; 87% cheaper than Voyage. Engineering overhead to deploy BGE is ~1-2 engineer-days, meaning ROI is positive in the first month at this volume.

**100M queries/month scenario** (high-scale search or platform — e.g., a search product, a large-scale AI assistant). Cohere: $100,000/month. Voyage rerank-2 (avg 200-token docs): ~$22,000/month. BGE bge-reranker-v2-m3 — at 100M queries/month, average QPS is 39. A single T4 handles this. But peak load (10x average = 390 QPS) requires 1-2 additional T4 instances with auto-scaling. 2× T4 always-on: ~$580/month. With Kubernetes auto-scaling on GKE, you might average 1.3× T4 = ~$377/month. Compare to $100,000/month on Cohere.

**Long-document exception**: Voyage token pricing becomes expensive with long documents. If your average document is 1,000 tokens (not 200), Voyage rerank-2 at 10M queries = ~$11,000/month, comparable to Cohere. If your average document is 2,000 tokens, Voyage = ~$22,000/month, worse than Cohere. BGE self-hosted is still ~$290/month regardless of document length, since compute cost per GPU-hour is fixed. For long-document RAG (legal, technical), BGE's cost advantage over API rerankers is even larger.

**Infrastructure cost footnote**: BGE self-hosting costs above assume GPU-only. Add ~$50-100/month per instance for CPU, memory, disk, and networking. Add Kubernetes cluster cost if using managed auto-scaling (~$50-200/month for GKE Autopilot at this scale). A realistic BGE production deployment at 10M queries/month is $400-600/month all-in, still 95%+ cheaper than Cohere. See our vector DB cost calculator to model the full RAG infrastructure budget.

**Decision threshold summary**: below 300k queries/month, API rerankers (Cohere or Voyage) are usually cheaper than a dedicated GPU instance. Between 300k-1M queries/month, Voyage token pricing is competitive; BGE self-hosted on spot is cheaper. Above 1M queries/month, BGE self-hosted wins on cost unless you have no ML infrastructure capability or multilingual requirements that BGE cannot meet.

Multilingual reranking is harder than multilingual embedding. An embedding model can represent query and document in a shared multilingual space, allowing cross-lingual similarity search at retrieval time. But cross-encoder rerankers process (query, document) pairs jointly — for cross-lingual reranking to work well, the model must understand the semantic relationship between a query in Language A and a document in Language B, which requires more sophisticated multilingual training than same-language reranking.

Cohere rerank-v3.5 is the strongest multilingual option. Cohere explicitly trains for cross-lingual reranking with 100+ language support. A query in English reranking documents in German, Japanese, Arabic, or Korean is a supported use case, not an edge case. For global enterprise RAG where document corpora span multiple languages, Cohere's multilingual reranker is a meaningful architectural advantage.

Voyage rerank-2 supports English and major European languages (French, Spanish, German, Italian, Portuguese, Dutch). Cross-lingual support between, say, English and Mandarin or English and Arabic is not a core capability. For primarily English or English-European multilingual use cases, Voyage is adequate. For broader multilingual needs, Cohere or BGE.

bge-reranker-v2-m3 (the 'm3' refers to Multi-Multilingual, Multi-Functionality, Multi-Granularity) is trained specifically for cross-lingual retrieval. The model handles code-switching queries and cross-language document pairs. BEIR scores on multilingual retrieval tasks are competitive with Cohere. The v2-m3 variant is the BGE option to choose for multilingual needs — the standard bge-reranker-large is English-optimized and not appropriate for multilingual reranking.

Practical cross-lingual test: a query 'What is the interest rate policy?' should surface a relevant Spanish-language document discussing 'la política de tipos de interés' even without translation. Cross-encoder models with strong multilingual training handle this; models trained on English-only data do not. If cross-lingual reranking matters for your use case, test empirically on representative query-document language pairs before deploying.

Language-pair coverage gap: even the best multilingual rerankers underperform on low-resource language pairs (e.g., English-Swahili, English-Yoruba, English-Tamil). If your document corpus includes low-resource languages, verify performance with actual test queries before assuming multilingual claims hold.

Reranking adds latency to the query path. The total RAG query latency is: (1) first-stage retrieval [10-50ms for dense or BM25 retrieval], plus (2) reranker inference [50-500ms depending on model, batch size, and candidate count], plus (3) LLM generation [500ms-5s]. Reranking is typically not the latency bottleneck but it is a non-trivial addition that needs to be budgeted.

Cloud API latency (Cohere, Voyage): real-world p50 reranker API latency is typically 100-300ms for a 100-document rerank request at typical document lengths, measured from request submission to response received. This includes network round-trip plus server-side inference. p95 can be 500ms-1s. Do not assume sub-100ms for production SLA budgeting without empirical measurement.

Self-hosted BGE latency: bge-reranker-v2-m3 on a T4 GPU with proper batching (all 100 (query, document) pairs in a single batch forward pass) takes 50-200ms wall-clock time depending on document length. The key is batching — processing pairs sequentially (one at a time) is 10-50x slower than batched inference. Any production BGE deployment must implement proper batching. Hugging Face Text Embeddings Inference (TEI) handles this correctly out of the box.

Optimizing latency for reranking: reduce the candidate count (rerank top-20 instead of top-100 — quality loss is typically small; going from 100 to 20 candidates saves 75% of inference time with <5% quality loss for most tasks). Use reranker-lite variants (Voyage rerank-2-lite, bge-reranker-base) for latency-critical applications. Apply reranking only to query categories that need precision (not all queries need reranking — keyword lookups on structured data do not benefit).

Async reranking pattern: for applications where result re-ordering can tolerate slightly higher latency (chatbot RAG where the LLM still needs time to generate an answer), trigger the reranker call in parallel with the LLM context assembly. The reranker result arrives before the LLM generation completes; the final context is assembled with reranked order. This hides most of the reranker latency behind LLM inference time.

Throughput at scale: for batch offline reranking jobs (re-scoring document collections, offline index enrichment), throughput matters more than latency. BGE on a single T4 achieves 500-1,000 pairs/second with optimal batching. A 1M-document reranking job at 100 candidates each = 100M pair scores needed; at 750 pairs/second = 37 hours on one T4. If you run offline reranking jobs, size accordingly or parallelize across multiple GPU instances.

Most RAG systems start with single-stage dense retrieval: embed all documents, store vectors, query with embedding similarity. Adding a reranker is the single highest-ROI improvement available to a mature RAG system. The migration pattern is well-established.

Step 1: add the reranker call after your existing retrieval step. Your existing embedding retrieval returns top-100 candidates. Pass those 100 candidates and the original query to the reranker API (or self-hosted model). The reranker returns relevance scores; re-sort the candidates by score. Use the top-5 or top-10 of the re-sorted list as the context for your LLM. Zero changes to your embedding model, vector store, or LLM — the reranker inserts as a post-processing step.

Step 2: validate the quality improvement on your evaluation set. Before deploying to production, run your held-out (query, relevant-doc) eval set through the pipeline with and without reranking. Measure recall@5, recall@10, nDCG@10 before and after. A production-quality reranker should add at least 3-8 nDCG@10 points; if it does not, you may have a first-stage retrieval problem (returning completely irrelevant candidates that no reranker can salvage) or your eval set is not representative.

Step 3: tune the candidate count. More candidates for the reranker to process = higher quality ceiling + more cost and latency. Typical production values are 20-50 candidates (first-stage retrieval returns top-50, reranker scores all 50, top-5 goes to LLM). 100 candidates is the ceiling for most use cases — going beyond 100 rarely helps because the additional candidates from dense retrieval are increasingly noisy. Find your specific quality/cost/latency trade-off empirically.

Step 4: monitor reranker quality over time. Embedding model drift (corpus updates, new document types) can change the distribution of first-stage retrieval candidates, which affects reranker performance. Add reranker confidence score distributions to your observability stack — a sudden shift in average reranker scores is a signal that first-stage retrieval quality has changed.

Common mistake during migration: not accounting for reranker latency in p99 SLA budgets. The median reranker call is fast; the 99th percentile (long documents, slow API day, cold GPU) can be 2-5x slower. Test your latency SLA under realistic load before going to production.

Use Cohere rerank-v3.5 when: (1) your query volume is under 1M/month and you value simplicity over cost; (2) you need multilingual or cross-lingual reranking with 100+ language support; (3) you are already using Cohere embeddings and want a native integrated solution; (4) you need the flat per-query pricing model to simplify billing (budget per query is predictable); (5) you have no ML infrastructure capability and need a fully managed reranker with Cohere's support.

Use Voyage rerank-2 when: (1) your documents are short (under 200 tokens average) and you want to optimize cost — token-based pricing is favorable for short docs; (2) you are already using Voyage embeddings and want a native pairing; (3) you need domain-specialized reranking (finance, law) and have verified with Voyage sales that domain-specific variants are available for your use case; (4) you want token-based pricing to more accurately model cost proportional to work performed.

Use BGE self-hosted when: (1) your query volume exceeds 500k/month and cost is a priority — the ROI on self-hosting becomes positive at this volume; (2) you need maximum data privacy (documents never leave your infrastructure); (3) you need the lowest possible latency via local inference with no network round-trip; (4) you have existing ML infrastructure (Kubernetes cluster, GPU node pools) that can host a small inference service; (5) you are building a product where the reranker is a competitive cost differentiator and 97% savings at 10M queries/month materially affects your unit economics.

Use bge-reranker-v2-gemma specifically when: quality is more important than cost within the self-hosted option set and you have A100 or L4 GPU capacity. The 2-5 nDCG@10 point improvement over bge-reranker-v2-m3 is meaningful for high-stakes retrieval (medical, legal, financial RAG where wrong answers carry real consequences).

Hybrid architecture: nothing prevents using Cohere or Voyage for low-volume production traffic while migrating to BGE for high-volume paths. A tiered reranker routing layer — Cohere for premium users (low volume, SLA matters), BGE for high-volume standard tier (cost matters) — is a reasonable production pattern for platforms with heterogeneous user segments. This is compatible with a vector DB setup that uses a single index but separate reranker routes per tier.

What not to do: do not skip reranking because the embedding model is good. The best embedding model in the world is still a bi-encoder producing an approximation of relevance. A cross-encoder reranker on even 20 candidates will improve precision measurably. The $0.001/query cost of reranking is the cheapest retrieval quality investment available.

Pricing data sourced from: Cohere pricing page at https://cohere.com/pricing (fetched June 2026); Voyage AI reranker docs at https://docs.voyageai.com/docs/reranker (fetched June 2026); BGE models on Hugging Face at https://huggingface.co/BAAI (fetched June 2026). GPU pricing from GCP compute pricing at https://cloud.google.com/compute/gpus-pricing and AWS EC2 pricing at https://aws.amazon.com/ec2/pricing/ (fetched June 2026).

BEIR benchmark data sourced from the MTEB leaderboard at https://huggingface.co/spaces/mteb/leaderboard, reranking category, fetched June 2026. MTEB leaderboard rankings update continuously as new models are submitted — the relative positioning of models can shift between publication and reading. Treat relative rankings as directional guidance; verify absolute scores before making procurement decisions.

GPU cost model assumptions: on-demand pricing for a single T4 GPU on GCP (n1-standard-4 + T4) at ~$0.40/hr. Spot instance prices fluctuate and are excluded from primary analysis. Prices vary by region — us-central1 pricing used as reference. AWS g4dn.xlarge (T4-based) is approximately the same price range. Azure NC4as T4 v3 is similarly priced. Your actual GPU cost may differ based on reserved instance discounts, committed use discounts, or negotiated enterprise pricing.

BGE throughput estimates (500-1,000 pairs/second on T4) are based on reported benchmarks and community testing with bge-reranker-v2-m3 at typical document lengths (128-512 tokens) using batch inference. Your actual throughput will vary with document length, batch size, CUDA version, and model quantization. Measure empirically on your workload before capacity planning.

All cost scenarios in this guide represent approximate models to illustrate relative economics. Actual production costs will differ based on infrastructure choices, reserved capacity, data transfer costs, and operational overhead. Verify before procurement. The BEIR benchmark figures and vendor pricing are as of June 2026 — verify before procurement that the figures in this guide match the current state of vendor pricing pages and the MTEB leaderboard.