Groq vs Cerebras vs Together AI (2026): Fast LLM Inference Real-Cost Comparison

Fast inference is sold on throughput — tokens per second the model can produce. Higher tok/s = lower end-to-end latency for the user (especially on long generations) and higher achievable RPM/TPM ceilings at the same hardware budget.

**Cerebras WSE on Llama 3.3 70B**: ~2,200 tok/s — the fastest publicly available 70B inference on the market. The wafer-scale architecture removes the inter-chip communication overhead that limits GPU clusters. For long generations (>500 output tokens), Cerebras finishes 5-10x faster than the next-fastest provider end-to-end.

**Groq LPU on Llama 3.3 70B**: ~275 tok/s — extremely fast by industry standards, ~5-10x faster than typical GPU-based inference, but ~8x slower than Cerebras. The LPU architecture's strength is cost-per-token efficiency, not absolute speed; Groq is the throughput-per-dollar winner.

**Together AI on H100/H200 (Llama 3.3 70B Turbo)**: ~120-180 tok/s depending on cluster configuration. Slower than the dedicated-chip providers but competitive with hyperscaler GPU inference. Together's bet is on model catalog breadth and fine-tuning flexibility, not raw speed.

**For context**: typical OpenAI gpt-4o output speed is ~80-120 tok/s. Anthropic Claude Sonnet 4.6 is ~60-100 tok/s. Both Groq and Cerebras meaningfully exceed hyperscaler defaults; Cerebras dwarfs them.

**Throughput vs latency**: tok/s measures sustained output speed. TTFT (time to first token) is a separate dimension — Cerebras and Groq both have sub-200ms TTFT on cached prompts; Together is typically 300-500ms. For voice agents and real-time UX, both numbers matter.

Throughput is the headline; cost-per-token is the operating expense. Standard worked example — a customer-service RAG agent producing 100,000 responses/month at avg 500 input tokens + 200 output tokens = 50M input tokens + 20M output tokens per month.

**Groq Llama 3.3 70B**: 50M × $0.59/1M + 20M × $0.79/1M = $29.50 + $15.80 = **$45.30/month**. Cheapest by a meaningful margin.

**Cerebras Llama 3.3 70B**: 50M × $0.85/1M + 20M × $1.20/1M = $42.50 + $24 = **$66.50/month**. ~47% more than Groq but ~8x faster throughput = far better UX on long generations.

**Together AI Llama 3.3 70B Turbo**: 50M × $0.88/1M + 20M × $0.88/1M = $44 + $17.60 = **$61.60/month**. Comparable to Cerebras on price, slower throughput, but the largest model catalog if you need Llama-Vision or DeepSeek V3 or one of the hundreds of other OSS models.

**Reference point — OpenAI gpt-4o-mini equivalent**: 50M × $0.15/1M + 20M × $0.60/1M = $7.50 + $12 = $19.50/month. Cheaper than all three fast-inference providers — because it's a smaller proprietary model, not Llama 70B. The fast-inference value prop is 'comparable quality to gpt-4o at 3-10x the speed,' not 'cheaper than gpt-4o-mini.'

**Verdict on cost**: Groq is the cheapest 70B-class inference. Cerebras is a 47% premium for ~8x the speed — worth it for latency-critical UX. Together is competitive on price with a much broader model selection.

**Groq** runs a curated catalog of ~15-20 models in 2026, deliberately narrow. Llama 4 Maverick, Llama 4 Scout, Llama 3.3 70B, Llama 3.1 8B, DeepSeek R1 Distill (Llama 70B), Mixtral 8x22B, Qwen 2.5 32B, a handful of others. Groq's strategy: pick the best OSS models, optimize them deeply on LPU silicon, ship them at scale. If your target model isn't on Groq's list, it isn't coming.

**Cerebras** runs an even narrower catalog — ~10-12 models. Llama 4 Maverick and Scout, Llama 3.3 70B, Llama 3.1 8B, DeepSeek R1 distills, a few others. Same strategy as Groq, narrower execution. Cerebras's bet is that the top OSS models cover 95% of production demand and that being the fastest at those models is more valuable than being competitive at everything.

**Together AI** runs a massive catalog — 100+ models across all major OSS families. Llama (all sizes), Qwen, DeepSeek (V3, R1, V3.5), Mixtral, Mistral, Gemma, code-specialized models (Qwen-Coder, DeepSeek-Coder), vision-language models (Llama-Vision, Qwen-VL), embedding models, and the long-tail of community fine-tunes. If a notable OSS model exists, Together probably hosts it.

**Trade-off**: Groq/Cerebras give you the fastest serving of the most common models. Together gives you any model you might want, at competitive but not best-in-class speed. For teams committed to a specific Llama or DeepSeek SKU, Groq/Cerebras win. For teams that experiment with model selection or need code/vision/embedding models alongside text, Together wins.

**Fine-tuning**: Together supports fine-tuning + deployment of custom models. Groq does not (as of June 2026). Cerebras offers limited custom-model deployment via enterprise contracts only. If you need to fine-tune and serve a custom model with sub-second TTFT, Together is the only viable option of the three.

Fast inference matters most for use cases where total response time is the UX. Three canonical workloads:

**Voice agents** (paired with TTS like Cartesia or ElevenLabs Flash). User speech → STT → LLM → TTS → audio response. End-to-end target: <500ms for natural conversation. LLM contribution to that budget is typically 100-300ms. **Cerebras at 2,200 tok/s** generates a 50-token response in ~23ms; **Groq at 275 tok/s** in ~180ms; **Together at 150 tok/s** in ~330ms. For voice agents, Cerebras is the only option that leaves meaningful budget for STT + TTS + network. Groq is borderline acceptable. Together is too slow for high-quality voice UX.

**Search re-ranking with LLM-as-judge**: typical pattern — vector search returns top-100 candidates, LLM re-ranks to top-10 with brief explanations. 100 documents × ~50 output tokens each = 5,000 output tokens per query. Cerebras: ~2.3 seconds. Groq: ~18 seconds. Together: ~30+ seconds. **Cerebras is the only provider fast enough to make LLM-reranked search feel snappy.**

**Code completion**: IDE completion needs <300ms TTFT to feel responsive. Output length is short (10-50 tokens). All three providers can hit this target; the differentiator is consistency under load. Groq and Cerebras have tighter p95 latency than Together because dedicated-chip silicon doesn't suffer the variance that GPU clusters can show under spiky load.

**RAG synthesis** (long context in, structured response out): typical pattern — 8k-20k tokens of retrieved context + ~500 token response. Time-to-first-token dominates here (the long input takes time to process). Cerebras and Groq both have aggressive prompt caching that brings TTFT to <500ms on repeated contexts. Together's TTFT is competitive when prompt caching is enabled.

Throughput-per-request is one number; sustained throughput across many concurrent requests is another. Production teams hit rate limits at scale, and the ceiling differs sharply across providers.

**Groq production tier (GroqCloud)**: published rate limits are model-dependent but generally generous for paying customers — Llama 3.3 70B at ~6,000 RPM and ~600,000 TPM on the standard tier, with custom tiers available for higher caps. The LPU architecture's serving efficiency means Groq can absorb large traffic spikes without queueing.

**Cerebras production tier**: published rate limits are conservative as of June 2026 (recently expanded but still notably tighter than Groq). Llama 3.3 70B at ~600 RPM on the developer tier; custom enterprise tiers required for high-throughput production workloads. Cerebras is built for low-latency per request, not maximum aggregate throughput — the wafer-scale architecture has fewer parallel serving units than a GPU cluster.

**Together AI production tier**: rate limits scale with the customer's paid tier and can be very high on dedicated-cluster deployments. Serverless tier limits are mid-range. Dedicated cluster (you reserve specific GPUs for your workload) gives you predictable throughput at predictable per-hour cost — useful for high-volume production where serverless rate limits would queue your traffic.

**Verdict on scaling**: Groq scales serverless better than Cerebras at high volume. Together's dedicated cluster option gives you the most predictable production capacity. Cerebras is best for moderate-volume, latency-critical workloads where each individual request must be fast but aggregate RPS isn't extreme.

**Groq enterprise**: published SLA on GroqCloud production tier (99.9% target uptime), enterprise support, custom rate limits, and contractual data-handling agreements. No on-prem deployment offered as of June 2026 — Groq is cloud-only. SOC 2 Type II compliance.

**Cerebras enterprise**: enterprise tier with custom SLAs, on-prem deployment available for large customers (full Cerebras CS-3 systems can be deployed on customer premises — the only fast-inference provider with this option as of 2026). SOC 2 Type II. The on-prem option is significant for healthcare, finance, and government customers who can't run inference in shared cloud environments.

**Together AI enterprise**: standard cloud SLAs, dedicated cluster deployments (your reserved GPU pool in Together's cloud), HIPAA-compliant tier, SOC 2 Type II, enterprise support. No on-prem option in the standard product line.

**Data handling**: all three offer 'we don't train on your data' contractual guarantees on paid tiers. Free/dev tiers have varying terms — read each provider's data-handling policy if you're sending sensitive data even during evaluation.

**Verdict on enterprise**: Cerebras is the standout for on-prem (the only option of the three). Groq is the simplest cloud-only enterprise story. Together's dedicated-cluster option gives enterprise customers predictable performance + cost with HIPAA compliance for healthcare workloads.

Voice-first customer support agent — 10,000 calls/day, avg 3 minutes, Llama 3.3 70B for the LLM reasoning. Latency-critical (>500ms LLM response time breaks the UX). 30M input + 12M output tokens/month.

**Cerebras**: 30M × $0.85/1M + 12M × $1.20/1M = $25.50 + $14.40 = **$39.90/month**. 2,200 tok/s output = response in ~50ms for typical 110-token reply. The only provider that comfortably leaves budget for STT, TTS, and network round-trips inside a 500ms total response target.

**Groq**: 30M × $0.59/1M + 12M × $0.79/1M = $17.70 + $9.48 = **$27.18/month**. 275 tok/s = response in ~400ms — borderline acceptable for voice but feels slow on longer replies. Cheaper but worse UX.

**Together AI Llama 3.3 70B Turbo**: 30M × $0.88/1M + 12M × $0.88/1M = $26.40 + $10.56 = **$36.96/month**. 150 tok/s = response in ~730ms — too slow for natural voice UX. Wrong tool for this workload.

**Verdict**: real-time voice → **Cerebras**, hands down. The 47% premium over Groq is justified by the 8x throughput advantage. Voice UX is the workload Cerebras was built for.

High-volume RAG product — 1M queries/day across multiple models (Llama 3.3 70B for synthesis, Qwen-Coder for code questions, DeepSeek V3 for reasoning-heavy queries). Latency target: <2 seconds end-to-end. 5B input + 500M output tokens/month total.

**Together AI**: full model catalog covers all three target models. Llama 3.3 70B Turbo @ $0.88/1M, DeepSeek V3 @ $1.25/1M, Qwen-Coder available. Mixed-model bill: roughly $5,000-7,000/month at this volume. Dedicated cluster deployment available to lock in performance + cost.

**Groq**: catalog covers Llama 3.3 70B and DeepSeek R1 Distill (Llama 70B), but not full DeepSeek V3 or Qwen-Coder. Llama 3.3 70B portion at $0.59 + $0.79 = ~$3,000/month for that share — cheapest per token, but you'd need to use a different provider for the models Groq doesn't host.

**Cerebras**: similar catalog gap to Groq. Llama 3.3 70B and a few DeepSeek distills, no Qwen-Coder or full DeepSeek V3 as of June 2026. ~$5,500/month for Llama portion.

**Verdict**: multi-model high-volume RAG → **Together AI** for catalog breadth + dedicated-cluster reliability. Groq is cheaper per token but catalog gaps force you to multi-provider, adding integration complexity. Cerebras is overkill on speed for non-latency-critical RAG.

Code completion product serving 1,000 active developers, ~500 completions per dev per day = 500k completions/day. Avg 200 input + 30 output tokens. Latency target: <300ms TTFT. Model: a code-specialized OSS model (Qwen 2.5 Coder 32B or similar).

**Together AI**: Qwen 2.5 Coder 32B hosted at ~$0.20/1M in/out. 100M input + 15M output tokens/day × 30 days = 3B input + 450M output tokens/month. Cost: $600 + $90 = **$690/month**. Acceptable TTFT at moderate load.

**Groq**: similar models available (Llama 3.1 8B for fast small-model completion, or larger options). Llama 3.1 8B @ $0.05/1M in, $0.08/1M out = $150 + $36 = **$186/month**. Fastest TTFT of the three. Smaller model may underperform on hard completions vs a 32B coder model.

**Cerebras**: Llama 3.1 8B available, similar pricing to Groq. Throughput advantage doesn't matter for 30-token completions; cost roughly comparable.

**Verdict**: IDE code completion → **Groq with Llama 3.1 8B** for the cost-efficient small-model story, or **Together with Qwen 2.5 Coder 32B** if completion quality matters more than cost. Cerebras's speed advantage is wasted on short completions.

**Mistake 1: paying for Cerebras speed when the workload doesn't need it.** Cerebras's 2,200 tok/s is transformative for voice agents and LLM-as-judge re-ranking. For batch RAG, document processing, or async workflows where 5-second latency is fine, Groq at half the speed and 47% lower cost is the right answer.

**Mistake 2: picking Together for raw speed.** Together AI's value prop is catalog breadth + fine-tuning + dedicated deployments — not best-in-class throughput. If raw speed is the priority, Groq or Cerebras win. If model selection or custom deployment matters, Together wins.

**Mistake 3: ignoring rate limits during evaluation.** A model that runs at 2,200 tok/s for single requests in a dev test can queue under production load if the provider's RPM/TPM ceiling is lower than your traffic. Verify your target RPS against the provider's published limits before committing.

**Mistake 4: locking into a provider before benchmarking quality on your prompts.** Same OSS model (Llama 3.3 70B) can show subtle differences across providers due to quantization choices, sampling defaults, and inference optimizations. Run your actual prompts against each provider's serving endpoint — output quality may differ even when the model name is identical.

**Mistake 5: ignoring prompt quality.** Whichever inference provider you pick, prompt quality determines most of the output quality. Fast inference of a bad prompt is a fast bad answer. Our [AI prompt generator](/) writes inference-tuned system prompts that work across Groq, Cerebras, and Together — same prompt structure, swap the API endpoint.

Pricing in this guide is sourced as follows. **Groq**: groq.com/pricing/, fetched 2026-06-20. Groq's pricing has trended down over 2024-2026 as LPU manufacturing scaled — Llama 70B output token cost fell from ~$1.40/1M in early 2024 to ~$0.79/1M in 2026. Free dev tier has remained generous through that period.

**Cerebras**: cerebras.ai/inference, fetched 2026-06-20. Cerebras launched public inference in late 2024 at premium pricing ($0.95/1M out on Llama 70B) and has come down to current $1.20/1M out — still a premium over Groq but the throughput-per-dollar gap has narrowed as the product matured. Developer tier at $50/mo introduced in 2025.

**Together AI**: together.ai/pricing, fetched 2026-06-20. Together's pricing model is heterogeneous — each model in the catalog has its own per-token pricing, with Llama 70B Turbo at $0.88/1M in/out as the reference point. Dedicated cluster deployments priced per-GPU-hour, useful for predictable production workloads at scale.

**Throughput benchmarks**: vendor-reported figures cross-checked against artificialanalysis.ai/leaderboards/providers — a third-party benchmarking site that measures real-world throughput and TTFT across inference providers. June 2026 measurements: Cerebras Llama 70B ~2,200 tok/s, Groq Llama 70B ~275 tok/s, Together Llama 70B Turbo ~150 tok/s.

**Live-verify before procurement**: open each vendor's pricing page and confirm per-million-token rates, rate limits, and any volume-discount tiers match this guide. Fast-inference pricing has moved more than text LLM pricing over 2024-2026 — verify current rates before committing to volume contracts.