Groq API Pricing for Llama 3 (2026): Every Model, Real Numbers

Most inference APIs run on NVIDIA GPU clusters — A100s, H100s, or the newer B200s. These chips are general-purpose CUDA accelerators optimized for training as much as inference. Groq designed the LPU (Language Processing Unit) ground-up for inference only: a deterministic, single-chip architecture with on-chip SRAM that eliminates the memory bandwidth bottleneck that throttles GPU inference at high sequence lengths.

The practical result is that Groq's Llama 3.1 70B routinely benchmarks at 250-400 tokens per second per request — compared to 30-80 tokens per second on typical GPU-backed endpoints at providers like Together AI, Fireworks, or Replicate. For Llama 3.1 8B, Groq has measured over 750 tokens per second in production. These numbers matter for real-time applications: a 400-token response takes under a second on Groq vs. 5-12 seconds on a standard GPU endpoint.

The pricing structure reflects a different cost model. Because LPU chips have high utilization efficiency, Groq can pass those savings down without sacrificing margin. The $0.05/$0.08 price point for Llama 3.1 8B is lower than most GPU-backed providers for the same model, while the speed is 10-20x higher. For high-throughput, latency-sensitive workloads, this combination is genuinely rare. See groq.com/pricing for the current rate card and any active promotional tiers.

Llama 3.1 8B Instant is Groq's fastest model: $0.05 per million input tokens and $0.08 per million output tokens, with a 128k context window. At over 750 tokens per second, it's the fastest publicly available endpoint for any Llama 3 model as of mid-2026. This is the tier to use for real-time applications where latency matters more than frontier-model reasoning: autocomplete, classification, structured extraction, streaming chat UI, or any workflow where the user is actively waiting for output.

To put $0.05/$0.08 in context: GPT-4o-mini is priced at $0.15/$0.60 per million tokens — 3x the input cost and 7.5x the output cost for a model with comparable capability on many tasks. Anthropic's Claude Haiku 3.5 lists at $0.80/$4.00 per million tokens. For output-heavy workloads (long-form generation, verbose JSON, chain-of-thought), the output token gap between Groq and the major proprietary providers is especially significant.

The 8B model has limitations: it lacks the reasoning depth of 70B on complex multi-step tasks, and its 8B parameter count means it will underperform on nuanced instruction following compared to larger models. For tasks like sentiment classification, named entity recognition, structured data extraction, or lightweight summarization, the gap is negligible. For code generation or complex reasoning chains, benchmark your specific task before committing to 8B.

Llama 3.1 70B Versatile is Groq's workhorse model: $0.59 per million input tokens and $0.79 per million output tokens, 128k context, running at 250-400 tokens per second. This is where Groq's value proposition is most compelling against proprietary alternatives. You get a model that competes with GPT-4o on many benchmarks (MMLU, HumanEval, MATH) at roughly 10-15% of the price.

GPT-4o is currently priced at $2.50/$10.00 per million tokens. Claude 3.5 Sonnet runs $3.00/$15.00. Groq's Llama 3.1 70B at $0.59/$0.79 isn't just cheaper on input — the output differential is where the real money is for generation-heavy workloads. A workflow generating 1 million output tokens per month pays $790 on Groq 70B vs. $10,000 on GPT-4o. For our full breakdown of these tradeoffs, see the OpenAI API pricing 2026 guide.

The 70B model handles complex instruction following, multi-turn dialogue, code generation, and synthesis tasks well. The 128k context window is large enough for long document analysis. The 250-400 token/second speed means even with large outputs, response times stay under 2-3 seconds for most user-facing applications. If you're evaluating whether to stay on a proprietary model or move open-weight workloads to Groq, the 70B is the right starting point for benchmarking.

Llama 3.3 70B Versatile is Meta's improved release on the same 70B architecture, priced identically to 3.1 70B: $0.59 input / $0.79 output per million tokens on Groq. The model shows meaningful improvements in instruction following, code quality, and multilingual performance vs. Llama 3.1 70B, while running at slightly faster throughput on Groq's LPU (280-420 tokens per second in published benchmarks).

For new projects starting in mid-2026, Llama 3.3 70B Versatile is generally the better choice over 3.1 70B unless you have specific compatibility requirements or are running established benchmarks on 3.1. The pricing is identical, so there's no cost reason to stick with the older version. Groq offers both simultaneously so you can A/B test within the same billing account.

Llama 3.3 70B particularly improved on coding tasks — HumanEval pass@1 is notably higher than 3.1 70B. If your workload involves code generation, review, or transformation, the 3.3 version is worth testing directly. The model is fully open-weight under Meta's Llama 3 Community License, meaning you can also self-host it if volume grows beyond where Groq makes economic sense. See our self-host vs. API cost breakeven analysis for the crossover math.

Llama 3.1 405B is Meta's largest open-weight release and competitive with frontier proprietary models on many benchmarks. Groq has offered 405B in limited preview; as of June 2026, pricing was approximately $2.00 per million tokens for both input and output (verify current status at groq.com/pricing as availability and pricing may have shifted).

Even at $2.00/$2.00, 405B on Groq undercuts GPT-4o ($2.50/$10.00) on output tokens by 5x — a significant advantage for long-form generation. The context window in preview was 32k, more limited than the 128k available on 70B. Speed is lower too: 50-100 tokens per second vs. 250-400 for 70B, because even LPU efficiency has physical limits at 405B parameter scale.

For most workloads, 405B is overkill. The accuracy gap between 3.3 70B and 405B is measurable but often smaller than the 3-4x cost difference. The cases where 405B makes sense are frontier-level reasoning tasks, complex multi-step agent orchestration, or quality-critical generation where you've benchmarked and confirmed 70B doesn't clear your quality bar. For reference benchmarks and a structured comparison to proprietary frontier models, see our LLM output speed and tokens-per-second guide.

Groq also hosts Mixtral 8x7B (Mistral's mixture-of-experts model) at $0.24 per million tokens flat for both input and output — the same rate for both, which simplifies billing for input-heavy workloads. Mixtral 8x7B is a strong choice for tasks requiring broad knowledge coverage with moderate reasoning depth. The symmetric input/output pricing means you don't need to optimize for output token reduction the way you do with models that have a large input/output price asymmetry.

Gemma 2 9B IT (Google's smaller open model) runs at $0.20/$0.20 on Groq, with 500-700 tokens per second throughput. Gemma 7B IT is priced at $0.07/$0.07 — the second-cheapest option behind Llama 3.1 8B, and the fastest per-dollar option for symmetric workloads. Gemma models tend to excel at instruction following and have strong multilingual performance relative to their size, making them worth testing for classification and extraction tasks before defaulting to a larger model.

The symmetric pricing on Mixtral and Gemma is a practical advantage: many production systems burn more on output than input (because output tokens usually cost 2-4x input on other providers), so the flat rate eliminates that calculation. If your use case generates long outputs from short inputs — summarization, expansion, template filling — check the effective cost per request carefully before assuming Llama 3.1 70B is cheaper overall.

Here's the practical comparison for the most common production tier — a capable mid-size model for general application use — as of June 2026: Groq Llama 3.3 70B at $0.59/$0.79 per million tokens; GPT-4o at $2.50/$10.00; Claude 3.5 Sonnet at $3.00/$15.00; Gemini 1.5 Pro at $1.25/$5.00. For output-heavy workloads (generation, drafting, long-form responses), the output token price is the dominant cost driver.

A concrete example: an application generating 10 million output tokens per month. Groq 70B: $7,900/month. GPT-4o: $100,000/month. Claude 3.5 Sonnet: $150,000/month. The 13-19x cost difference is real and repeatable. The question is whether Llama 3.3 70B is good enough for your specific quality bar — which is an empirical question requiring a benchmark on your actual workload, not a spec sheet comparison.

Where OpenAI and Anthropic win: proprietary model quality on the hardest reasoning tasks, more mature tooling ecosystems (function calling, structured output APIs, code interpreter), more consistent uptime SLAs at enterprise scale, and better multi-modal capabilities. Groq's current model catalog is text-only; for vision, audio, or multi-modal workloads, you're on OpenAI or Anthropic regardless of cost. For the full per-model cost breakdown across all providers, see our cost-per-token comparison for all major models.

Llama 3 models are open-weight — you can download and run them yourself. The question is when Groq's per-token pricing beats the total cost of ownership of your own infrastructure. The math involves GPU rental cost (or capital cost), DevOps time, model loading overhead, batching efficiency, and the opportunity cost of engineering time spent on infrastructure vs. product.

Rough breakeven for Llama 3.1 70B on a dedicated A100 80GB: you need about 2-3 million output tokens per day (60-90M per month) before self-hosting beats Groq's $0.79/M output price, accounting for A100 spot pricing at ~$2-3/hour and 40-60% GPU utilization efficiency in practice. Below that volume, Groq is cheaper than self-hosting even on spot instances. Above it, the math shifts — but you've also taken on model serving infrastructure, quantization decisions, CUDA driver updates, and failover.

For the full breakeven analysis with interactive cost curves, see our self-host vs. API cost breakeven guide. The short version: for teams spending under $5,000/month on Groq, self-hosting almost never wins on TCO. For teams spending $20,000+/month on a single model and a single task, self-hosting deserves a serious evaluation. Groq fits the sweet spot in between: cheaper than proprietary APIs, faster than GPU-backed open-model endpoints, no infra to manage.

Groq offers a free tier with limited requests per day and per minute — specific limits are documented in the Groq developer documentation and change periodically. As of mid-2026, free tier limits are generous enough for development and testing but will throttle production workloads. For production use, the pay-as-you-go tier activates automatically when you add a payment method, with rate limits that scale with your account tier.

Production rate limits on Groq are measured in tokens per minute (TPM) and requests per minute (RPM) per model. The 70B models have lower RPM limits than the 8B model due to the higher per-request compute cost. If you're building a high-concurrency application, check the current rate limit table at groq.com before architecting your concurrency model — hitting rate limits on Groq manifests as 429 errors with retry-after headers, which your client needs to handle with exponential backoff.

Unlike OpenAI and Anthropic, Groq does not currently offer a Batch API with discounted async pricing, prompt caching, or a fine-tuning API. If your cost optimization strategy relies on batch discounts or prompt caching (both of which can cut bills 50-90% at OpenAI and Anthropic), those tools aren't available on Groq today. For workloads where those features matter, check our AI cost optimization checklist to see if the Groq base pricing still wins after accounting for what you'd save from caching elsewhere.

Groq is the right choice when three conditions align: you need low latency (real-time streaming to a user or a tight pipeline SLA), you're comfortable with open-weight model quality (Llama 3.x is good enough for your task), and you want the lowest per-token cost without managing infrastructure. These conditions describe a large share of production AI workloads — real-time chat, streaming autocomplete, classification pipelines, structured extraction, and lightweight summarization.

Groq is not the best fit when: you need multi-modal capabilities (image, audio, video — Groq is text-only as of June 2026); you need fine-tuning on your own data (not available on Groq); you need the absolute quality ceiling of GPT-4o or Claude 3.5 Sonnet for genuinely hard reasoning tasks; or you need enterprise SLAs with contractual uptime guarantees. In those cases, you'll pay more for OpenAI or Anthropic and it may be worth it.

A practical starting point: run your most common production prompt through Groq Llama 3.3 70B and your current model side by side, evaluate the outputs on your quality criteria, and calculate the monthly cost difference using our AI Prompt Cost Calculator. If 70B passes your quality bar (it does for the majority of production text workloads), the cost savings are immediate and require no infrastructure changes — just update the API endpoint and model name in your client code. For a broader view of how fast-output models compare on both speed and cost, see our LLM output speed: tokens per second guide for 2026.

Let's work through three representative workloads to make the pricing concrete. First, a customer-support chatbot handling 100,000 conversations per month, averaging 500 input tokens and 200 output tokens per turn, with 4 turns per conversation: total 200M input tokens + 80M output tokens per month. On Groq Llama 3.3 70B: (200M × $0.59 + 80M × $0.79) / 1M = $118 + $63.20 = $181.20/month. On GPT-4o: (200M × $2.50 + 80M × $10.00) / 1M = $500 + $800 = $1,300/month. The Groq option is 86% cheaper for functionally the same chatbot capability in most support scenarios.

Second, a document summarization pipeline processing 50,000 documents per month at 3,000 tokens input and 400 tokens output: 150M input + 20M output. Groq Llama 3.1 8B (sufficient for summarization): (150M × $0.05 + 20M × $0.08) / 1M = $7.50 + $1.60 = $9.10/month. Even at the 70B model level, it's $88.50 + $15.80 = $104.30/month. Compare to GPT-4o-mini at ($0.15 × 150 + $0.60 × 20) = $22.50 + $12 = $34.50/month — Groq 8B actually wins on price AND speed.

Third, a code generation assistant with 10,000 developer sessions per month, averaging 1,500 input tokens (code context) and 800 output tokens (generated code): 15M input + 8M output. Groq Llama 3.3 70B: $8.85 + $6.32 = $15.17/month. GPT-4o: $37.50 + $80 = $117.50/month. Claude 3.5 Sonnet: $45 + $120 = $165/month. At this volume Groq is 87-90% cheaper — and the 280-420 tokens/second throughput means the developer sees code appear near-instantly rather than watching a cursor blink. Plug your own numbers into our AI Prompt Cost Calculator to generate your exact bill across all providers side by side.

Groq's prices have generally held stable relative to competitors, even as OpenAI and Anthropic have cut rates on their proprietary models. The LPU chip economics are different from GPU economics — Groq's cost floor is less tied to NVIDIA pricing pressure, and more to their own chip manufacturing and data center costs. This means Groq pricing may not track proportionally with GPU-backed provider price cuts.

What to watch in the second half of 2026: Groq has announced plans to expand the LPU chip generation, which should increase throughput ceilings and potentially allow 405B to run at higher token-per-second rates. Groq has also hinted at enterprise tiers with committed spend discounts, which would change the calculus for high-volume users. Prompt caching and batch API features have been requested by the developer community — if Groq ships these, the effective cost would drop significantly for workloads with repeated context.

The broader inference market is compressing margins across all providers. Self-hosted open-weight models on rented GPUs are getting cheaper as H100 and B200 spot capacity expands globally. Groq's sustainable advantage is LPU speed, not just price — the 10x throughput advantage is a moat that GPU price drops can't eliminate. For teams where latency matters as much as cost, Groq's position remains strong regardless of where the per-token price converges.