How to Reduce GPT-4 API Costs

Prompt caching is the single most effective way to reduce GPT-4 API costs for any workflow with a stable system prompt, repeated retrieved context, or reused few-shot examples. OpenAI's automatic prompt caching prices cached input tokens at 10% of the standard rate. For GPT-4o at $3.00/1M input tokens, cached tokens cost $0.30/1M — a 90% reduction. For GPT-5-mini (priced at approximately $0.40/1M input tokens), cached tokens run $0.04/1M. The savings scale linearly with how often you re-send the same prefix. Full pricing is on openai.com/pricing.

Caching is automatic on OpenAI as of 2025 — no API parameter needed. The model server caches prefixes after 1,024 tokens and keeps the cache active for a sliding window of 5-10 minutes. To maximize cache hits: (a) put the stable content at the top of your prompt, before any dynamic variables; (b) keep the system prompt identical across calls in a session; (c) if you use retrieved documents, prepend them in a fixed order rather than sorting dynamically each call. For agent loops that call the model 10-50 times per session, this alone typically cuts the input-token portion of the bill by 80-90%.

Line-item example: a customer support bot with a 6,000-token system prompt (instructions + knowledge base excerpt) responding to 50,000 tickets per month. Without caching: 50,000 × 6,000 tokens × $3.00/1M = $900/month in system-prompt input tokens alone. With caching (assuming 85% cache hit rate): 50,000 × 6,000 × (0.15 × $3.00 + 0.85 × $0.30) / 1M = $50.25/month. **$849.75 saved per month.** The implementation change is restructuring the prompt so the system message is first and static — roughly 1 hour of work.

The only trap: OpenAI's cache window is session-scoped and time-limited. If calls are spread across hours with no requests in between, you will not get cache hits. For workloads that batch-process overnight with gaps, combine prompt caching with the Batch API (see section 2) and set cache-warming calls at the start of each batch. See the platform.openai.com prompt caching docs for the technical details.

OpenAI's Batch API applies a flat 50% discount to both input AND output tokens in exchange for up to 24-hour processing time. For any workload that does not need real-time responses — overnight content generation, bulk document classification, scheduled report generation, training data annotation, email draft generation — the Batch API is the correct default. There is no change in model quality; you get the exact same model with the same accuracy at half the price.

Current Batch API prices (June 2026): GPT-5 standard is $5.00/1M input and $20.00/1M output in synchronous mode; $2.50/1M input and $10.00/1M output via Batch API. GPT-5-mini is $0.40/1M input and $1.60/1M output synchronously; $0.20/1M and $0.80/1M via Batch. These prices are confirmed at openai.com/pricing. Batch API documentation is at platform.openai.com/docs/guides/batch.

The implementation pattern: build a JSONL file with one request object per line (each containing a custom_id, method, url, and body), submit via POST /v1/batches, store the batch_id, then poll GET /v1/batches/{batch_id} until status is 'completed' and download the output file. The full round-trip is typically 2-4 hours of engineering time. For teams already running scheduled jobs, this often replaces a rate-limited synchronous queue with a single batch submit + cron poll.

Worked example: a marketing team running 1,000 product description rewrites per night at GPT-5 standard (500 tokens input, 300 tokens output). Synchronous: 1,000 × (500 × $5 + 300 × $20) / 1M = $8.50/night = $255/month. Batch API: $4.25/night = $127.50/month. **$127.50 saved per month, $1,530/year, from an afternoon of work.** For teams at higher volumes, the absolute dollar savings scale proportionally. Our Batch API savings calculator lets you run this math for your own token volumes.

The 2026 OpenAI model family spans a price range that most teams dramatically underutilize. GPT-5-nano handles classification, intent detection, routing, simple extraction, and short-form summarization at a fraction of the cost of its larger siblings. GPT-5-mini handles multi-step reasoning, conversational replies, moderate code generation, and data transformation. GPT-5 standard handles complex code, long-form writing, nuanced reasoning, and tool orchestration. GPT-5-pro and o-series handle frontier agentic tasks, advanced reasoning chains, and tasks where quality loss is genuinely unacceptable.

Approximate June 2026 price anchors from openai.com/pricing: GPT-5-nano ~$0.05/1M input, $0.20/1M output. GPT-5-mini ~$0.40/1M input, $1.60/1M output. GPT-5 standard ~$5.00/1M input, $20.00/1M output. GPT-5-pro ~$15.00/1M input, $60.00/1M output. The nano-to-pro spread on output tokens is 300x. Most teams route 100% of traffic to standard or pro and lose 50-80% of their budget on tasks a nano or mini model would handle identically.

The correct approach: define task categories for your product (intent classification, reply generation, code completion, summarization, etc.), run each category against nano/mini/standard in evaluation against your quality benchmark, then lock each category to the cheapest model that meets your threshold. This typically requires 1-2 days of engineering time and immediately delivers 50-70% cost reduction. For a team spending $10,000/month on GPT-5 standard for mixed-complexity tasks, a routing split of 60% nano / 30% mini / 10% standard typically results in a final bill around $1,500-2,000/month — $8,000 to $8,500 in monthly savings.

See our cost per token all major models 2026 reference for the full pricing table across models and providers, which makes the tiering math easy to run before committing to a routing architecture.

Output tokens are more expensive than input tokens on every major model. On GPT-5 standard, output is $20.00/1M vs $5.00/1M for input — a 4x premium. Despite this, most production applications leave max_output_tokens at its default (4,096 or higher), meaning the model can generate up to 4,096 tokens of output even when your application only needs 50 tokens for a routing decision or 200 tokens for a short reply.

The fix is one line of code: set max_output_tokens to the realistic maximum output length for that specific call type. Classification prompts: 10-20 tokens. Short replies: 150-300 tokens. Structured JSON objects: 200-500 tokens. Summaries: 300-800 tokens. Long-form content: 1,500-3,000 tokens depending on target length. Only open-ended generation tasks should be left at 4,096+.

The savings come from two places: (1) the model occasionally generates verbose output when given room to, and capping eliminates token bleed from over-generation; (2) even when the model self-terminates before the cap, setting a low cap speeds up response time and communicates to the API that this call requires only short output, which affects queue prioritization. Teams that audit their token usage logs typically find 15-35% of output tokens are "tail padding" — verbose model explanations and caveats that the downstream application discards. Eliminating that with a 30-minute audit of your prompt library is one of the fastest cost cuts available.

When your application parses the model's response into structured data — extracting fields, routing to downstream systems, populating a database — using OpenAI's structured outputs (response_format with a JSON schema) or function calling forces the model to emit compact JSON rather than natural-language prose. The typical result is 40-60% fewer output tokens, because the model stops explaining its reasoning, adding caveats, or wrapping the answer in conversational framing.

The implementation is straightforward: define a Pydantic model or JSON schema for the expected output, pass it as the response_format parameter (using the json_schema type added in late 2024), and the model returns valid JSON that matches your schema without markdown fences, prose explanation, or token bleed. OpenAI's structured output documentation is at platform.openai.com/docs/guides/structured-outputs.

Worked example: extracting 10 fields from a legal document. Unstructured prompt asking the model to describe what it found: average output 800 tokens at $20/1M = $0.016 per call. Structured output with JSON schema: average output 320 tokens = $0.0064 per call. 60% output reduction. At 200,000 extractions per month, that saves $1,920/month on output alone — plus the downstream parsing is cleaner and error rates drop to near zero because schema violations are rejected before they reach your application. Structured outputs also pair cleanly with prompt caching, since the schema definition in the request body can itself be cached when it is stable.

Most production system prompts contain 20-40% redundant tokens: repeated instructions, verbose explanations of what the model should do, natural-language politeness padding, and multi-paragraph context that could be expressed in one tight bullet list. Prompt compression is the practice of systematically auditing and trimming those tokens without degrading output quality.

The audit process: copy your system prompt, count the tokens (use OpenAI's tiktoken library or the tokenizer playground at platform.openai.com), then rewrite each instruction section to be as dense as possible — bullet points over prose, imperative verbs over explanatory sentences, abbreviations where context makes them clear. After rewriting, run your quality eval suite to confirm output quality is unchanged. Most teams achieve 20-35% token reduction in a 2-3 hour session.

Prompt compression becomes especially powerful in combination with prompt caching (section 1). A compressed 4,000-token system prompt cached at $0.30/1M costs $0.0012 per session read. An uncompressed 6,000-token version of the same prompt costs $0.0018 per read — 50% more. At 100,000 sessions per month, that is $60/month just on cached reads, with no improvement to output quality. Compression also reduces cold-start costs (uncached reads) proportionally. Our AI cost optimization checklist 2026 has a full 17-item protocol that includes prompt compression as step 8, with worked examples.

In multi-turn chat applications, every API call includes the full conversation history in the messages array. A 20-turn conversation with 200 tokens per message is 4,000 tokens of history that the model re-reads on every single call — even though the first 15 turns are mostly irrelevant to answering the current question. At GPT-5 standard input prices of $5.00/1M tokens, that is $0.02 per call just in redundant history.

The fix is conversation history management: instead of passing the raw message array, maintain a sliding window of the N most recent turns (where N is enough to give meaningful context — typically 5-10 turns), and optionally prepend a rolling summary of earlier conversation generated by a cheap nano model. The summary approach preserves full semantic context while replacing 15 turns of raw text with a 3-sentence summary (typically 60-80 tokens vs. 3,000+).

Implementation detail: build a history manager class that tracks all messages but only sends the last 8 turns + a rolling summary to the API. The rolling summary is refreshed every 5 turns using GPT-5-nano at $0.20/1M output tokens — a cost of roughly $0.000004 per refresh. This pattern cuts input token counts 40-70% in long conversation workflows. For a customer support product with average 15-turn sessions at 200 tokens/turn, moving from full history to managed history + summary reduces per-session input from 15 × 200 = 3,000 tokens to roughly 8 × 200 + 100 summary = 1,700 tokens — a 43% cut on every session's input spend.

OpenAI's o-series reasoning models (o3, o4-mini, o4) generate internal 'thinking tokens' before producing their final answer. These thinking tokens are billed at the same output rate as visible tokens but are invisible in the API response — they appear only as reasoning_tokens in the usage object. On complex tasks, o3 may generate 5,000-20,000 thinking tokens before responding, multiplying the effective output cost 3-10x versus a naive estimate based on response length alone.

The cost lever: set reasoning_effort to 'low' or 'medium' instead of the default 'high'. With reasoning_effort=low, the model uses fewer thinking tokens, responds faster, and costs 40-70% less per call. Quality loss on tasks that do not actually require frontier reasoning — structured extraction, classification, summarization, most customer support — is minimal to zero. The reasoning_effort parameter is documented at platform.openai.com/docs/guides/reasoning.

Practical guidance: default new o-series workloads to reasoning_effort=low, run your quality eval, and only escalate to medium/high if the eval shows genuine degradation. For tasks where you chose o-series for reliability rather than raw reasoning depth, low effort usually matches high effort output quality while cutting cost by half. For agentic tasks where the model needs to plan multi-step tool use, try medium first. Reserve high only for math proofs, complex code generation, and tasks where getting it wrong has downstream cost. Combined with the Batch API (50% off), o4-mini at reasoning_effort=low can undercut GPT-5 standard costs while outperforming it on reasoning-heavy tasks.

A large fraction of production LLM calls are effectively lookup operations dressed up as generation: "does this customer message match category A, B, or C?", "is this text similar to any document in our knowledge base?", "does this input match any of these 200 rule patterns?". These tasks can often be replaced by embedding similarity search at 60-95% lower cost.

OpenAI's text-embedding-3-small costs $0.02/1M tokens. A classification call that previously used GPT-4o (200 input tokens + 50 output tokens = $0.67/1K calls at combined input/output rates) can often be replaced by one embedding call ($0.02/1M × 200 = $0.000004/call) plus a cosine similarity lookup against pre-embedded category exemplars. The throughput is higher, latency is lower, and the cost per classification is ~100x cheaper.

This is not always the right trade-off: embeddings do not follow complex instructions, cannot handle multi-step reasoning, and fail on tasks that require genuine language generation. But for routing, filtering, tagging, and FAQ matching — tasks that are currently running on expensive LLMs — the replacement pays for itself in 1-2 days of engineering time. Use the LLM only for the hard cases the embedding classifier fails on (typically 5-15% of volume). This hybrid architecture is documented in detail in our agent loop cost optimization guide.

Agent frameworks that pass tool definitions on every API call are burning significant input tokens that almost never change between calls. A set of 15 tool definitions averaging 200 tokens each is 3,000 tokens per call — re-sent on every single step of the agent loop. On a 20-step agent session at GPT-5 standard input rates ($5.00/1M), that is 3,000 × 20 × $5/1M = $0.30 just in tool definitions, per session, for data that is identical on every call.

The fix: structure your messages array so the tool definitions (passed via the tools parameter or as part of the system message) are at the front of the stable prefix that benefits from prompt caching. When OpenAI's automatic caching kicks in, tool definitions cached at $0.30/1M input rate cost 10% of the $5.00/1M standard rate. That same 3,000-token tool definition block costs $0.0009 per read instead of $0.009 — 90% off. Over 20 steps in a session: $0.018 total in tool definition tokens instead of $0.18. Annualized across 10,000 sessions/month, that is $1,620 saved per year on tool definitions alone.

Agent loop cost optimization is covered in depth in our agent loop cost optimization guide. The combination of tool caching, conversation history management, and prompt caching typically reduces per-session costs 70-80% for agentic workflows. That guide includes patterns for LangChain, CrewAI, and raw OpenAI function calling implementations. For a cross-provider comparison of how these costs stack up against Claude Sonnet and Gemini Flash in agent settings, see our Anthropic vs OpenAI pricing 2026 deep-dive.

Apply these techniques in this order to maximize savings for minimum engineering time. Start with the changes that require the least code change and deliver the most savings.

Week 1 (configuration changes only, no architecture changes): (1) Audit your system prompts and restructure them so stable content is at the front — this activates prompt caching automatically. Measure cache hit rate via the usage.prompt_tokens_details.cached_tokens field in the API response. (2) Set max_output_tokens on every call based on the realistic maximum output for that call type. (3) Enable structured outputs (response_format with json_schema) for any call where you parse the model's response into structured data. (4) Switch any o-series calls from reasoning_effort=high to reasoning_effort=low and re-run your quality eval.

Week 2 (routing changes): (5) Audit your call log and categorize calls by task type. Run each category through GPT-5-nano and GPT-5-mini evals. Route everything that passes quality threshold to the cheaper model. (6) Identify any async workloads (nightly jobs, bulk processing, non-real-time enrichment) and migrate them to the Batch API.

Week 3-4 (architecture changes): (7) Implement conversation history management with a sliding window + rolling summary for any multi-turn product. (8) Identify the top 3 highest-volume LLM call types and evaluate whether embeddings + classifier can replace them. (9) Cache tool definitions by restructuring your agent loop's system message.

Expected cumulative savings: after Week 1, expect 40-60% reduction. After Week 2, expect 65-80% reduction. After Weeks 3-4, expect 75-90% reduction from your starting baseline. Use our AI Prompt Cost Calculator to track the before/after numbers at each stage. If your spend is above $5,000/month, also evaluate self-hosting quantized open models for high-volume nano-tier tasks — but that threshold matters because the TCO breakeven on dedicated inference infrastructure only turns positive at significant volume. Below that, the hosted API path described in this guide is the correct answer.

All prices below are sourced from openai.com/pricing as of June 2026. Use these in your own ROI spreadsheet before deciding which optimizations to prioritize.

GPT-5-nano: $0.05/1M input, $0.20/1M output. Cached input: $0.005/1M. Batch API: $0.025/1M input, $0.10/1M output. Best for: classification, routing, simple extraction, intent detection.

GPT-5-mini: $0.40/1M input, $1.60/1M output. Cached input: $0.04/1M. Batch API: $0.20/1M input, $0.80/1M output. Best for: conversational replies, moderate code, summarization, multi-step QA.

GPT-5 (standard): $5.00/1M input, $20.00/1M output. Cached input: $0.50/1M. Batch API: $2.50/1M input, $10.00/1M output. Best for: complex code, long-form writing, nuanced reasoning, tool orchestration.

GPT-5-pro: $15.00/1M input, $60.00/1M output. Cached input: $1.50/1M. No Batch API. Best for: frontier reasoning tasks, advanced agentic workflows, production code generation at scale.

o4-mini (reasoning): $1.10/1M input, $4.40/1M output + reasoning tokens billed as output. Batch API: $0.55/1M input, $2.20/1M output. reasoning_effort=low reduces thinking tokens by 50-70%.

text-embedding-3-small: $0.02/1M tokens. text-embedding-3-large: $0.13/1M tokens. For most production embedding use cases, small is sufficient and 6.5x cheaper.

All prices confirmed against openai.com/pricing and platform.openai.com on 2026-06-27. For cross-provider comparison including Anthropic Claude Sonnet 4.5 and Google Gemini 2.5 Flash, see our cost per token all major models 2026 reference page.