How to Reduce Token Usage in Prompts

Before trimming anything, find out where your tokens are going. In most production prompts the cost is not the user's question — it's the **standing context**: a giant system prompt, few-shot examples you no longer need, retrieved documents that are only partly relevant, and conversation history that grows every turn. That context is re-sent on every single call, so a 2,000-token system prompt used a thousand times a day is two million input tokens before anyone types a word.

Count tokens, don't guess. Roughly four characters of English equal one token, but use a real tokenizer for anything you're billing against. The shape of the bill tells you where to aim: if input tokens dwarf output, optimize context and caching first; if output dominates, cap length and tighten format.

For the economics behind this, see what is a context window and cost per token across all major models. The goal of the audit is to spend your effort on the 80% of tokens that come from 20% of the prompt.

The fastest win is deleting words that don't change the output. Verbose politeness ("I would really appreciate it if you could possibly..."), restated instructions, and duplicated rules add tokens without adding information. Replace prose rules with terse directives and bullet points; models follow a crisp imperative as well as a paragraph, for fewer tokens.

Be surgical, though — trimming is not the same as starving the model. Removing a worked example that was load-bearing, or a constraint that prevented a class of errors, costs you in quality and rework. The rule is to cut redundancy and keep signal: anything that demonstrably changes the output stays. The complete guide to prompt engineering covers which elements earn their place.

For long retrieved context, don't paste whole documents — summarize or chunk and rank, sending only the top passages. In RAG pipelines this is the single biggest input-token lever; what is RAG explains the retrieve-then-rank pattern. And on the output side, ask for exactly what you need: "Return JSON with these three fields" produces far fewer tokens than "explain your reasoning and then give the answer."

If you send the same long prefix on many calls — a system prompt, a style guide, a fixed set of few-shot examples, a tool catalog — **prompt caching** lets the provider store that processed prefix and reuse it, so you are not paying full price to reprocess identical tokens every time. For high-volume, repeated workloads this is usually the largest cost reduction available, with the bonus of lower latency.

The pattern is to put the stable content first and the variable content (the user's actual request) last, so the cacheable prefix is as long as possible. Several providers expose caching; see Anthropic's prompt caching documentation for how it is priced and the minimum cacheable length, and check provider pricing pages for current cache read/write rates: OpenAI, Google Gemini.

Caching does not reduce the token count of a single one-off prompt — its payoff is reuse. If your prefix changes on every call, restructure the prompt so the changing part moves to the end. For a deeper treatment of caching strategies, see LLM caching strategies: prompt, KV, semantic.

Not every task needs the flagship. Classification, extraction, short rewrites, and routine formatting run well on the fast, low-cost tiers — Anthropic's Claude Haiku 4.5, Google's Gemini 3.5 Flash, OpenAI's smaller GPT-5.5 tiers, or open-weight models like Llama or Mistral that you can host yourself. Reserve the premium reasoning models for genuinely hard, multi-step work.

A practical architecture is a **router**: a cheap model (or a simple rule) first classifies the request, then sends only the hard cases to the expensive model. This caps spend on the long tail of easy requests while preserving quality where it matters. For choosing tiers, see how to choose an AI model 2026 and best AI chatbots compared 2026.

Because per-token prices move, treat any specific rate as something to verify rather than memorize — check the live pages: OpenAI pricing, Anthropic pricing, Gemini pricing, Mistral, DeepSeek. Open-weight models like Llama shift the cost from per-token fees to your own compute.

Here is a typical bloated prompt that re-sends a full document, restates rules, and invites a chatty answer:

``` Hello! I hope you're doing well. I have a really important task and I'd love your help. Below is our entire 4,000-word product manual. Please read all of it carefully. Now, based on the manual, can you please tell me, in a nice friendly detailed paragraph with your reasoning, what the return window is? Thank you so much! [...4,000-word manual pasted every call...] ```

It pastes the whole manual on every request, repeats courtesy, and asks for reasoning the caller doesn't need. The slimmed version retrieves only the relevant passage and caps the output:

``` SYSTEM (cacheable, sent once and cached): You answer policy questions from the provided excerpts only. If the excerpt doesn't contain the answer, say "Not specified." Reply with the value only, no preamble. USER: Excerpt: "Returns are accepted within 30 days of delivery for unused items." Question: What is the return window? ```

Same answer, a fraction of the tokens: the static rules are cached, only the relevant excerpt is sent instead of the full manual, and the output is one line instead of a paragraph. For high-volume systems, layer this with model routing and caching as described in LLM caching strategies.