How I cut my OpenAI API bill 73% in a weekend
Prices in this post are as published in May 2026. For today's numbers, use the cost calculator. Update, 1 Nov 2026: OpenAI retires GPT-5 and GPT-5 Nano on 11 Dec 2026, so apply the same three levers to their successors. Note that OpenAI bills cache writes at 1.25 times the input rate on GPT-5.6 and later.
Route easy requests to a smaller model, cache the long system prompt, and cap output length. On a 600,000-message-a-month classifier those three changes cut the bill from $1,800 to $480 a month with the same accuracy on the eval set.
On this page
What was the workload costing?
The workload was a customer-support intent classifier. About 600,000 messages a month came in. Every message hit GPT-5 with a 4,500-token system prompt (rules, examples, schema) and got back a 60 to 250 token JSON response.
Quick math at GPT-5 list pricing ($1.25 input, $10 output per million tokens). Today's price per 1M tokens for every current model is on our pricing page, and the batch and cache-write rates for each GPT model are under OpenAI API pricing.
- Input: 600k requests at roughly 4,700 input tokens (system plus user) equals 2.82 billion tokens. At $1.25 per million, that's $3,525 a month.
- Output: 600k requests at 150 average output tokens equals 90 million tokens. At $10 per million, that's $900 a month.
Total list price: $4,425 a month. Negotiated enterprise pricing brought it to $1,800, which is still a real number for a feature with a fixed quarterly budget.
How much does prompt caching save on a long system prompt?
The system prompt was 4,500 tokens, and it was identical on every call. That's exactly the shape OpenAI's prompt caching is designed for. Cached input tokens cost 10% of standard input on the GPT-5 family ($0.125 per million instead of $1.25, per OpenAI's pricing page).
Two prerequisites tripped me up the first time. First, the cached prefix has to be byte-identical across calls. Even one timestamped value in the system prompt breaks the cache. I had a {currentDate} placeholder in mine. I pulled it out and reformatted to write the date in the user message instead. Second, you need at least 1,024 tokens of stable prefix. Below that, caching is a no-op.
After fixing both, my cached input cost on the system prompt dropped to $0.125/M instead of $1.25/M. Working it out: 600k requests at 4,500 tokens at $0.125 per million equals about $337.50 a month for the system-prompt portion of input, down from roughly $3,375 at full price. That's about a $3,000 a month cut at list price on its own.
You can model your own savings on the LLM cost calculator. Toggle prompt caching, set system-prompt length, and see what the breakeven looks like. If you're writing the system prompt from scratch, the system prompt builder structures it into role, rules and output format.
Which requests can go to GPT-5 Nano?
70% of traffic was three trivial intents: "where's my order," "cancel," and "reset password." A 4-token classifier could solve those. The other 30% (refunds, escalations, multi-issue threads) actually benefited from GPT-5's reasoning.
So I added a 50-line router. A tiny GPT-5 Nano call (cheap and fast) classifies each message into "simple" or "complex," then routes accordingly. Nano lists at $0.05/$0.40 per million, which is 25x cheaper than GPT-5 on input and 25x cheaper on output.
Reshaped budget after routing:
- 420k simple-intent requests routed to nano. Cost: about $15 a month, all in.
- 180k complex requests stayed on GPT-5 with the cached system prompt. Cost: about $140 a month.
- 600k tiny router calls (nano, ~50 tokens each). Cost: about $8 a month.
Eval-set accuracy held: 96.2% before routing, 96.0% after. The 0.2% drop came entirely from edge cases the router miscategorized as simple. Adding a single rule (if the message contains refund or chargeback, force it to complex) recovered the gap. If you are choosing the stronger model for that hard 30%, my 200-prompt test of Claude Sonnet 4.6 against GPT-5 breaks results down by task type.
How much does capping output length save?
Output is the expensive half. GPT-5 charges 8x more per output token than per input token. The original prompt let the model generate a JSON response of any length. On complex tickets it was producing 350-token responses with a paragraph of reasoning baked in.
The downstream consumer didn't need the reasoning paragraph. Setting max_tokens=80 and tweaking the prompt to ask for a strict JSON-only response cut average output from 150 tokens to 70 tokens. That's a 53% reduction on the single most expensive line item.
What did the bill drop to?
Final monthly cost after all three changes: about $480 a month, down from $1,800. Same accuracy. Slightly faster average latency, since nano handles the simple intents quicker than GPT-5 did.
The audit trail:
- Prompt caching alone: about 70% input-cost reduction on the slow path.
- Model routing: shifted 70% of traffic to a 25x cheaper model.
- Output cap plus JSON-only: 53% reduction in output tokens.
Stack these together and the savings compound. Each lever individually is a 30 to 60% win. Combined, it's a 73% win.
What would I do differently?
Audit cost before writing the prompt, not three months in. The 4,500-token system prompt was longer than it needed to be. Half of it was example refunds I could have moved into a vector retrieval. If I'd built it caching-aware from day one, the "optimization" would just have been the original architecture.
For new workloads I now run the math in the cost calculator before writing any code, and I count tokens with the token counter on candidate prompts before they ship. Two minutes of math saves three months of bills.
What should you change first?
- Enable prompt caching on any system prompt over roughly 1,000 tokens that's reused more than once a minute.
- Route trivial traffic to a budget tier (GPT-5 Nano at the time; see the model comparison for today's cheapest models).
- Cap
max_tokensaggressively. Output is the expensive half. - Use the cost calculator to model the bill before shipping, not after.
- Prototype on a free tier before you pay for anything: several LLM APIs still have a free tier, with real limits listed.
Sources
Frequently asked
What does OpenAI prompt caching need to work?
Two things: the cached prefix must be byte-identical on every call, so no timestamps or per-request values in the system prompt, and it needs at least 1,024 tokens of stable prefix. On the GPT-5 family cached input cost $0.125 per million instead of $1.25.
Which requests can a smaller model handle?
In this workload, 70% of messages were three simple intents (order status, cancel, reset password). A small GPT-5 Nano router sent those to Nano and kept refunds, escalations and multi-issue threads on GPT-5. Eval accuracy went from 96.2% to 96.0%, and one rule for refund messages closed the gap.
How much does capping max_tokens save?
Setting max_tokens to 80 and asking for strict JSON cut average output from 150 to 70 tokens, a 53% cut on output, which is the most expensive part of the bill because GPT-5 output costs 8 times its input price.
An independent site run by an AI engineer. Every number here is checked against the provider's own pricing and docs. How we verify.