How to — cut your LLM bill without switching models

Your token spend keeps climbing, and the obvious fix — swap in a cheaper model — is the one you should try last. In a live AI feature, most of the waste isn't the price per token. It's paying full price for tokens the provider has already seen, and generating output tokens nobody asked for.

Work the list in this order. The first three moves usually move the number more than a model swap would, and none of them cost you quality.

1. Find out where the tokens actually go.

Before changing anything, get the per-feature token counts out of your own logs: input tokens, output tokens, and — where the provider reports it — the share of input that came from cache. Almost every team that does this finds the same thing: one feature is responsible for most of the bill, and it isn't the one they suspected. Usually it's a background job that re-sends a large document on every row, or an assistant that carries the entire conversation history into every turn. You cannot cut what you haven't measured, and the provider's usage numbers are already sitting in your logs.

2. Stop paying full price for a prefix you send every time.

Every request to a modern API is split into a stable front part — system instructions, tool definitions, reference documents — and a variable tail: the actual question. Providers now cache that front part and charge a fraction for the reuse. Anthropic's cache hits on Claude Opus 5 cost $0.50 per million tokens against $5 for fresh input, a 90% discount; OpenAI advertises cached input discounts of up to 90% on supported models.

The catch is order. Caching matches from the beginning of the prompt, so anything that changes must come last. One timestamp, one reordered tool list, one interpolated user name near the top and every request is a miss. Two more details decide whether this works at all: there's a minimum length before a prefix is cacheable at all (1,024 tokens on current OpenAI models), and the cache has a lifetime measured in minutes, not hours. Traffic that arrives in bursts hours apart will never hit it. If your requests are genuinely spread out, pay for the longer cache window instead of pretending the short one works.

3. Move the work that can wait off the interactive path.

If a job doesn't need an answer in seconds, it doesn't need the synchronous price. Anthropic's Message Batches API and Google's Gemini Batch API both cut cost by 50%, with most batches finishing inside an hour and a 24-hour outer limit. Evaluations, backfills, moderation sweeps, nightly summarisation, embedding refreshes — all of it belongs here. This is the single largest percentage cut available and it costs you nothing but scheduling discipline. The failure mode is a "background" job that quietly blocks a user-facing request; keep the two queues separate.

4. Attack the output side, not just the input side.

Output tokens are the expensive ones — commonly five times the input rate. Reasoning models make this worse, because they generate a thinking trace before the answer, and that trace bills as output. Two levers: lower the reasoning effort for tasks that don't need it, and cap the verbosity of the answer. A support-triage call that returns one category label does not need a paragraph of deliberation. This is where the "reasoning tokens" line on your bill hides.

5. Fix the context bloat instead of paying for it.

A growing conversation history is the classic silent cost: turn twenty carries nineteen turns of context you mostly don't need, and every one of those tokens bills again. Retrieve less, not more — fewer, better chunks beat a stuffed context window, and a short rolling summary of earlier turns beats the raw transcript. If you're unsure what's actually being sent, the mechanics are worth ten minutes: the "What is a context window?" explainer covers how the window fills and why the tail is what gets dropped first.

6. Only then route — and route on measured cost, not price sheets.

Routing sends simple requests to a cheap model and hard ones to an expensive one, which sounds like a free win and usually isn't. IBM Research published a routing case study where GPT-4.1 looked cheaper on both input and output pricing, then cost $155 across 417 agent tasks against Claude Sonnet's $79 — nearly double — because Sonnet's lower cache-read price and shorter trajectories dominated the sticker price. The lesson generalises: your effective cost per task is the product of pricing, cache behaviour, retry rate, and how many steps the model takes. Measure the whole thing. If you do switch models, you need an eval set that can tell you whether quality moved, and "How to — build a small eval set for your own AI feature" is the cheapest way to get one.

The move not to make.

Don't turn on caching and assume it's working. Cache writes cost more than normal input — roughly 1.25 times the base rate on the short window — so a prefix that changes on every request buys you the write premium and zero hits, which is worse than having done nothing. Watch the cached-token share for a week before you believe the saving. The same scepticism applies to routing: a router that picks models by guessing task difficulty often lands at a higher cost than the baseline it replaced.

How you'll know it worked.

Three checks, in order of trustworthiness. First, the cached-token share in your usage numbers rises and stays up — if it bounces around, your prefix isn't stable. Second, cost per completed task falls while your error and retry rates hold steady; a cheaper request that fails twice is not cheaper. Third, your eval scores don't move. Cut the bill by degrading the answer and you haven't saved anything — you've just moved the cost to your users.

Which line on your AI bill surprised you most when you finally looked — caching, retries, or reasoning tokens? Tell us in the comments.

Sources: Anthropic prompt caching · OpenAI prompt caching guide · Anthropic Message Batches API · IBM Research: Model Routing Is Simple. Until It Isn't. · TNG: How Long Prompts Block Other Requests