Cutting LLM API Costs by 70% Without Rewriting Your App: An FAQ
A developer on Dev.to recently described watching a side project's monthly LLM bill land at around $208, then spending three months bringing the same features down to roughly $60 — no retraining, no rewrite, no visible drop in output quality. Most of the change came from how requests were shaped, not from the application's architecture. Below is the practical version, framed as the questions teams actually ask when the invoice arrives.
Why do LLM bills grow faster than user numbers?

Figure: Editorial context image from the source page (preview only; rights remain with the publisher) · Credit / further reading: dev.to
Short answer: because you pay per token, not per user, and token count per request tends to inflate silently.
Cost is the product of four things: input tokens, output tokens, number of calls, and price per token for the model you chose. Only the third scales with users. The other three drift upward on their own — conversation history that gets resent in full on every turn, system prompts assembled from copy-pasted blog snippets, no cap on completion length, and retry loops that fire a second full-price request whenever the first stalls.
A user who sends three messages in a chat session is not three requests' worth of cost. They are three requests, each carrying a transcript that grows every turn. Quadratic-ish growth hides inside what looks like linear product usage.
The FAQ
1. What actually drives the bill?
Input tokens, output tokens, call volume, retries, and model tier — in that order of typical waste. Output tokens usually cost several times more per token than input, so an uncapped completion length is one of the most expensive defaults in a codebase.
2. Can you really get savings without touching code?
Mostly, yes — but "without touching code" really means "without changing the architecture." In practice the levers live in three places: environment configuration (model IDs, token caps, timeouts), prompt templates (which are text, not logic), and provider-side controls like caching and budget alerts. That is editing, not re-engineering.
3. How much of a typical request is wasted context?
Often the majority. Old turns, stale retrieved documents, duplicated instructions, and boilerplate that repeats on every call all bill as input tokens. A useful habit: print one real request payload in full and read it as if you were paying for it by the word. Most teams find at least a third of it is removable without hurting quality.
4. What is prompt caching and why does it matter?
If your request begins with a long, stable block — a system prompt, tool definitions, reference documentation — several providers let you cache that prefix so repeat requests bill it at a reduced rate. The design rule is simple: put the stuff that never changes at the front, put the per-request variables at the end. Anything that shifts position breaks the cache hit.
5. Does a cheaper model always mean worse output?
No — but mixed fleets do. A large share of production calls are classification, extraction, summarisation, or formatting. Those tasks often survive on a small, fast model. Reserve the expensive tier for genuine multi-step reasoning and route everything else down. The engineering work is a routing function plus an evaluation set, not a rewrite.

Figure: HEXOPS original illustration inspired by the topic (not a copy of the source article imagery) · Credit / further reading: dev.to
6. How do you cut output tokens?
The same way you cut any unbounded resource: cap it. Set a max_tokens that reflects the longest reasonable answer, request structured output instead of prose when a downstream system is consuming it, add stop sequences, and remove instructions that invite essays ("explain in detail", "be thorough"). Shorter outputs are also usually better outputs for product UI.
7. How do retries quietly inflate spend?
A timeout followed by an automatic retry means you paid twice for one user action, and the first call may still have completed server-side. Add exponential backoff, cap the retry count, and make sure idempotent paths don't re-fire on transient errors that would have resolved themselves. Streaming connections that hang are a common culprit.
8. Which metrics should you watch weekly?
Five are enough to start: cost per request, cost per active user, input-to-output token ratio, cache hit rate, and p95 latency. If cost per active user is flat while cost per request falls, you have a usage-growth problem, not an efficiency problem — different fix.
9. What should you optimise first?
In order of effort-to-impact: cap output length, delete redundant system-prompt text, window or summarise conversation history, cache the stable prefix, route simple tasks to a smaller model, then batch anything that isn't user-facing. Most teams get the bulk of the savings from the first four.
10. Does trimming context hurt quality?
Sometimes — which is why you measure instead of guessing. Keep a small set of real prompts with known-good answers, run it before and after each change, and compare. Without that harness, every optimisation is a coin flip, and the team reverts everything after the first bad support ticket.
11. Do streaming and batching change cost?
Streaming improves perceived latency but doesn't reduce tokens — it can even increase them if users abandon mid-response. Batching is the real lever for non-interactive work: nightly summarisation, backfills, and re-scoring can often move to a cheaper batch tier or an off-peak window.
12. How do you stop savings from eroding?
Make cost visible where changes happen. A per-feature spend dashboard, a budget alert at a threshold you'd actually act on, and a short checklist in code review (token cap set? context trimmed? model tier justified?) do more than any one-off cleanup. Costs regress the moment nobody is looking.
Key takeaways
- LLM spend scales with tokens per request, not just users — history resends and verbose prompts grow silently.
- The biggest wins are usually configuration and prompt text: output caps, trimmed system prompts, history windows, cached prefixes.
- Route by task difficulty. Most production calls are simple and don't need the most expensive model.
- Retries and timeouts are a hidden second invoice; treat them as a cost surface, not just a reliability feature.
- Optimise against a small evaluation set, or you'll trade quality for savings you can't defend.
Next steps
- Pull last month's usage breakdown by feature and by model.
- Log one full request payload per feature and mark what's removable.
- Set
max_tokensand timeouts everywhere they're missing. - Build a 20–50 prompt evaluation set before changing model tiers.
- Add cost per active user to your weekly dashboard and a budget alert.



