The line item nobody puts in the budget
Watch a coding agent work a long task and you'll notice something odd about the rhythm. It reads a file. It thinks. It runs the test suite. It thinks. It applies the edit it had already decided on three steps ago. It thinks again.
Only some of those "thinks" are reasoning. The rest are the model being re-summoned to confirm what the surrounding program already knows — that the file was read, that the tests failed, that the edit goes on line 42. You are paying frontier-model rates for a filesystem call and a diff.
This isn't a bug in any one product. It's the default shape of the agent loop as it's been built so far: every step is a full inference pass, and every pass re-ingests the entire conversation to date. Postman's team recently framed this as an execution-runtime problem rather than a prompting problem, and shipped tooling aimed at taking non-reasoning work out of the model's path (details here). Whatever you make of any single vendor's answer, the diagnosis is right — and it generalizes far beyond coding assistants.

Figure: Editorial context image from the source page (preview only; rights remain with the publisher) · Credit / further reading: dev.to
Where the money actually goes
Break an agent trajectory into phases and the cost distribution is rarely what teams expect. The expensive part is usually not the clever planning step. It's the tail: the forty tool round-trips, the re-serialized context, the retry after a malformed tool response, the second and third read of the same 800-line file because the agent forgot it already had it.
Three mechanics compound here.
Context is re-sent, not remembered. Each turn ships the whole transcript back to the model. As the transcript grows, every additional step costs more than the one before it, even when the step itself is trivial.
Deterministic work is routed through a stochastic interface. Listing a directory, running a linter, applying a known patch — none of this needs a language model. But in the default loop, these actions are parsed, decided, and confirmed by one.
Tool output is unbounded. A test suite that dumps 4,000 lines of log into the context window is a silent tax on every subsequent turn. Nobody approved that spend; it just happened.
The architectural correction: split the planes
The fix that's emerging across the ecosystem is less a feature than a boundary. Call it the reasoning plane and the execution plane.
The reasoning plane is where the model earns its price: interpreting intent, choosing a strategy, recovering from ambiguity, deciding when to stop. The execution plane is a runtime that does deterministic things deterministically — sandboxed file access, test execution, structured tool results, context compaction, cached prefixes, sub-agent isolation so a noisy subtask never pollutes the main transcript.
Once you draw that line, several things that felt like prompt-engineering problems become infrastructure problems. Test output gets summarized before it reaches the model, because a runtime can do that. Repeated file reads collapse into a single cached handle. Long exploratory side quests run in a child context that returns a paragraph, not a megabyte.

Figure: HEXOPS original illustration inspired by the topic (not a copy of the source article imagery) · Credit / further reading: dev.to
The strategic read: this layer is going to commoditize fast. Model providers already sell prefix caching and prompt caching; gateway vendors are selling execution runtimes; frameworks are shipping context managers. Whatever advantage you get from a smarter loop will be table stakes within a couple of quarters.
The counterpoint: cheap loops can be wrong loops
Before anyone rewrites their agent stack around cost, the honest objection.
Optimizing the loop can hide a quality regression. If you cache aggressively and never re-verify, you get a cheaper agent that is confidently operating on stale assumptions. The failure mode is quiet — fewer tokens, same-looking logs, worse answers.
Abstraction has a carrying cost. A bespoke execution runtime is another system to debug at 2am. For a team running a short-lived internal tool, the engineering hours may cost more than the tokens ever will.
Some redundancy is load-bearing. Re-reading a file after an edit, re-running a test after a patch — that's not waste, that's verification. The trick is distinguishing verification the model chose from verification the harness forced. Only the second category is free money.
And the deepest risk: making the wrong thing cheaper. If your agent's success rate on real tasks is 60%, a 40% cost reduction buys you more of the same disappointment, faster. Fix the outcome first, then the invoice.
A 30-day plan that doesn't require a rewrite
Week 1 — Instrument before you optimize. Log token spend per tool call and per phase, not per session. Label each step as reasoning (required model judgment) or deterministic (the answer was already known). Most teams find 50–70% of steps land in the second bucket; the exact number matters less than seeing your own shape.
Week 2 — Cut the obvious bleed. Truncate and summarize verbose tool output at the boundary. Batch file reads into one call instead of five. Turn on prefix caching where your provider offers it. Cap sub-agent output length. These are hours of work, not sprints.
Week 3 — Move deterministic sequences into code. Where the agent reliably performs the same three-step dance (read → run → patch), write one script and let the agent call it once. Isolate exploration in a child context that returns a summary. Give the model structured results, not raw streams.
Week 4 — Put a quality gate on every cost change. Build a small eval set of 20–30 real tasks from your own backlog. Any optimization that touches the loop gets measured against it, with a spend ceiling per task and an alert when it's breached. Cost work without an eval is just hoping at scale.
Key takeaways
- In most long agent runs, a large share of inference spend goes to steps that require no reasoning at all.
- The core mechanics are context re-sending, deterministic work routed through a stochastic interface, and unbounded tool output.
- The emerging architectural response is a hard boundary between a reasoning plane and an execution plane.
- That runtime layer will commoditize quickly — treat it as infrastructure, not as differentiation.
- Cost optimization without an eval set is how teams ship faster, cheaper, wrong answers.
- Verify first, then economize: measure per-phase spend, cut obvious bleed, restructure, then gate everything on quality.
Next steps
- Pull one week of agent traces and tag every step as reasoning or deterministic.
- Pick the single largest deterministic bucket and script it this sprint.
- Add output truncation and prefix caching at the tool boundary.
- Stand up a 20-task eval set from your own backlog before any further tuning.
- Set per-task spend alerts so regressions surface as noise, not as a quarterly surprise.
Sources
- Postman / Dev.to: Why is your agent paying for inference to read the next file?



