Tokens Got 90% Cheaper. Your Bill Tripled. Start Measuring Cost Per Completed Task.

Tokens are cheap now. That was supposed to be the good news.

Per-token pricing has fallen more than 90% since 2023. Gemini 3.1 Flash sits around $0.10 per million input tokens. GPT-4 launched in March 2023 at $30 per million. That is roughly a 99.7% cut in three years.

Enterprise AI spend rose about 320% over the same stretch.

Both of those numbers are true at once. If your finance lead keeps asking why the invoice climbs every month while every vendor announcement says prices are falling, this is why.

Cheap tokens changed what you build

William Stanley Jevons noticed in 1865 that more efficient coal engines did not reduce coal consumption. They increased it, because efficiency made coal worth burning for jobs that were previously too expensive to bother with.

Same thing here. When a token cost real money, you built a chatbot. One turn, a few thousand tokens, done. Now that tokens are nearly free per unit, you build an agent. The orchestrator decomposes the task, picks tools, hits a retrieval system, hands context to a sub-agent, validates the output, retries when validation fails, escalates to a bigger model when the retry fails, then assembles a response.

That is not one call. That is forty. Inference costs fell roughly 1,000x. Demand rose roughly 10,000x.

The unit got cheaper. You started buying vastly more units.

The number nobody is tracking

Most teams we work with can tell you their monthly spend and their per-million rate. Almost none can tell you what one completed task costs.

That is the number that matters. Published figures for coding agents land anywhere between $0.03 and $2.60 per task, and the spread is almost entirely down to how much codebase context gets loaded and how many times the loop runs before it succeeds. An 80x range inside the same tool means the model choice is not your main cost lever. Your loop design is.

Start here:

• Tag every model call with a task ID and a run number
• Log input tokens, output tokens, and cached tokens separately
• Define what "completed" means for each task type before you measure anything
• Report cost per completed task, not cost per call

Retries are where the budget goes

Once you have run numbers attached, look at the distribution rather than the average. In most agent systems the median task is cheap and boring. A small tail of tasks burn ten or twenty attempts, escalate to the expensive tier, and never finish.

That tail is usually most of your bill.

Three things that reliably help:

1. Cap the loop. Three attempts, then stop and route to a human. An agent that fails four times will fail eleven times.
2. Start on the cheap tier and escalate on failure, not the other way round. Most tasks never need the flagship.
3. Cache your context aggressively. Reloading the same repository or the same policy document on every attempt is the single most common waste we find.

None of that is exotic. It is the same discipline you would apply to a retry-happy background job that was hammering your database.

The takeaway

Falling token prices are not a cost strategy. They are a reason your usage grew. Treat inference like any other metered dependency: instrument it, attribute it to a unit of work, and put a ceiling on the loop.

If you cannot answer "what does one successful run of this cost", you are not managing AI spend. You are watching it.

We're here to help founders and teams design and build digital products that are built to scale with you, not slow you down. If you're looking to build something, get in contact with us today!