AI Costs Are the New Budget Line: How to Budget AI Spend

For most of my career, an engineering budget had two lines that mattered. People, and cloud. Everything else — licenses, laptops, the occasional conference — was rounding error. That is no longer true. The third line item on the P&L is AI cost, and in my teams it went from rounding error to a line the CFO asks about by name in less time than it takes to run a hiring cycle.

I want to write about this from the operator’s seat, because most of what I read about AI spend is either vendor marketing or panic. This is neither. These are scars, not slides.

The claim: AI is a cost line, not a tool

We spent the last decade learning to treat cloud as a variable cost that needs engineering discipline: tagging, budgets, reserved capacity, someone who actually reads the bill. AI spend is following the same curve, only faster and with less visibility. It scales with usage, usage scales with how good the tools are, and the tools keep getting better. That is a feedback loop, and feedback loops do not respect quarterly plans.

The mistake I see leaders make is filing AI under “software subscriptions”. A subscription is flat. AI consumption is not. Once your engineers run agents that read whole repositories, generate tests, and review every pull request, you are not paying for a seat. You are paying for compute that your team decides to burn, one prompt at a time.

The scar: I gave my team a Ferrari

I wanted to give my team a Ferrari. The strongest models, no throttling, no friction. My reasoning was sound on paper: engineers are the expensive part, so anything that makes them faster pays for itself. Don’t make people ration the good tools.

What I forgot is that a Ferrari needs more gas. With unlimited access to the most capable models, people used the most capable model for everything — renaming variables, summarising a log file, writing a commit message. Nobody was being careless. The default was the strongest model, so the strongest model is what got used. We burned through the month’s allowance well before the month was over, and the last stretch turned into exactly the rationing I had been trying to avoid, except now it was unplanned and resented.

The lesson was not “give people less”. The lesson was that defaults are policy. If you don’t design the default, the default designs your bill.

Don’t count on the discount

My first instinct after the Ferrari month was to go negotiate. I’ve negotiated with almost all of the big providers at this point, and I’ll save you some time: enterprise discounts are thin. You get better terms, support, and contractual comfort, but you do not get a price that fixes bad usage patterns. Procurement can shave the edges. Engineering has to fix the shape of the curve.

Getting value from this? Subscribe and get the next essay by email.

The mechanism: how the spend actually works now

Pooled tokens change who owns the problem. The new trend is pooled token access: instead of each seat carrying its own allowance, the organisation buys a shared pool and everyone draws from it. That is more efficient — light users subsidise heavy ones — but it moves the management burden onto you. With per-seat plans, the vendor enforces the limit. With a pool, you have to manage how your team is spending it, or one enthusiastic squad drains it for everyone.

Per-seat plans still have a place. For people whose usage is steady and predictable — chat, writing, light coding help — a flat seat is easier to budget and harder to blow up. I now run a mix: seats for the predictable majority, pooled or metered access for the heavy agentic work where the variance lives.

Model choice is a per-task decision. Not every task deserves the strongest model. Architecture questions, gnarly debugging, and long-horizon agent runs do. Formatting, summarising, boilerplate, and classification usually don’t. Treat model selection the way you treat instance sizing: you would never run a cron job on your biggest machine.

Route cheap first, escalate on purpose. The pattern that worked for us is simple routing: send the request to a cheaper, faster model by default, and escalate to the strong model when the task type demands it or the cheap answer fails a check. The important part is that escalation is explicit and visible, not the silent default.

Diagram of model routing: a request passes through a router, takes a thick default path to a cheaper model, and a thin side path escalates to a stronger model only when needed.

Test local models for code review. Code review is high-volume, repetitive, and runs on every pull request. That makes it the most obvious candidate for a local open-source model. The question I keep asking is: what about using a local model for code reviewing? Open models running on your own hardware are now good enough to catch a meaningful share of the obvious issues — style, missing tests, suspicious patterns — and their marginal cost is close to zero once the hardware is there. Run them side by side with your hosted reviewer for a few weeks and compare what each one catches. Let the evidence decide, not the brand.

Visibility before control. You cannot manage what you cannot see. Every team should be able to see its own consumption, broken down by tool and by model, at least weekly. Budgets per team come after visibility, not before. When I set budgets without visibility, people experienced them as arbitrary walls. When teams could see their own curve first, most of them corrected course without being asked.

What this is not

This is not an argument for austerity. The productivity gains are real, and I’d rather overspend on AI than under-invest in it. It is an argument for treating AI spend with the same engineering discipline we eventually learned to apply to cloud. The teams that win will not be the ones that spend the least. They’ll be the ones that know exactly what each unit of spend buys them. I made a related case about measuring real returns in The AI ROI Reality Check, and about engineering the system rather than the agent in The 100x Agent Illusion.

Your Monday action list

Give AI its own line. Pull every AI-related charge out of “software” and into a dedicated budget line you review monthly, next to people and cloud.
Map spend to teams and tools. Find out who is consuming what, through which tool, on which model. If you can’t answer that today, that is the first fix.
Change the default model. Make a capable-but-cheaper model the default in your tools, and make the strongest model an explicit choice.
Split seats from pools. Put predictable users on per-seat plans; reserve pooled or metered access for heavy agentic workloads.
Start one local-model pilot. Run an open-source model on code review alongside your current reviewer for a few weeks and compare findings.
Publish weekly usage to teams. Visibility first. Set per-team budgets only once people have seen their own curve.
Stop expecting procurement to save you. Negotiate, but plan as if the discount won’t change the shape of the bill — because it won’t.

Give your team the Ferrari. Just know where the gas station is, and who is holding the card.

Leave a Reply Cancel reply