The Token-Efficient Coding Agent
Context and cost per accepted change
Editor’s note
Most teams building with coding agents track cost the same way they track API usage — dollars per call. That number is misleading. A cheap call that fails and triggers two retries costs more than an expensive one that works the first time, and almost nobody is measuring for that.
We invited Lucas to write this because he’d already done the work of reframing the problem: instead of asking “how do I make this call cheaper,” ask “what’s my cost per accepted change.” That single shift changes how you think about context size, model selection, and prompt structure — not as separate levers, but as one system you’re optimizing together.
Here’s Lucas.
I’m an AI engineer at Elevance Health and founder of Goliath.ai, so I spend most of my working hours inside Claude Code sessions. Here’s the principle that took me longest to internalize: the prompt you type is not the bill.
By the time a task finishes, the agent has read system instructions, repository guidance, tool schemas, retrieved files, terminal output, and prior turns — often more than once, because the first patch failed. The cheapest single call in that loop can produce the most expensive finished task.
The number that actually matters is effective cost: total spend on a task divided by the probability the result gets accepted. A run that costs twice as much but succeeds on the first try usually beats a cheaper run that needs three retries. Cheaper input tokens don’t automatically mean lower end-to-end cost — treating them as the same thing is the most common mistake I see.
Here’s what to do instead.
Give the agent a map, not the repository
Attaching an entire repo “just in case” costs more than tokens — it costs the agent’s attention. A bigger haystack means more exploration before it finds the needle.
Keep a short, current repo guide (a CLAUDE.md, for example) covering architecture, key directories, and a real definition of done. Give the agent an explicit starting point — the files involved, the failing test — rather than a blank search. Narrow the working set with code or symbol search before injecting full files, enable only the tools the task needs, and give it a stopping condition so it doesn’t wander into unrelated refactoring.
❌ Vague: “Fix the login bug”
✅ Specced: “The failure is in the OAuth callback flow. Start with callback.ts and session.ts. Reproduce with npm test -- auth-callback. Stop once that test and the typecheck pass.”
Spec the task, don’t just shorten it
“Write shorter prompts” isn’t the fix. A few extra tokens spent killing ambiguity are cheaper than the retry a vague prompt triggers. A prompt that holds up has five parts:
The goal — what done actually looks like
The location to start — the specific files or module
The constraints that must not change — what’s off-limits
The exact verification steps — the command that proves success
A stopping condition — when to stop, full stop
That’s task specification, not clever wording. It’s closer to writing a good ticket than a good sentence.
Keep the counterbalance in mind too: this isn’t a case for a giant static instructions file loaded on every call. A small, stable core, with task-specific procedures loaded only when needed, works better.
The context window is working memory
A bigger context window is capacity, not a target — having room for more information doesn’t mean the model uses it well. Research on long-context retrieval has shown models can be worse at pulling information from the middle of a long input than from the start or end.
Think in four tiers instead:
Stable instructions — safety rules, a short repo map, core commands. Rarely changes.
Task spec — issue, acceptance criteria. Stays visible throughout.
Retrieved working set — relevant files, tests. Pulled in and pruned as needed.
Transient evidence — logs, search results. Summarized or dropped once you’ve pulled out the fact you needed.
Not every phase deserves your best model
Planning — working out what’s wrong and what approach to take — is where ambiguity is highest, and a stronger model earns its cost there. Execution — applying a plan that’s already decided — tolerates a cheaper model well. Verification shouldn’t involve model judgment where you can avoid it: run the test suite and the type checker, and treat their output as ground truth rather than asking another model call to eyeball the result.
What to leave alone
A few things worth resisting:
Don’t strip relevant tests or constraints just to shrink the first call — it’s a false saving if it triggers retries.
Don’t default every task to the cheapest model, and don’t default every task to the frontier model either.
Don’t judge any of this on API price alone — developer review time and rework belong in the number too.
An agent that always reaches for the cheapest model isn’t efficient — it’s just cheap. The one worth building hands the model the smallest working set it needs, spends its best reasoning where a decision is actually being made, and stops the moment the tests say the job is done.
Thanks for reading, Lucas — if anyone’s CLAUDE.md is doing double duty as both a repo guide and a definition of done, hit reply and tell me what’s in it.
Charu, Co-Editor-In-Chief, Build With AI
About the author: Lucas Germinari Carreira is an AI engineer and computer science grad at Indiana University. He builds production AI systems at Elevance Health, where he works on healthcare AI and large language model infrastructure, and serves as a Claude Builder Ambassador with Anthropic, leading technical workshops and hackathons that help students build with frontier AI.
Outside of work, he enjoys building products and developer communities at the intersection of AI, software engineering, and entrepreneurship
 | A guest post by
|