AI token cost, also referred to as LLM token cost, is the price AI API providers charge for model usage, billed per token. Both terms describe the same thing: the metered cost of sending text to a language model and receiving a response. Search interest in these terms has grown dramatically over the past twelve months as AI API spend has moved from a rounding error in engineering budgets to a line item that finance teams actively scrutinise. Understanding what token cost actually is, and, critically, what it does not capture, is the starting point for managing it.
This article covers how AI token cost is calculated, why the invoice figure almost always understates your real AI spend, and what the highest-impact actions are for bringing costs down.
What is a token?
AI language models do not read text the way humans do. They process text broken into fragments called tokens. In English, a token is roughly 0.75 words: "the quick brown fox" is approximately five tokens. Longer or less common words often split into multiple tokens. Numbers, code, and non-English text tokenise differently, sometimes less efficiently.
Every API call involves two categories of tokens. Input tokens are what you send to the model: the system prompt, any conversation history, and the user message. Output tokens are what the model generates in response. Both are metered. Both appear on the bill.
How AI token cost is calculated
AI providers publish pricing per million tokens, with separate rates for input and output. Output tokens are consistently more expensive than input tokens, typically by a factor of three to five. This reflects the computational cost of generating new content versus processing existing text.
Representative pricing across the main model tiers as of mid-2026:
| Model tier | Input (per million tokens) | Output (per million tokens) |
|---|---|---|
| Frontier reasoning (e.g. o3, Claude Opus) | $15+ | $60+ |
| Frontier standard (e.g. GPT-4o, Claude Sonnet) | $2.50 to $3.00 | $10 to $15 |
| Mid-tier (e.g. GPT-4o mini, Claude Haiku) | $0.15 to $0.40 | $0.60 to $1.25 |
| Batch API (same models, async) | 50% of standard rate | 50% of standard rate |
| Prompt cache reads (where supported) | 10% of standard input rate | Not applicable |
A practical example: a workflow making 10,000 API calls per month with an average of 1,000 input tokens and 300 output tokens per call, using a frontier standard model at $3 input and $15 output per million, costs roughly $30 in input and $45 in output per month, a total of $75. At 100,000 monthly calls, the same workflow costs $750. At one million calls, $7,500. The arithmetic scales linearly with volume, which is why token cost can grow faster than teams expect when a product gains traction.
Why your AI token cost is probably higher than the invoice suggests
The token invoice captures one thing: the metered API cost for tokens processed and generated. What it does not capture is the broader picture of AI spend. At FinOps X in June 2026, analysts identified nine distinct cost buckets in a typical AI deployment. Only one of them appears on a vendor bill.
The eight buckets that do not appear on the invoice are not accounting abstractions. They represent real, quantifiable inefficiencies that inflate the effective cost of every useful output the model produces. A team that manages its invoice total without understanding these buckets is optimising the wrong number.
This is why BCA Research's observation about the end of the token subsidy matters at an engineering level. Frontier model prices are no longer artificially suppressed. As pricing normalises, the difference between teams that understand their true AI token cost and those that only watch the invoice becomes commercially significant.
The eight hidden AI cost buckets
System prompt bloat is the most common structural inefficiency. System prompts accumulate instructions over time until they account for a disproportionate share of tokens on every API call. In TokenomicsIQ's first live diagnostic, system prompt bloat was the single largest inefficiency found, present on 43% of API calls. Full detail is covered in the system prompt bloat guide.
Cache misses occur when content that appears repeatedly across API calls (typically the system prompt or shared conversation context) is not being served from the provider's cache. Anthropic and OpenAI both support prompt caching, with cache read pricing at 90% below standard input rates. On high-volume workflows with consistent system prompts, failing to enable caching can double the effective input token cost.
Model-task mismatch is running a frontier model on a task that a mid-tier or small model could handle to the same standard. A frontier reasoning model costs 100 times more per token than a fast lightweight model. Routing classification tasks, short-form generation, and structured extraction to the appropriate model tier, rather than defaulting everything to the most capable model available, is typically one of the highest-ROI changes available.
Agent loop redundancy arises in agentic systems where the orchestration layer is making more model calls than the task actually requires. Redundant calls, unnecessary re-summarisation steps, and poorly scoped tool calls compound AI token cost quickly in multi-agent pipelines.
Sync-on-batch overhead is running jobs synchronously when they could run as batch. The Batch API offered by Anthropic and OpenAI processes requests asynchronously at 50% of standard pricing. Any workflow where latency is not critical and jobs can tolerate a processing window is a candidate for batch routing.
Uncached context is the cost of re-sending conversation history or shared documents on every turn in a multi-turn workflow, even when that content has not changed. For long documents or extended conversations, this can represent a significant fraction of input token cost per session.
Infrastructure and orchestration overhead are the supporting costs of running the AI layer: compute for preprocessing, embedding costs, retrieval costs in RAG architectures, and the token cost of orchestration prompts in multi-agent frameworks. These are real costs that sit outside the primary token invoice.
Why AI token cost is accelerating
The era in which AI vendors subsidised token costs to drive adoption is ending. BCA Research, writing in the Financial Times in July 2026, described this as the shift from tokenmaxxing to token rationing. For three years, frontier model pricing was effectively a market development expense. That is no longer the case.
Meanwhile, usage is compounding. UBS research published in June 2026 found that companies actively managing AI budgets are already switching to cheaper models or open-source alternatives. Those that are not are encountering $35,000 monthly bills and quota overruns at 200% of plan. Uber burned through its annual AI budget in four months. A San Francisco fintech confirmed a single employee spent $81,267 in one week. The pattern is consistent: spend grows faster than visibility.
The teams navigating this well are not applying blanket cuts. They are running a diagnostic first, identifying where the spend actually sits across all nine buckets, and making targeted changes with estimated savings attached to each one.
How to reduce your AI token cost
Five actions ranked by typical impact
Implement prompt caching. For any workflow where the same system prompt or document appears across multiple requests, enable prompt caching. Cache reads cost 10% of standard input pricing. On a high-volume workflow, this is the single largest saving available per hour of engineering time invested. The prerequisite is a clean, stable system prompt.
Audit and trim system prompts. Review every system prompt. Calculate what percentage of average total tokens per request it accounts for. Flag anything above 30% and categorise each instruction as essential, conditional, or stale. Remove the stale content. Extract conditional content so it only fires when the relevant scenario is triggered. The full process is described in the system prompt bloat guide.
Route tasks to the correct model tier. Map your workflows by task type and identify which ones are running on a frontier model unnecessarily. Classification, extraction, summarisation, and short-form generation tasks are typically well-served by mid-tier models at a fraction of the cost. This is explored in more detail in the guide to auditing your LLM token spend.
Convert non-latency-sensitive jobs to batch. Identify workflows where the result does not need to be returned in real time. Batch API pricing is 50% of standard rates. For reporting pipelines, content generation at scale, and overnight processing jobs, this halves the token cost without any quality trade-off.
Audit agent loop call counts. In any agentic workflow, count the number of model calls made per task completion. Compare against the theoretical minimum. Redundant calls, unnecessary summarisation steps, and over-scoped tool use are common sources of inflated AI token cost in multi-agent architectures.
The challenge is prioritisation. Every team can identify potential savings. The question is which changes will produce the largest return for the available engineering time. That requires usage data at workflow level: not the aggregate invoice total, but a breakdown of which workflows are consuming which models, at what token volumes, with which inefficiency patterns active.
Without that data, teams are making educated guesses. That is why the structured LLM spend audit is the right starting point before committing engineering time to any specific optimisation.
What the token invoice does not tell you
AI providers are building their own visibility tools. AWS ships a FinOps Agent for AI cost monitoring. Google Cloud has released a FinOps Explainability Agent and Spend Caps for AI budgets. Both are free, and both stop at their own cloud boundary. Neither sees direct OpenAI or Anthropic API spend. For teams running AI across multiple providers, which is the majority of companies at meaningful scale, the hyperscaler tools show a partial picture by design.
Ramp launched AI Spend Intelligence in 2026, positioned around financial visibility into AI tool subscriptions. The comparison with TokenomicsIQ illustrates the distinction clearly: financial monitoring at the subscription level is a different thing from a technical diagnostic at the token and workflow level.
The token invoice tells you what you spent. A workflow-level diagnostic tells you where you spent it, which of the nine buckets the waste sits in, and what to change first.