Who this is for
- Finance leaders seeing AI costs grow faster than the business can explain
- Engineering and AI leads managing multiple workflows, models, and teams
- COOs and CTOs who need to forecast AI spend at 3x and 5x current usage
Definition
AI token spend optimisation (also called token optimisation) is the process of reducing LLM API costs by identifying and eliminating structural waste patterns in how AI workflows consume tokens. The four main patterns are system prompt bloat, agent loop redundancy, prompt cache misses, and model-task mismatch. Most teams can reduce spend by 30-60% without changing output quality.
A single prompt change can double a team's monthly AI bill before anyone notices. Unlike SaaS or cloud infrastructure costs, AI token spend is not predictable, not proportional to headcount, and not governed by the same tooling that finance teams use for everything else. Most organisations that have moved beyond AI experimentation into production now have at least one workflow where the cost has surprised them.
The problem is not that organisations do not care about the cost. A UBS survey found that 60% of companies actively watching AI costs have already moved some workloads to cheaper models. The problem is that most of those moves are made on instinct rather than data. Teams know the number on the invoice. They do not know which of their workflows is generating the waste, what pattern is driving it, or what the fix is worth.
Early diagnostics across multi-team deployments consistently surface 30-60% potential savings from model-task mismatch and caching alone, without any change to output quality. This guide explains what makes AI token spend structurally different, what data you need, where most of the waste lives, and how to build a process that keeps costs under control as usage scales. Our overview of AI spending patterns covers the broader landscape, and our guide to auditing LLM token spend walks through the data collection process in detail.
Why AI token spend is not like cloud spend
Cloud infrastructure costs are measurable, attributable, and governed by mature tooling. You pay for compute by the hour, storage by the gigabyte, and egress by the terabyte. The unit economics are stable. A 10% cost increase means someone provisioned 10% more capacity. FinOps practices built for cloud apply well here.
AI token spend breaks every one of those assumptions:
- The cost unit is a token, not an hour or a gigabyte.
- Cost per call varies by model, input length, output length, and whether caching is applied.
- The same workflow can cost 10 times more after a single prompt change, with no alert firing.
More importantly, AI costs are driven by usage patterns inside workflows, not by infrastructure provisioning decisions. The engineering team that routes every task to a frontier model, or builds an agent loop that does not converge, or fails to enable prompt caching, is generating waste that no cloud FinOps tool will surface. The cost is invisible until the invoice arrives, and even then the aggregate total tells you nothing about which workflow to fix first.
The key difference: Cloud costs reflect provisioning decisions. AI token costs reflect usage patterns inside your code. The tooling built for one does not transfer to the other.
Enterprise AI API spend tripled on average in the twelve months to mid-2026. In RAG and agent-heavy deployments, analysis of gateway usage data consistently puts 40-70% of token consumption down to structural waste rather than productive usage. The scale of the problem is not immediately visible from an invoice total, which is why many teams underestimate it until usage is already compounding.
The discipline emerging to address this is sometimes called AI cost management or AI FinOps. Unlike traditional FinOps, which governs infrastructure provisioning decisions, AI cost management works at the token and workflow level. It requires per-request data, task-level labelling, and analysis built for how AI costs actually behave. The tools and practices that work for cloud spend do not transfer here.
This is the gap that most organisations discover too late: visibility into the total tells you that there is a problem. It does not tell you what is causing it, which workflow to fix, or what the fix is worth. That requires a different kind of analysis.
The six data dimensions that matter
Before you can optimise anything, you need the right data. These six dimensions are what move analysis from "we spent $40,000 last month" to "this workflow is wasting $9,000 a month and here is how to fix it." Most teams can access all six without any additional integration. OpenRouter provides them in a downloadable CSV. TypingMind provides them in a JSON export.
| Dimension | What it tells you |
|---|---|
| Model | Which model handled each call. The foundation for any routing analysis. |
| Input tokens | How much context was sent. Unusually high input counts often signal system prompt bloat. |
| Output tokens | How much was generated. High output counts on simple tasks often indicate model over-serving. |
| Cost per call | The monetary cost of each request. Needed to rank waste by impact, not just frequency. |
| Finish reason | Whether the response completed naturally or was truncated. Truncated completions often signal context length issues or misconfigured max tokens. |
| Workflow or task type | What the call was actually for. This is the critical dimension most teams are missing. Without it, you can see the spend but not the structure. |
Workflow labelling is the hardest dimension to add if you do not already have it, but it is also the one that unlocks the most actionable insights. The other five are usually available without any instrumentation work. If you are not capturing all six per request, our guide to auditing LLM token spend walks through how to get them from OpenRouter, TypingMind, or your own gateway.
Red flags to watch for: Input tokens significantly higher than output tokens on simple tasks often signals system prompt bloat. A high percentage of length or max_tokens finish reasons signals context length misconfiguration. And if input token counts are high and consistent across calls, even for simple tasks, that is usually system prompt bloat multiplying at volume.
The four patterns driving most AI token waste
Across usage data from a range of AI deployments, the same four structural patterns account for the majority of recoverable waste. They are not bugs. They are the natural result of teams building quickly without token spend as an explicit design constraint.
System prompt bloat
Long, repetitive system prompts sent with every call, often containing persona definitions, lengthy instructions, and static context that does not change between requests. Every token in the system prompt is charged on every call. In high-volume workflows, this multiplies rapidly.
Signal: high and consistent input token counts across calls, even for simple tasks.
Agent loop redundancy
Agentic workflows that call the same model multiple times for tasks that could be consolidated, or that loop without converging on a result. Common in RAG pipelines and multi-step agents where each iteration adds cost without proportional value.
Signal: multiple calls per user task with similar prompts and marginal output changes.
Prompt cache misses
Anthropic and other providers offer significant discounts on tokens that can be cached across calls. Teams using static system prompts, repeated prefixes, or shared context windows that qualify for caching but have not enabled it are leaving meaningful savings unused.
Signal: low or zero cached token percentage despite repeated static prefixes.
Model-task mismatch
Using frontier models for tasks that a smaller, cheaper model handles equally well. Simple classification, data extraction, and formatting tasks regularly run on GPT-4o or Claude Sonnet when Haiku or GPT-4o Mini would produce the same output at a fraction of the cost.
Signal: high-cost models used for tasks with low complexity and short outputs.
These four patterns rarely appear in isolation. A workflow with system prompt bloat often also has a model-task mismatch: the large system prompt was written with a frontier model in mind, and nobody has revisited the model selection since. Addressing them together is where the largest savings are found. In optimised deployments, each pattern is either absent or confined to a small, justified subset of workflows. Our articles on system prompt bloat, token minimizing strategies, and token spend efficiency cover each pattern in more detail.
See these patterns in your own data
A TokenomicsIQ diagnostic identifies which of these patterns are present in your usage data, ranks them by monthly saving, and tells you which workflow to fix first — in under 24 hours.
Request ReportA practical framework: instrument, label, diagnose, fix, govern
Knowing the patterns is not enough. The question is how to build a repeatable process for finding and fixing them. The following five-step framework applies whether you are running one AI workflow or fifty.
Instrument
Get your per-request log data. If you are using OpenRouter, export the CSV from your dashboard. If you are using TypingMind, export the JSON. If you are running direct API calls, configure your gateway or logging layer to capture model, token counts, cost, and finish reason for every call. This step requires no new infrastructure if you are already routing through a gateway.
Outcome: a complete per-request log with model, tokens, cost, and finish reason.
Label
Assign a workflow or task type to each request. This is the step that transforms a cost log into an actionable dataset. Labelling can be done retroactively using model, prompt pattern, and call frequency as signals, or prospectively by adding a metadata field to every API call. Even a rough taxonomy of five to ten task types is enough to surface the main opportunities.
Outcome: every request mapped to a workflow or task type.
Diagnose
Analyse the labelled data to identify which patterns are present in which workflows, at what cost. This means calculating cost per call by workflow, input token distribution by model and task type, and cache hit rate where applicable. The output of this step is a ranked list of findings, not a list of raw observations.
Outcome: a ranked list of waste patterns by monthly saving.
Fix
Address findings in order of saving versus effort. Model substitution (routing a workflow to a cheaper model) is typically the highest-impact fix and requires only a configuration change. Prompt caching can often be enabled in an afternoon. System prompt compression takes longer but compounds across call volume. The priority order matters: not every fix is worth the same engineering time.
Outcome: implemented changes with measured cost reduction per workflow.
Govern
Set a cost baseline after fixes are applied and review it monthly. New workflows will introduce new patterns. Prompt changes will shift token counts. The governance layer is what prevents the waste from returning as usage grows. A 30-minute monthly review of per-workflow spend, with an alert for any workflow whose cost per call has increased by more than 20% month-on-month, is sufficient for most teams.
Outcome: a monthly review that catches drift and prevents waste from creeping back.
The most common failure mode is teams that complete steps one through four and skip step five. Fixes without a governance layer are temporary. Usage grows, new developers add to the codebase, and within a quarter the waste patterns are back. Building the monthly review into an existing engineering rhythm is the step that makes the savings permanent.
What good looks like, and what it looks like at scale
The practical question for most teams is not whether to optimise, but what the optimisation is worth and whether it holds as usage scales. The answer depends heavily on current baseline efficiency, but the comparison between an optimised and unoptimised approach at scale is consistently significant.
The key insight is that optimisation is time-sensitive. The waste patterns that exist today are cheaper to fix on today's call volume than on next quarter's call volume. A model-task mismatch that costs $800 per month at current scale costs $4,000 per month at 5x. The fix takes the same engineering time either way.
For finance, this matters beyond the immediate saving. Being able to show the CFO a current-3x-5x cost curve, with and without a specific fix applied, turns an abstract optimisation conversation into a concrete business case. It also turns AI spend from an open-ended cost into a forecastable, controllable line item. Our guide to AI cost optimisation covers the CFO-facing framing in detail.
How a diagnostic works in practice
A TokenomicsIQ diagnostic is a purpose-built diagnostic tool for AI token spend: it turns your existing usage data into a prioritised savings plan in under 24 hours. Most teams understand the problem but lack the time to run a full AI token audit from scratch. The instrument-label-diagnose process takes time to build, and most engineering teams have more pressing priorities than a bespoke token analytics layer. Unlike general-purpose AI cost management tools, which surface aggregate spend without diagnosis, a structured diagnostic tells you which workflow to fix first and what the fix is worth in pounds and pence.
Export your usage CSV from OpenRouter, or your JSON from TypingMind, and upload it. No integration is required. No code is added to your stack. The diagnostic runs the full analysis and produces a report containing:
- Which workflows are driving the most waste, named and ranked.
- Which of the four patterns are present and in which workflows.
- Estimated monthly saving per fix, ranked by implementation effort.
- Spend projections at 3x and 5x current usage, with and without each fix applied.
The output is designed to be read by your engineering lead and presented to your CFO in the same sitting. It gives engineering a prioritised action list with enough technical detail to act on immediately. It gives finance a savings forecast with enough commercial context to make the case for the work.
Frequently asked questions
What is AI token spend optimisation?
AI token spend optimisation is the process of reducing the cost of LLM API usage by identifying and fixing structural waste patterns in how AI workflows consume tokens. This includes replacing oversized models with cheaper alternatives for simple tasks, fixing system prompt bloat, enabling prompt caching, and eliminating redundant agent calls. The goal is to reduce cost without reducing output quality.
What is token optimisation?
Token optimisation is the practice of reducing the cost and volume of tokens consumed by AI workflows without reducing output quality or speed. It covers model selection, prompt design, caching configuration, and agent architecture. AI token spend optimisation is the application of this practice to a specific deployment, using per-request data to identify where waste is concentrated and prioritise fixes by saving.
What are the biggest sources of AI token waste?
The four main patterns are system prompt bloat, agent loop redundancy, prompt cache misses, and model-task mismatch. Model-task mismatch and cache misses are typically the highest-impact findings and the fastest to fix. System prompt bloat is common in mature deployments where prompts have grown incrementally over time. Agent loop redundancy is most prevalent in RAG and multi-step agent architectures.
How much can I save by optimising AI token spend?
Savings vary by usage pattern, but teams that address model-task mismatch and prompt cache misses consistently reduce their AI API spend by 30-60% without changing the quality of outputs. The highest-impact single fix is usually routing simple, high-volume tasks to smaller models. The savings multiply as usage scales, which is why addressing the patterns at current volume is significantly cheaper than waiting.
What data do I need to start?
You need per-request log data covering: which model handled each call, input token count, output token count, cost per call, and finish reason. OpenRouter provides this in a downloadable CSV from the dashboard. TypingMind provides it as a JSON export. Most organisations have this data already available without any additional integration or instrumentation. See our AI token cost guide for a breakdown of what each field means.
What is model routing and how does it reduce costs?
Model routing is the practice of directing each AI task to the most cost-appropriate model based on the task's complexity requirements. Simple classification, extraction, and formatting tasks can run on smaller models at a fraction of frontier model prices while producing equivalent output. Effective routing requires task-level spend data to identify which workflows are currently running on models that are more capable than the task requires.
How long does it take to see savings from AI token spend optimisation?
Most teams implement the highest-impact fixes within one to two weeks of receiving a diagnostic report. Model routing and prompt caching changes are typically configuration-level, not code changes, and their effect appears on the next billing cycle. System prompt compression takes a little longer but is still usually a matter of days rather than weeks. The diagnostic gives you a prioritised action list so the fixes with the highest return for the least effort are addressed first.