How to audit your LLM token spend: a practical guide for engineering teams

Get the raw data from your provider dashboard

Start with your raw usage log. The richness of available data varies significantly by provider.

OpenRouter data is the most diagnostic-friendly because it includes application-level cost attribution, which lets you see spend by internal product or service rather than just by model.

Understand what you are actually spending money on

Raw token counts tell you how much you spent. Task-type classification tells you what you spent it on. The eight categories that capture most LLM workloads are: summarisation, code generation, reasoning, data extraction, simple Q&A, translation, classification, and general-purpose.

The goal is to identify which task types are running on which models. A classification task on claude-opus-4 is a likely optimisation target. A complex reasoning workflow on claude-haiku-3-5 is a likely quality risk. Both matter, for different reasons.

If you have application context in your export (OpenRouter's app_name field, or workflow names from Typing Mind), use that to group API calls by internal workflow before applying task classification. This gives you spend by workflow, which is more actionable than spend by task type alone.

Find where the wrong model is doing the work

For each task type there is a cheapest adequate model: the one that produces acceptable output quality at the lowest cost.

Any workflow running a tier above what the task requires is an optimisation opportunity. At volume, even a single mismatched workflow can represent thousands of dollars per month. The mismatch identification is the core of the diagnostic: you are looking for places where frontier model capability is being purchased for tasks that do not require it.

Find the tokens you are paying for twice

If you are using Anthropic or OpenAI models and have not implemented prompt caching for system prompts that repeat across requests, you are paying full price for tokens you could serve from cache at 10% of the input cost.

For high-volume workflows with consistent system prompts, caching alone can reduce prompt token spend by 30 to 70%. The engineering effort is typically a single day of work, and the saving is permanent and grows with request volume.

OpenRouter exports include a tokens_cached field that shows your current cache hit rate. If this is zero or near-zero for models that support caching, addressing it is the first structural fix to prioritise. Before implementing caching, conduct a system prompt audit to remove bloat: caching a bloated prompt locks in the inefficiency rather than fixing it.

Turn findings into a ranked action plan

Each optimisation opportunity should be expressed as a monthly saving estimate: the current cost for this workflow minus the projected cost if the recommended change is applied. Rank by saving, highest first.

The top three recommendations are your implementation roadmap and your finance conversation in one document. When the CFO asks what you are doing about the AI bill, "we have identified three structural changes worth $X per month and we are implementing them in order" is a materially different answer to "we are looking at ways to optimise."

Express each recommendation with: the workflow name, the current monthly cost, the specific change recommended, the projected saving, and the engineering effort estimate. This gives the engineering lead enough to prioritise the work and the CFO enough to understand the return.