LLM cost tracking

LLM cost tracking is turning the token counts each model call reports into money, and reading that money the way your business asks about it: by customer, by user, by request, by agent and by model.

This page explains where the cost of a call comes from, how to keep the figures exact, and how Agent Pulse reads them.

Where the cost of a call comes from

A provider reports what each call used, and the price follows from that.

  • Each kind of token has its own price: input, output, cached input, cache writes and reasoning. They are priced apart, because adding them up first would lose the difference.
  • Some charges are not tokens, such as a web search billed per use. Those are priced from the rate for that unit.
  • The price is the one in force when the call was made. When a provider changes its prices, calls made before the change keep their old price.

What every model costs today, per million tokens, is on the public model prices page, which also prices your own token counts on each model. When providers plan to turn a model off is on model retirement dates: a call to a retired model fails.

Keeping the figures exact

A cost figure is only useful if it adds up and stays put. In Agent Pulse:

  • The agent never states its own cost. It reports the provider's token counts, and the call is priced on the server.
  • Past figures never change. Every call records the rate card it was priced at. A model missing from the rate card is priced at nothing, and the console counts those calls so you know a figure is incomplete.
  • Money is stored as whole millionths of a US dollar, never as floating-point numbers, so figures add up to the cent however many calls they cover.

See how cost is calculated.

Estimates and the bill

Every call has an estimate as soon as it is recorded: its token counts at list price. For traffic billed through Google, which is Gemini and other models on Vertex AI, you can send your Google bill each day and each hour and model is checked against what was recorded. A call carries the billed figure when its hour and model on the bill came within 5% of the estimate. A provider you pay directly, such as OpenAI or Anthropic, stays estimate only.

Figure Where it comes from When
Estimated Token counts at list price As soon as the call is recorded
Billed Your Google bill, which you send When you send the day, for traffic billed through Google only

Your own code reads the bill and sends it, so Agent Pulse never gets access to your billing account. See check your bill.

Reading cost the way your business asks

A total per model says little. The questions that matter are about your product:

  • What does each customer cost? Tag each request with your customer and the signed-in user once, where your code checks the sign-in. See track cost per customer and user.
  • What did this request cost? The console opens one request and shows every model and tool call in it, in order, with what each cost.
  • What does a result cost? Count outcomes, such as a booking made, against the request that produced them, and read cost per outcome. See organisations, workspaces and requests.
  • Where is it going over time? Read the same figures in your own screens, a warehouse or a report. One call adds up spend on the server, grouped by any of customer, user, agent, model, request, day and more:
curl -G "https://$AP_GATEWAY/v1/organisations/<organisation>/activities:aggregate" \
  -H "X-Api-Key: $AP_API_KEY" \
  -H "X-Api-Secret: $AP_API_SECRET" \
  --data-urlencode 'groupBy=date' \
  --data-urlencode 'groupBy=model' \
  --data-urlencode 'startTime=2026-09-01T00:00:00Z' \
  --data-urlencode 'endTime=2026-10-01T00:00:00Z'

groupBy=date&groupBy=model is a stacked daily chart, and groupBy=user is a cost-per-person table. See read your spend over the API.

Finding what to change

Tracking cost is the start. These show where to act:

  • Failed calls. A failed call is usually still billed for the work done before it failed. Agent Pulse keeps that spend apart and shows which agents are failing, and why. See find what is failing.
  • Cheaper models. For each agent, its own calls are priced against other models that support what it used: its longest prompt, tool calls and reasoning. The estimate is about price, not quality, so try a model on real traffic first. See compare cheaper models.
  • Limits. A budget alerts you as spend nears a limit, and an enforced budget stops covered model calls at it. See set a budget that warns or stops calls, and get alerts in Slack or a webhook.

Tracking cost from the code you have

How calls are recorded depends on how your code calls models:

  • A provider's SDK called directly. Wrap the OpenAI, Anthropic or Gemini client once where you create it, and every call through it is recorded.
  • An agent framework. LangChain and LangGraph, Google ADK, the OpenAI Agents SDK and the Vercel AI SDK each have a recorder that can also let a budget stop a call before it is sent.
  • A LiteLLM proxy. One line in the proxy's config records every model and service behind it.
  • OpenTelemetry. If your agent already emits GenAI spans, point its exporter at Agent Pulse; see OpenTelemetry. Frameworks recorded from spans, such as CrewAI or Pydantic AI, count against your budgets and raise alerts, but a budget cannot stop a call recorded this way, because the span arrives after the call.

Whichever way, prompts and answers are never recorded: only counts, identifiers, timings and error codes. See what is recorded, and what never is.

How to start

  • Assess your codebase: a prompt for your coding agent that reads your repository, without changing it, and reports what Agent Pulse would show you and what it would take to connect.
  • The quickstart: an agent's first recorded model call, in Python, TypeScript or Go.
  • What to record about each call, and why an agent needs a meter of its own: AI agent metering.