Models and the price table
Dependency-free helpers for budgeting and context decisions.estimateTokens(input, { model?, estimator? })is a heuristic (about 4 characters per token for English, more for CJK and other scripts, plus per-message overhead and tool-call JSON). Expect roughly 15-20% error on English: fine for compaction and budgets, not for billing. Plug in a real tokenizer withsetTokenEstimator(fn)oroptions.estimator.getModelInfo(id)matches the exact id, thenprovider/id, then dated snapshots (gpt-4o-mini-2024-07-18resolves togpt-4o-mini). Unknown models returnundefined. (IncreateAgent({ model })andresolveProvider()strings thevendor/model prefix chooses the provider; with OpenRouter useopenrouter/<vendor>/<model>, e.g.openrouter/openai/gpt-4o-mini.)registerModel(info)adds or overrides an entry; the latest registration wins. Register local models (LM Studio, Ollama, vLLM) with the context window they were loaded with: compaction and tool search otherwise assume 128,000 tokens and log a one-time warning. Local models have no built-in price (the Ollama entries carry only a context window), so theircostUsdisundefinedandlimits.maxCostUsdonly warns; register a price,0included, to have it counted and the limit enforced:registerModel({ id: 'llama3', provider: 'ollama', contextWindow: 8192, inputCostPerMTok: 0, outputCostPerMTok: 0 }).estimateCost(usage, model)returns USD, orundefined(not0) when the model or its prices are unknown.usage.cachedInputTokensandusage.cacheWriteTokens, when given, are the parts ofinputTokensread from and written to the provider’s prompt cache. They are priced atcachedInputCostPerMTokandcacheWriteCostPerMTok, or at the input price when the model has none. The built-in OpenAI entries carry OpenAI’s cached-input price (for example 50% ongpt-4o-mini; writes cost nothing extra). The Anthropic entries bill reads at 10% and 5-minute writes at 125% of the input price.
src/models/modelData.ts). Providers change prices and models, so override entries with registerModel when you need billing-grade numbers.
Usage and cost of a run
EveryExecutionResult (and agent.send() result) carries usage, the running total of the whole run:
- Reported vs estimated. Providers report
promptTokens/completionTokens; the built-in providers also pass oncachedInputTokens/cacheWriteTokens/reasoningTokenswhen the provider reports them (usage.cachedInputTokens,usage.cacheWriteTokensandusage.reasoningTokensonly appear then).inputTokensalways counts the cached and cache-written tokens too. A backend that reports nothing givesundefinedusage, never zeros. For that step the executor falls back toestimateTokensand setsusage.estimated(a leading~informatUsage). A custom provider should leaveGenerateResult.usageunset rather than fill in zeros. - Cost.
costUsdis the sum of the per-call costs: the cost the provider reported (OpenRouter returnsusage.costfor every model, priced or not), elseestimateCostfor the model. A call with no known price (a model missing from the registry whose provider reported no cost) is left out of the sum and setsusage.costPartial: true, so the total is a lower bound rather than being discarded;byModelshows which models are priced (byModel[model].costPartialmarks a model with both priced and unpriced calls, andusage.delegated.costPartialdoes the same for children).costUsdisundefinedonly when no call has a known cost.limits.maxCostUsdis checked against the partial total. InformatUsagethe cost prints with 4 decimals under $1 (2 at $1 or more), so a run under $0.00005 reads `$0.0000`. - Hosted tools.
usage.hostedToolCallscounts the calls the provider ran for hosted tools per tool name (e.g.{ web_search: 2 }). Their per-call fees are billed by the provider and are not incostUsd. - Delegation. A delegated child’s usage is added to the parent’s totals and
byModel, and is also shown on its own asusage.delegated({ inputTokens, outputTokens, totalTokens, costUsd, modelCalls, estimated, runs }, plustoolErrors: the children’s tool calls that ended in an error result, absent when none did). A remote sub-agent’s usage is what the remote agent reports, filed underbyModel['remote:<name>']with the remote’s owncostUsd. - Resume. A run resumed from a checkpoint, or after an approval, continues from the saved totals instead of restarting at zero. Checkpoints written by older versions start from their saved token counts with an unknown cost.
- Events and traces. The
finishevent (and every lifecycle event that carriedusage) now carries the running totals;text-completealso hasstepUsage, andonLLMResponsereceives the call’s usage as a third argument. Thechatspan’sgen_ai.usage.*attributes use the same numbers, withlousho.usage.estimatedset totruewhen they are estimates. - Streaming.
agent.stream()events carry the same accounting:step.doneandrun.donehaveusagewithinputTokens,outputTokens,estimatedcostUsdandcostPartialwhen partial (run-level alsomodelCalls), alongside the olderpromptTokens/completionTokens. promptTokensandcompletionTokensonusageremain as deprecated aliases ofinputTokensandoutputTokens.
Prompt caching
Anthropic caches a request prefix only up to acache_control breakpoint that the request marks. OpenAI, and most other providers, cache long prefixes on their own. With promptCaching: 'auto' (the default), the built-in anthropic provider and the openrouter provider mark breakpoints for Claude models (anthropic/..., openrouter/anthropic/...). They mark three of the four that Anthropic allows:
- the system prompt (the last system message),
- the last tool definition,
- the last user turn. With the
anthropicprovider, a trailing tool-result turn is marked too, so each step of a tool loop reads the conversation so far from the cache.
anthropic provider, tool and system breakpoints need ai 6 or 7 (@ai-sdk/anthropic 3 or 4). The @ai-sdk/anthropic 0.0.x peer pinned for ai 4 does not send them. A custom provider gets the setting as GenerateOptions.promptCaching.
Measured through OpenRouter on anthropic/claude-haiku-4.5, with a 6,834-token prompt (system prompt and one tool):