Models, tokens and cost

​

Models and the price table

import { estimateTokens, estimateCost, getModelInfo, registerModel } from '@lousho/build-ai-agent';

// A custom or self-hosted model: add it (or override a built-in) before use.
registerModel({
  id: 'my-llama',
  provider: 'ollama',
  contextWindow: 32768,
  inputCostPerMTok: 0.2, // optional; omit for unknown or free
  outputCostPerMTok: 0.6,
});

const used = estimateTokens([{ role: 'user', content: 'Summarise this report' }], { model: 'my-llama' });
const info = getModelInfo('openai/gpt-4o-mini'); // exact id, `provider/id`, or a dated snapshot
const usd = estimateCost({ inputTokens: used, outputTokens: 500 }, 'my-llama'); // undefined if unpriced

• estimateTokens(input, { model?, estimator? }) is a heuristic (about 4 characters per token for English, more for CJK and other scripts, plus per-message overhead and tool-call JSON). Expect roughly 15-20% error on English: fine for compaction and budgets, not for billing. Plug in a real tokenizer with setTokenEstimator(fn) or options.estimator.
• getModelInfo(id) matches the exact id, then provider/id, then dated snapshots (gpt-4o-mini-2024-07-18 resolves to gpt-4o-mini). Unknown models return undefined.
• registerModel(info) adds or overrides an entry; the latest registration wins.
• estimateCost(usage, model) returns USD, or undefined (not 0) when the model or its prices are unknown.

​

Usage and cost of a run

import { createAgent, formatUsage } from '@lousho/build-ai-agent';

const result = await createAgent({ provider, instructions: 'You compare cities.' }).send('Compare 3 cities');

result.usage.inputTokens; // all model calls of the run, delegated children included
result.usage.costUsd; // number, or undefined if any model used has no known price
result.usage.estimated; // true if some call reported no usage and was estimated
result.usage.byModel['gpt-4o-mini']; // { inputTokens, outputTokens, costUsd?, calls }
result.stepUsage?.[0]; // { step, model, usage, estimated, costUsd? } per model call
console.log(formatUsage(result.usage)); // 1,234 in / 567 out tokens · $0.0042 (2 model calls)

• Reported vs estimated. Providers report promptTokens/completionTokens; the built-in providers also pass on cachedInputTokens/reasoningTokens when the ‘ai’ SDK’s provider metadata carries them (usage.cachedInputTokens and usage.reasoningTokens only appear then). A backend that reports nothing gives undefined usage, never zeros. For that step the executor falls back to estimateTokens and sets usage.estimated (a leading ~ in formatUsage). A custom provider should leave GenerateResult.usage unset rather than fill in zeros.
• Cost. costUsd is the sum of estimateCost per model. It is undefined, never a misleading partial sum, as soon as any model used has unknown pricing; byModel shows which ones are priced.
• Delegation. A delegated child’s usage is added to the parent’s totals and byModel, and is also shown on its own as usage.delegated ({ inputTokens, outputTokens, totalTokens, costUsd, modelCalls, estimated, runs }).
• Resume. A run resumed from a checkpoint, or after an approval, continues from the saved totals instead of restarting at zero. Checkpoints written by older versions start from their saved token counts with an unknown cost.
• Events and traces. The finish event (and every lifecycle event that carried usage) now carries the running totals; text-complete also has stepUsage, and onLLMResponse receives the call’s usage as a third argument. The chat span’s gen_ai.usage.* attributes use the same numbers, with lousho.usage.estimated set to true when they are estimates.
• Streaming. agent.stream() events carry the same accounting: step.done and run.done have usage with inputTokens, outputTokens, estimated and costUsd (run-level also modelCalls), alongside the older promptTokens/completionTokens.
• promptTokens and completionTokens on usage remain as deprecated aliases of inputTokens and outputTokens.

import { registerModel } from '@lousho/build-ai-agent';

registerModel({
  id: 'ft:gpt-4o-mini:acme',
  provider: 'openai',
  contextWindow: 128000,
  inputCostPerMTok: 0.3,
  outputCostPerMTok: 1.2,
});