LLMProvider is what an agent generates with. The SDK ships OpenAI,
Anthropic, Google (Gemini), OpenRouter, Ollama and pi providers (each backed by an optional
peer package, see Installation) and a
deterministic mock for tests. Name one with a provider/model string, or pass
a provider instance. The vendor/ model prefix chooses the provider; with
OpenRouter use openrouter/<vendor>/<model> (e.g.
openrouter/openai/gpt-4o-mini).
google/ prefix needs the optional @ai-sdk/google package (^1.2.0 with
ai 4, ^3.0.0 with ai 6, ^4.0.0 with ai 7) and reads
GOOGLE_GENERATIVE_AI_API_KEY, the variable @ai-sdk/google itself uses.
Any AI SDK vendor by name
Every official@ai-sdk/* provider resolves from a model string too. Install
the vendor’s package (^1.0.0 with ai 4, ^3.0.0 with ai 6, ^4.0.0 with
ai 7); it is loaded on the first call and reads its own credentials
(MISTRAL_API_KEY, GROQ_API_KEY, the AWS chain for Bedrock, and so on).
azure, bedrock (or amazon-bedrock), vertex (or
google-vertex), mistral, groq, xai, deepseek, cerebras, together
(or togetherai), fireworks, deepinfra, cohere, perplexity, baseten,
huggingface and vercel. ai-sdk/<vendor>/<model> loads @ai-sdk/<vendor>
for any other vendor, including ones newer than your Lousho version. A missing
package fails on the first call with
LOUSHO_PEER_MISSING and the install
command. A provider you add with LLMProviderRegistry.register() under the
same name wins over the AI SDK package.
Vendors without a prefix
A prefix the SDK does not know fails withLOUSHO_PROVIDER_UNKNOWN. A near-miss
(anthopic/...) gets a “did you mean”; any other vendor gets the ways to reach
it anyway:
openrouter/<vendor>/<model>: OpenRouter serves most vendors with oneOPENROUTER_API_KEY(e.g.openrouter/mistralai/mistral-large; OpenRouter’s own vendor names can differ from the AI SDK’s).pi/<vendor>/<model>, when pi serves that vendor (xai,groq,mistral,deepseek,amazon-bedrock,azure, …); see thepiprovider.- the vendor’s own AI SDK package with
fromAiSdk(), e.g.createAgent({ provider: fromAiSdk(mistral('mistral-large-latest')) }).
Choosing the provider in createAgent()
createAgent() takes exactly one of:
model: 'provider/model'- resolved withresolveProvider(); the key comes from the provider’s conventional environment variable (see Provider credentials).provider: <LLMProvider>- your own provider, a configured built-in one, or a mock. You may also passmodel(a bare id such as'gpt-4o'): it becomes this agent’s model, overriding the provider’s default.- Neither - resolved from the environment:
LOUSHO_MODEL(aprovider/modelstring) if set, otherwise the first ofOPENAI_API_KEY,ANTHROPIC_API_KEY,OPENROUTER_API_KEY,OLLAMA_BASE_URLthat is present. With none set it throws an error listing the fixes.
npm install command.
Retries and fallback models
createAgent() retries a failed model call (rate limit, timeout, network
error, 5xx) twice by default, and with fallbackModels moves on to the next
model when the call still fails:
model string and each fallback are resolved with the ai SDK’s own
retries off, so retry is the only retry layer. A built-in provider instance
and a fromAiSdk() provider are wrapped the same way (with their own
maxRetries, default 2, when retry is unset), and their ai SDK retries
are then off (retry: false sends each call once). Any other provider
instance is wrapped only when you set retry. A streamed step that fails before
any text or tool call (after reasoning only) is retried as well. agent.stream() reports provider.retry
and provider.fallback events. Details in
Provider retries and fallback.
Which model runs?
In order: the agent’s ownsettings.model (set with
AgentBuilder.setSettings({ model }), or model next to provider in
createAgent()), then the model the provider was built with
(resolveProvider('openai/gpt-4o-mini'), a spec’s provider.model, or
defaultModel), then the provider’s built-in default. That last default only
applies when nothing configured a model — a model you set is never overridden
by a hard-coded one.
Multimodal input
Message.content is a string or a list of parts: { type: 'text', text },
{ type: 'image', image, mimeType? } (an http(s) URL, a data: URL or the
bytes as a Uint8Array) and { type: 'file', data, mimeType, filename? }.
agent.send(), agent.stream(), session.send() / stream(), t.send() in
evals and the React hook’s send() all take an AgentInput: a string, a list
of parts (sent as one user message) or a Message[] (passed through as it is;
the agent’s system prompt is added in front unless the list already starts
with it, so a result.messages can be sent back with a new user message).
Parts sent through createAgent():
Message[] is passed through as it is; textOf() returns the text parts of a message:
-
Images go to the model on
usermessages with every built-in provider (an image URL is downloaded by theaiSDK first for Anthropic and Ollama;pisendsdata:URLs and bytes only - anhttp(s)URL degrades to a text note with a one-time warning). Pick a vision model: Ollama ignores images for a text-only model, and OpenRouter rejects them for one. -
Files (
{ type: 'file', data, mimeType }) are sent asfileparts for these media types, by provider andaimajor:Any other file part rejects the call withLOUSHO_UNSUPPORTED_CONTENTbefore anything is sent, so the model never answers without the file (and underwithFallback()the next provider gets the call). To send a text note in its place instead ([file report.pdf (application/pdf) not sent], with one warning per media type), set the provider config’sunsupportedFiles: 'text-note'(fromAiSdk()takes the same option). Onai4 the accepted@ai-sdk/*range includes 0.0.x packages that have no file parts, so every file is unsupported there. Thepiprovider still sends every file part as a text note with a one-time warning. A provider subclass whose model takes more types overridesprotected fileMediaTypes(): readonly string[]to list them, or setsprotected readonly acceptsFileParts = trueto send every type. -
system,assistantandtoolmessages are sent as their text parts. -
Everything that reads message text uses
textOf():estimateTokens()(each image or file part counts as a flat 1,000 tokens), compaction,recordReplay()cassettes (which store the text and a digest of each image or file, or its URL) andmockModel().FileSessionStoresaves bytes as base64 ({ "$bytes": "..." }) and loads them back asUint8Arrays;SqliteStore(sessions and checkpoints) does the same.
Advanced: the executor API
AgentExecutor.execute() takes the same messages as its input (see the executor API):
Provider classes
To use another backend, first check whether it has an AI SDK provider package:
then wrap its model with
fromAiSdk() (below). Otherwise implement the
LLMProvider interface (name, generate(), stream(), supportsTools(),
supportsStreaming(), getModels(), optionally defaultModel) and pass the
instance as provider, or register a factory with
LLMProviderRegistry.register(name, factory); a registered name then works in a
model string (model: 'name/model-id'), and the factory receives defaultModel and reads its
own credentials. A factory can take its own config type, an interface that
extends LLMProviderConfig (for example BedrockConfig with a region):
register('bedrock', (config: BedrockConfig) => new BedrockProvider(config)),
then LLMProviderRegistry.create('bedrock', { region: 'eu-west-1' }).
A built-in name ('openai', 'anthropic', 'google', 'openrouter', 'ollama', 'pi',
'mock') takes that provider’s own config type (BuiltinProviderConfigs), so
create('openai', { baseUrl: '...' }) is a type error; if you registered your
own factory under a built-in name, pass its config type:
create<MyOpenAIConfig>('openai', ...).
Request timeout
The built-in providers’timeout option (ms) is the longest wait for a model
response to start and between two of its chunks. A call that waits longer
fails with a retryable TimeoutError (compacted.category: 'timeout'), which
retry retries. It never cuts off an answer that keeps streaming. The default
is 10 minutes; 0 or Infinity turns it off.
The timeout applies to each attempt, and a timed-out attempt is retried (a
hung connection is often transient), so a provider that never answers fails
after up to timeout × (maxRetries + 1) plus the backoff: about 30 minutes
with the defaults (10 minutes, 2 retries). A stream that stalls after it sent
text is not retried, so it fails after one timeout. To fail sooner, lower
timeout or retry.maxRetries.
OpenAI response storage (store)
On the Responses API (the default api, on ai 6/7), OpenAIProvider sends
store: false, so OpenAI does not keep the calls server-side: the SDK sends
the whole transcript on every call and never refers to a stored response.
Pass store: true to have OpenAI store them (for its dashboard logs or
evals). store is ignored with api: 'chat'.
Slow local models: a custom fetch
OpenAIProvider, AnthropicProvider, OpenRouterProvider and
OllamaProvider take a fetch option that their model calls go through.
Node’s built-in fetch gives up on a response whose headers take over 300
seconds (Headers Timeout Error), which a large local model can hit on a
long non-streamed generation. The SDK classifies that failure as a retryable
timeout, but every retry starts the generation over, so raise the limit
instead. Install undici and pass its Agent as the dispatcher:
A custom fetch for proxies and tests
Besides longer timeouts, fetch is the hook for a corporate proxy, for
logging every request, or for fault injection in tests. OpenAIProvider,
AnthropicProvider, OpenRouterProvider and OllamaProvider all take it
the same way, and OpenRouterProvider and OllamaProvider also use it for
their model-catalog requests. Their error handling and request changes wrap
your function, so a failure you return from it is classified as it would be
from the real network:
Any AI SDK model: fromAiSdk()
fromAiSdk(model) turns a language model from any AI SDK provider package
into an LLMProvider, so you do not write an adapter:
@ai-sdk/google (google('gemini-2.5-flash')),
@ai-sdk/amazon-bedrock, @ai-sdk/azure, @ai-sdk/mistral and the AI Gateway
(gateway('openai/gpt-4o') from ai 7 or @ai-sdk/gateway). Calls go through
your installed ai, so install the provider package major that pairs with it
(see Vercel AI SDK versions).
The provider serves the one model it wraps, with tools and streaming on, and
leaves reasoning settings to the model you built (the
reasoning option is not
mapped). Each throws LOUSHO_CONFIG_INVALID:
- a model id string (
fromAiSdk('openai/gpt-4o')): pass the provider function’s model instead; - a model for another
aimajor than the installed one (aspecificationVersion: 'v1'model withai6 / 7, or a newer one withai4); - a run with another
modelid than the wrapped one (inside a run it is the cause of theLOUSHO_PROVIDER_REQUEST_FAILEDerror): wrap each model withfromAiSdk()and combine them withwithFallback().
tool({ ... })) come over with fromAiSdkTools(); see
Migrating from the AI SDK.
The pi provider
pi/<pi-provider>/<model> routes a call through
@earendil-works/pi-ai
(the engine pi-coding-agent runs on), an optional peer that needs no ai
package:
pi/ is a pi provider (openrouter, anthropic,
openai, google, …); the rest is the catalog model id verbatim, slashes
included. Credentials are per nested provider: pi reads each one’s own
conventional env var (OPENROUTER_API_KEY, ANTHROPIC_API_KEY, …), so a
pi/anthropic/... spec never sees an OpenRouter key. lousho init --provider pi scaffolds the pi/openrouter/... default. A nested provider
whose key is missing fails the call with LOUSHO_PROVIDER_MISSING_API_KEY
naming the variable, and a missing @earendil-works/pi-ai fails it with the
usual MissingPeerDependencyError.
What maps, per call:
- messages:
systemto pi’ssystemPrompt,user/assistant/toolin place; assistantreasoningblocks become thinking parts and atoolmessage’stoolCallIdlinks it to its call; - tools:
parametersis sent as JSON Schema - raw JSON Schema, zod 3, zod 4 and Standard Schema inputs are converted once and cached; - streams: pi’s
text_delta/thinking_delta/toolcall_*/done/errorevents become the sametext-delta/reasoning-*/tool-call/finish/errorchunks theaiproviders emit, soagent.stream()events andresult.usageare unchanged; - usage and cost: pi’s per-call usage (including
cacheRead) is carried into the run total, and the catalog’s prices register the model for the run’susage.costUsd; - retries: pi’s own
maxRetriesis pinned off -retry/fallbackModelsstay the only retry layer, like every built-in.
PiProviderConfig.models injects catalog entries (pi’s faux provider, a
private catalog) ahead of the built-in list. pi is Node-only: the
cloudflare-worker target refuses pi/... specs and its bundle redirects
the package to a coded shim.
Local and OpenAI-compatible servers
LM Studio, Ollama’s/v1, vLLM, llama.cpp’s llama-server and hosted
OpenAI-compatible APIs (Groq, DeepSeek, Together, …) all speak the OpenAI
wire format. Point OpenAIProvider at the server’s /v1 URL and pick the API
it implements with api:
apidefaults to'responses'(POST /v1/responses, what api.openai.com serves). Most compatible servers implement only Chat Completions, and then a call fails with a 404, so setapi: 'chat'. With'chat'the built-in hosted tools (webSearch(),codeInterpreter(),fileSearch()) are refused, since they are Responses API tools;hostedTool()still passes through, andreasoningsends only the effort (nosummary).- The key is a dummy: the provider needs a non-empty
apiKey, and local servers ignore it. - Structured output: LM Studio’s
/v1/responsesdoes not enforce the JSON schema, so the model follows it from the prompt alone. Useapi: 'chat', which sends the schema asresponse_format. - Set the context window. The model registry does not know local models, so
compaction assumes
128_000tokens and warns once. Passcompaction: { contextWindow }, or callregisterModel({ id, provider: 'openai', contextWindow })once (see Local and unknown models).
openai/<model> string with OPENAI_BASE_URL. OpenAIProvider passes
OPENAI_BASE_URL to @ai-sdk/openai when no baseURL is configured (on every
major, including the 1.x line that pairs with ai 4), so createAgent({ model: 'openai/qwen3-8b' }) with
OPENAI_BASE_URL=http://localhost:1234/v1 and a dummy OPENAI_API_KEY=any
calls the local server without code changes. That route uses the Responses
API unless LOUSHO_OPENAI_API=chat is set too, which selects Chat Completions
for every OpenAIProvider without an explicit api (what vLLM, llama.cpp and
Ollama’s /v1 need).
Any AI SDK model. fromAiSdk() wraps a Chat Completions model built with
@ai-sdk/openai directly (or @ai-sdk/openai-compatible, if you use it):
npx lousho doctor shows OPENAI_BASE_URL when it is set, and
npx lousho doctor --ping calls GET <base>/models on each configured
provider to check that the server answers.
Vercel AI SDK versions
Every built-in provider exceptpi is an adapter over the Vercel AI SDK
(ai). What works
on each major today:
The peer ranges accept all three majors. Pair each with its
provider packages:
The install hint of a missing provider package and
lousho doctor name the
version for the ai you have installed, and lousho doctor flags a mismatched
pair (for example ai 7 with @ai-sdk/openai 1.x). Only lousho doctor and
fromAiSdk() check the pairing: a built-in provider (a model string) loads
whatever major is installed without checking it, so run lousho doctor to catch
a mismatch before the first call. OllamaProvider loads
ollama-ai-provider on ai 4 and ollama-ai-provider-v2 on ai 6/7; the v2
package peers on zod 4, which the SDK accepts, so install zod 4
with it (zod 3 projects use Ollama with ai 4). lousho init scaffolds ai@^7.0.0 with @ai-sdk/*@^4.0.0 for
OpenAI, Anthropic, Google and OpenRouter, and ai@^7.0.0 with ollama-ai-provider-v2@^4.0.0 and zod 4 for Ollama.
CI installs and tests ai 4, 6 and 7, each with zod 3 and zod 4 (six pairings;
ai 4 with zod 4 is the one pairing the provider packages do not declare), with
the real ollama-ai-provider-v2 on ai 6 and 7 in the zod 4 jobs.
OpenRouter uses @ai-sdk/openai against OpenRouter’s base URL. From
@ai-sdk/openai 2 on, the default openai(modelId) call targets the Responses
API, which OpenRouter does not implement, so OpenRouterProvider asks for the
Chat Completions model (provider.chat(modelId)) on every major; it works on
ai 4, 6 and 7 with the pairing above.
OllamaProvider takes the server’s base URL (baseURL, or OLLAMA_BASE_URL
for resolveProvider()) and appends /api to a bare host, as both Ollama
packages expect it: http://host:11434 and http://host:11434/ become
http://host:11434/api. A URL that already ends in /api (or /api/), or has
any other path such as a reverse-proxy prefix, is used as it is.
generate() and stream() pick the call shape from the installed ai
module: when it exports stepCountIs (v5 and later), the request is sent in
the v6/v7 shape and the result is read back into the same GenerateResult
or StreamResult:
- messages become
ModelMessages: a tool call’s arguments are itsinput, a tool result is anoutput(jsonortext,error-jsonorerror-textwhen the tool failed), and an image or file part’smimeTypeis itsmediaType. System messages are kept in place (allowSystemInMessages). maxTokensis sent asmaxOutputTokens, one step asstopWhen: stepCountIs(1), tools asinputSchema(a zod schema as it is, a JSON Schema throughjsonSchema()) with noexecute, andresponseFormatas a JSON-modeoutputthat leaves the reply text alone.- usage comes from
inputTokens/outputTokens/totalTokens, withcachedInputTokensfrominputTokenDetails.cacheReadTokensandreasoningTokensfromoutputTokenDetails.reasoningTokens. stream()reads the SDK’sfullStreamon either major into the same chunks:text-delta(v6/v7text, v4textDelta),tool-call(the whole call v6/v7 sends aftertool-input-start/-delta/-end, or one assembled from that input when the model sends none), andfinishwith the SDK’s finish reason and the usage above (v6/v7totalUsage). Anerrorpart rejects the stream with its error, on v4 too (v4 used to end such a stream with finish reasonerror), and anabortpart with the signal’s reason. Reasoning deltas are reported asreasoning-deltachunks and surface asreasoning.*events (see Reasoning).
0.0.x/1.x packages produce models ai v7 rejects. Image
parts are sent as file parts with an image media type (the shape ai v6/v7
takes for images, so there is no deprecation warning). ai v5 is not supported:
lousho doctor fails an installed ai 5 with an upgrade hint (move to ai 6
or 7 and the matching @ai-sdk/* major).
Where each provider runs
All providers run on Node. Thecloudflare-worker deploy target supports
mock, openai, anthropic and openrouter (key from the
OPENROUTER_API_KEY binding); google, ollama and pi need the node-server or
docker target (see Cloudflare Workers).