Compacting an agent
WithcreateAgent(), turn compaction on with the compaction option. true
installs the hook with its defaults (prune old tool results above 90% of the
model’s window); an object configures it, and summarizer selects
twoPhaseStrategy() with that model (a "provider/model" string or an
LLMProvider). The vendor/ model prefix chooses the provider; with
OpenRouter use openrouter/<vendor>/<model> (e.g.
openrouter/openai/gpt-4o-mini).
strategy, thresholdPercent, contextWindow,
protectedTokens, reserveOutputTokens and onCompaction (as in the table below) and summarizer; combining
summarizer with strategy is a configuration error (pass
twoPhaseStrategy({ model }) as the strategy instead). createAgent({ hooks })
takes any other AgentHooks, which run before the compaction hook, so
compaction sees what they added. Hooks and compaction apply to send(),
stream() and sessions.
Stream events
agent.stream(), session.stream() and AgentExecutor.stream() report each
compaction as two events, inside the step and before the model call that
triggered it (see Streaming):
Every
compaction.start is followed by exactly one compaction.done. When the
strategy could not shrink anything, tokensAfter equals tokensBefore and
unchanged is true. The hook then compacts that transcript quietly: later
steps that still change nothing emit no events, and the next attempt that does
change something emits its compaction.start / compaction.done pair as usual,
so a run stuck over the threshold does not report a pair on every step. When it
failed (or the summarizer failed and the hook fell back to pruning), error is
set and the run continues. A summary that does not get the conversation under
the threshold - or that makes it larger - is rejected the same way: error is
set, the run keeps the pruned result instead, and appliedStrategy is
'prune-tool-results'. The hook then only prunes that transcript until it has
twice as many messages as at the rejection, and then lets the summarizer try
again, so a summarizer that cannot help costs a few calls per run, not one per
step, and one that was rejected for having too little to fold gets another
chance. strategy is the strategy that ran, the same on compaction.start and
compaction.done ('prune-tool-results' while summaries are paused).
summary is true when old turns were replaced by a summary (the text itself
is not sent; use onCompaction, which the object form also takes). A
send() without a listener emits no events. Hooks add their own events with
ctx.emit?.(...) on the preGenerate context, which exists when the run has
listeners (createAgent({ onEvent }), onAgentEvent) or is streamed.
Compacting a run
createCompactionHook() returns an AgentHook named compaction. Pass it in
createAgent({ hooks }). The recommended setup is the two-phase strategy with
a cheap summarizer model:
estimateTokens (see Models, tokens and cost):
the messages (the system prompt included), the tool definitions and the output
schema. After a call whose provider reported its prompt tokens, the hook also
adds what that report exceeded the estimate by to the next estimate of the same
run, so a provider’s hidden framing is counted too. Tool definitions cannot be
compacted, so the conversation has to fit in what they leave of the threshold.
The compaction.start / compaction.done token counts are these request
sizes. When it is above thresholdPercent of the context window, the hook runs its
strategy (and waits for it, if it is async) and calls onCompaction with the
token counts, the pruned toolCallIds, the summary if there is one and the
strategy name.
Compaction never fails a run. If the strategy throws, or the summarizer call
fails, the run continues (with the pruned conversation, or unchanged) and
onCompaction receives the error.
Inside a run, the summarizer’s model call is part of the run: its tokens are in
result.usage (under the summarizer’s model in byModel, and in
modelCalls), it gets a chat span with lousho.call.purpose: 'compaction'
under the run’s span, and it counts toward the run’s limits budgets.
(session.compact() and compactMessages() run outside a run, so their
summary calls are not counted anywhere.)
The hook rewrites the run’s transcript itself, not a copy: the request’s
messages array is the run’s message list, and the hook changes it in place.
A pruned result or a summary therefore stays in later steps, in checkpoints
(a resumed session does not bring the full history back) and in
result.messages, and it is not compacted again on the next step.
Compaction is lossy: if the model needs a pruned result again, it has to call
the tool again.
Local and unknown models
The threshold is a share of the model’s context window, so compaction needs the real window. For a model the model registry does not know (most local models), the hook assumes128_000 tokens and logs a
one-time console.warn naming the model. A local server usually runs with a
much smaller window, and then compaction never fires before the server rejects
the request. Set the window you loaded the model with: in LM Studio it is the
model’s loaded context length (loaded_context_length from
GET /api/v0/models/<id>), in Ollama the num_ctx option (ollama show <model>
prints the model’s default).
protectedTokens (40,000) is capped at half the
threshold, so the newest ~3,700 tokens of an 8K window stay intact and older
tool results can still be pruned. Set protectedTokens yourself to protect
more or less; keep it below the threshold.
Tool search reads the window the same way and warns the same
way; its own setting is toolSearch.contextWindow.
Advanced: the executor API
Registering the hook in aHookRegistry is for code that calls the executor
directly (see the executor API):
Strategies
Pruning tool results
pruneToolResultsStrategy() replaces the content of each tool result older
than the protected tail with a short stub that names the call, such as
[pruned: search({"query":"deploy"}) result, 18234 chars; already read]
(arguments longer than 120 characters are cut), so the model knows what it
already fetched and does not call the same tool again just to see the result.
Markers in the older [pruned: search result, 18234 chars] form are
recognized and left alone. It never changes:
- system messages, user messages (the first one included) or assistant turns, so every tool call keeps its result message and the transcript stays valid for every provider;
- the results of the latest assistant turn, which the model has not read yet,
even when they are bigger than
protectedTokens; - pinned results, results that are already a marker, or results shorter than a marker.
Summarizing old turns
summarizeStrategy() sends the messages before the protected tail to the
summarizer with a focused instruction and replaces them with one user
message:
The summary keeps the transcript valid for every provider:
- system messages and pinned messages stay where they are, and the summary goes where the first summarized message was;
- an assistant turn is summarized or kept together with all its tool results, so no tool call loses its result (and no result loses its call);
- the last message and the latest assistant turn’s results are always kept.
error.
Two phases
twoPhaseStrategy() takes the same options. It prunes tool results first,
which is free, and calls the summarizer only when the pruned conversation is
still above thresholdPercent of the context window. The summarizer then
reads the pruned conversation. If it fails, the pruned conversation is kept and
error is reported.
Pinning messages
pinMessage(message) returns a copy of the message marked as pinned (it sets
metadata.pinned: true; Message.metadata is application data that is never
sent to the model). The built-in strategies never prune or summarize a pinned
message; a pinned tool result keeps its whole assistant turn. Pin what the
agent must always see in full, such as the task statement or a key document:
isPinned() too.
Compacting by hand
compactMessages(messages, options) runs a strategy once, whatever the size
of the conversation, and resolves to a new array (its input is not modified).
Use it to shrink a stored transcript before you continue it:
Compacting a session
session.compact(options?) compacts a session’s transcript
now, whatever its size, saves it to the session’s store (memory, file, SQLite
or KV) and resolves to { messagesBefore, messagesAfter, tokensBefore, tokensAfter, strategy, error? }.
It runs options.strategy, else the strategy of agent.session({ compaction })
(the same value as createAgent({ compaction }), and the agent’s own
compaction when the session does not set one), else prunes old tool results.
Pinned messages stay. A session with no messages resolves without doing
anything. It rejects with LOUSHO_SESSION_BUSY while a turn is running, and
with LOUSHO_SESSION_TURN_PENDING / LOUSHO_SESSION_AWAITING_APPROVAL while a
durable turn is unfinished. Listeners added with session.on() get
compaction.start and compaction.done with trigger: 'manual'.
session.clear() empties the transcript instead, deletes every turn’s
checkpoint and checkpoint history, and emits context.cleared.
Writing a strategy
A strategy is a name and acompact() function, which may be async. It
receives the messages, a token counter for the request’s model, the context
window, protectedTokens, thresholdTokens (the size to get under) and the
run’s abort signal, and returns the new messages with before/after token
counts (and optionally a summary or an error). Return the input array
unchanged when there is nothing to do: the hook then leaves the run alone and
does not call onCompaction.
compaction.start / compaction.done events as
the built-in ones. onCompaction (on createCompactionHook()) still receives
the summary text and the Error.