Self-healing and self-improvement

​

Recover a run now

import { createAgent, createProgressGuard } from 'lousho';
import { SqliteStore } from 'lousho/sqlite';

const agent = createAgent({
  provider,
  store: new SqliteStore('./.lousho/jobs.db'),
  hooks: [createProgressGuard()],
});

const recovery = agent.recover({
  retryWithDiagnosis: true,
  onRecovered: (run) => console.log('recovered', run.sessionId),
  onQuarantined: (run) => console.warn('needs a person', run.sessionId, run.lastError?.message),
  onOutage: (paused) => console.warn(paused ? 'provider down, paused' : 'provider back'),
  onError: (error, run) => console.error('recovery failed', run?.sessionId, error),
});

• Backoff. A run that failed is skipped until its backoff passes: intervalMs doubled for each failed attempt in a row, up to 15 minutes.
• Quarantine. After maxAttempts failures in a row (default 10), the loop stops resuming the run and calls onQuarantined(run) once. agent.pending() still lists it, and agent.resume() still runs it, so a person can decide. Saving progress resets the count.
• Retry with a diagnosis. With retryWithDiagnosis: true (default false), a run that reaches the cap first gets one more try: a fork named <id>.heal, resumed from the same step, with a note saying why the run failed. The note quotes the error as data and caps its length. If the fork finishes, the original is deleted and onRecovered reports the original’s id. If the fork also reaches the cap, both are quarantined. This applies to send() runs, not session turns, and the store must keep a checkpoint history.
• Errors that won’t go away. A run whose last error has retryable: false, such as a 400 from the provider, is left alone.
• Provider outages. After 3 provider failures in a row, across any runs, the loop pauses and calls onOutage(true). It then tries one run at a time, at growing intervals, and calls onOutage(false) once one gets through. Runs aren’t quarantined for provider failures while it’s paused. A bad API key doesn’t count as an outage.
• Approvals. A run paused on an approval or a question is left for a person.

​

Stop a run that makes no progress

import { createAgent, createProgressGuard } from 'lousho';

const agent = createAgent({ provider, hooks: [createProgressGuard()] });

for await (const event of agent.stream('Where is order A-1?')) {
  if (event.type === 'heal.action') console.log(event.action, event.toolName, event.reason);
}

​

Watch it

• The recover() callbacks: onRecovered, onQuarantined, onOutage and onError. A throwing callback doesn’t stop the loop.
• agent.pending() lists every unfinished run with lastError and attempts, including quarantined ones.
• A result the progress guard replaced is marked on the tool’s span as lousho.tool.replaced_by_hook, and a denial is recorded as the span’s error. See Tracing and observability for spans and the local observe daemon.

​

Propose edits for later

import { createAgent } from 'lousho';
import { formatImproveResult, improveAgent, renderTranscript } from 'lousho/improve';

const result = await improveAgent({
  harness: { instructions: 'You answer from the docs.', skills: {} },
  cases: ['discount-question'],
  holdout: ['refund-question'],
  editor: { provider },
  maxTokens: 500_000,
  probe: async (candidate, caseId) => {
    const agent = createAgent({ provider, instructions: candidate.instructions });
    const run = await agent.send(caseId);
    return { passed: (run.text ?? '').includes('cannot'), transcript: renderTranscript(run.messages), tokens: run.usage?.totalTokens ?? 0, ended: run.finishReason };
  },
});

if (result.outcome === 'proposal') console.log(formatImproveResult(result));

• It only returns a proposal. An outcome of proposal comes with a diff. A person reviews it, with formatImproveResult() for a readable card, and applies it by hand.
• It can only change instructions and skills. It can’t change tools, permissions or limits.
• It tests before it proposes. Each candidate runs your cases again and must improve at least one training case without making any case worse, including held-out cases the editors never see. It also checks the cost per passing run against maxCostIncrease (default 0.5).
• Your probe runs the agent, so it owns side effects. Add createShadowHook({ readOnly: [...] }) unless every tool is safe to run.

​

How the layers fit together

​

Live evidence (S29 and S30)

• Every scenario that tested something passed. planted-editor passed 2 of 2, and one more attempt never reached the editors, so it wasn’t counted.
• The audit log verified in every attempt directory.
• The report’s verdict is INCOMPLETE: the go/no-go bars need 5 to 10 attempts per scenario.