gbrain/skills/migrations/v0.14.0.md

version, feature_pitch

version

feature_pitch

0.14.0

headline description recipe tiers Move deterministic crons off the LLM gateway. Zero tokens per fire. OpenClaw operators: your gateway pins at 100% CPU because most cron jobs boot a full Opus session to do deterministic work (API fetch, token refresh, scrape + markdown write) that doesn't need reasoning. v0.14.0 adds a `shell` job type to Minions so those jobs run as child processes under the existing Minions worker. ~60% gateway CPU reduction at typical scale. Retry, backoff, DLQ, unified `gbrain jobs list` visibility, all free. The LLM-reasoning crons stay on the gateway where they belong. docs/guides/minions-shell-jobs.md null

v0.14.0 Migration: Adopt the shell job type

Audience: host agents (OpenClaw deployments, Hermes operators, future hosts) reading this AFTER gbrain apply-migrations has run. There is no mechanical orchestrator for this release — every change described here is host-specific and requires operator judgment per cron. Shell jobs are off by default on upgrade (nothing breaks). This file walks the host agent through enabling and adopting them.

Iron rules

1. Never auto-rewrite the operator's crontab. Crontab entries are host-specific code per the CLAUDE.md "host-specific code" exception. Every rewrite is an explicit human approval, per cron, with a diff.
2. LLM-requiring crons stay on the gateway. If the cron's work includes reasoning (sentiment classification, triage, synthesis, prose generation), it belongs on the gateway. Moving it to a shell job throws away the LLM session the cron needs.
3. Deterministic crons are the only candidates. API fetch, token refresh, database read + markdown write, HTTP refresh call, scrape. If the script is node scripts/x.mjs or curl ... | jq or similar and does zero LLM work, it's a candidate.

Step 1: Enable the worker

Pick the engine the operator is on:

Postgres (most OpenClaw/Hermes deployments):

# In the worker bootstrap, export the env flag and run the daemon:
GBRAIN_ALLOW_SHELL_JOBS=1 gbrain jobs work

The worker claims shell jobs from the queue and executes them. Retries, backoff, and dead-letter all work the same as sync/embed jobs.

PGLite: no persistent worker, per-tick inline execution only:

# Every crontab invocation must use --follow; PGLite's worker daemon
# exits immediately due to exclusive file lock.
GBRAIN_ALLOW_SHELL_JOBS=1 gbrain jobs submit shell \
  --params '{"cmd":"...","cwd":"..."}' --follow

Step 2: Audit the operator's cron manifest

Read the operator's cron manifest. Typical locations:

• ~/.claude/cron/jobs.json (OpenClaw)
• scripts/service-manager.sh in the host repo
• System crontab (crontab -l)

For each entry, classify:

Pattern	Class	Action
agentTurn <skill> or any OpenClaw-dispatched LLM skill	LLM-requiring	Leave as-is. Needs gateway.
node scripts/*.mjs that hits an API and writes markdown	Deterministic	Propose shell-job rewrite.
Token refresh (ycli token-refresh, x-oauth2-refresh)	Deterministic	Propose shell-job rewrite.
Scrape + write (frameio-scan, flight-tracker)	Deterministic	Propose shell-job rewrite.
Audio transcription or any LLM-dependent extract	LLM-requiring	Leave as-is.
bash wrapper scripts that may call LLM tools internally	Ambiguous	Ask the operator. Don't assume.

Step 3: Propose rewrites per cron

For each deterministic cron, propose the exact rewrite with a diff. Show the operator both sides. Let them approve per-cron, not in bulk.

Before (LLM gateway):

OpenClaw cron: x-garrytan-unified, 3 13,16,19,22,1,4,7,10 * * *
  → runs agentTurn x-garrytan-unified
  → boots Opus context, invokes script, returns

After (Minions worker):

3 13,16,19,22,1,4,7,10 * * * \
  gbrain jobs submit shell \
    --params '{"cmd":"node /data/.openclaw/workspace/scripts/x-garrytan-daily.mjs","cwd":"/data/.openclaw/workspace"}' \
    --max-attempts 3 --timeout-ms 300000

Rewrite rules:

• cwd is required and must be an absolute path. Operator picks it. It should be the directory the script expects to run in (the host repo root, typically).
• --max-attempts 3 matches the default Minions retry policy. Override if the script is non-idempotent and should only run once per fire.
• --timeout-ms N caps the child's wall-clock runtime. Set to the 95th percentile of the script's observed runtime, plus slack. Examples: token refresh → 30s; API fetch → 300s; scrape → 600s.
• PGLite operators: add --follow to every line. Skip Step 1.

Step 4: Secrets that the script needs

Shell jobs receive a minimal env allowlist by default: PATH, HOME, USER, LANG, TZ, NODE_ENV. They do NOT inherit OPENAI_API_KEY, ANTHROPIC_API_KEY, DATABASE_URL, or any other worker env vars.

If a cron's script needs an API key, name it explicitly:

gbrain jobs submit shell \
  --params '{"cmd":"node scripts/yc-sync.mjs","cwd":"/data/.openclaw/workspace","env":{"YC_API_TOKEN":"'"$YC_API_TOKEN"'"}}'

The shell expands $YC_API_TOKEN at submit time. The worker receives the JSON with the literal token value. Audit log does not log env values (keys don't carry sensitive data; values never appear).

Step 5: Verify the first migrated cron

After rewriting ONE cron with the operator's approval:

1. Wait for the next scheduled fire (or trigger manually: gbrain jobs submit shell --params '...' --follow).
2. Check gbrain jobs list --status completed --name shell --limit 5 for the result.
3. gbrain jobs get <id> shows exit_code, stdout_tail, stderr_tail, duration_ms.
4. Compare against the pre-migration behavior: did it do the same work? Same output files changed? Same side effects?

Only after one cron is verified working end-to-end should the operator approve the next batch.

Step 6: Starvation sanity check

If the operator submits shell jobs but forgot to set GBRAIN_ALLOW_SHELL_JOBS=1 on the worker, jobs sit in waiting indefinitely. The CLI warns on submission, but for daemon-style deployments the warning scrolls past. Add this to the operator's ops-check runbook:

gbrain jobs list --status waiting --name shell

If rows pile up here, either (a) no worker has the env flag set, or (b) the worker crashed. Fix by restarting with the flag.

Non-goals (explicitly deferred to later releases)

• Automatic crontab rewrites. Deferred to a future gbrain crontab-to-minions <file> helper. P1 in TODOS.md.
• DB-backed scheduler. minion_schedules table replaces host crontab entirely. P1 in TODOS.md.
• Orphaned-shell-job stats. gbrain jobs stats --orphaned would surface the "no worker with env flag" case. P2 in TODOS.md.
• Configurable buffer sizes. Output tails are fixed at 64KB stdout / 16KB stderr. P2 in TODOS.md.

When to stop

The migration is done when:

1. The worker runs with GBRAIN_ALLOW_SHELL_JOBS=1 (Postgres) or every cron uses --follow (PGLite).
2. Every deterministic cron the operator approved has been rewritten.
3. The operator has verified at least one full cron fire cycle end-to-end and confirmed the output matches pre-migration.
4. gbrain jobs stats shows shell jobs completing at expected rates with few or zero retries.

Gateway CPU should visibly drop after the first few rewrites. That's the signal the adoption is working.