Minions v7 + v0.11.1 canonical migration + skillify (#130)
* feat: add minion_jobs schema, migration v5, and executeRaw to BrainEngine Foundation for the Minions job queue system. Adds: - minion_jobs table (20 columns) with CHECK constraints, partial indexes, and RLS. Inspired by BullMQ's job model, adapted for Postgres. - Migration v5 creates the table for existing databases. - executeRaw<T>() method on BrainEngine interface for raw SQL access, needed by the Minions module for claim queries (FOR UPDATE SKIP LOCKED), token-fenced writes, and atomic stall detection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Minions job queue — queue, worker, backoff, types BullMQ-inspired Postgres-native job queue built into GBrain. No Redis. No external dependencies. Postgres transactions replace Lua scripts. - MinionQueue: submit, claim (FOR UPDATE SKIP LOCKED), complete/fail (token-fenced), atomic stall detection (CTE), delayed promotion, parent-child resolution, prune, stats - MinionWorker: handler registry, lock renewal, graceful SIGTERM, exponential backoff with jitter, UnrecoverableError bypass - MinionJobContext: updateProgress(), log(), isActive() for handlers - 8-state machine: waiting/active/completed/failed/delayed/dead/ cancelled/waiting-children Patterns stolen from: BullMQ (lock tokens, stall detection, flows), Sidekiq (dead set, backoff formula), Inngest (checkpoint/resume). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: 43 tests for Minions job queue Full coverage of the Minions module against PGLite in-memory: - Queue CRUD (9): submit, get, list, remove, cancel, retry, duplicate - State machine (6): waiting→active→completed/failed, retry→delayed→waiting - Backoff (4): exponential, fixed, jitter range, attempts_made=0 edge - Stall detection (3): detect stalled, counter increment, max→dead - Dependencies (5): parent waits, fail_parent, continue, remove_dep, orphan - Worker lifecycle (5): register, start-without-handlers, claim+execute, non-Error throws, UnrecoverableError bypass - Lock management (3): renewal, token mismatch, claim sets lock fields - Claim mechanics (4): empty queue, priority ordering, name filtering, delayed promotion timing - Cancel & retry (2): cancel active, retry dead Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Minions CLI commands and MCP operations Wire Minions into the GBrain CLI and MCP layer: CLI (gbrain jobs): submit <name> [--params JSON] [--follow] [--dry-run] list [--status S] [--queue Q] [--limit N] get <id> — detailed view with attempt history cancel/retry/delete <id> prune [--older-than 30d] stats — job health dashboard work [--queue Q] [--concurrency N] — Postgres-only worker daemon 6 MCP operations (contract-first, auto-exposed via MCP server): submit_job, get_job, list_jobs, cancel_job, retry_job, get_job_progress Built-in handlers: sync, embed, lint, import. --follow runs inline. Worker daemon blocked on PGLite (exclusive file lock). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update project documentation for Minions job queue CLAUDE.md: added Minions files to key files, updated operation count (36), BrainEngine method count (38), test file count (45), added jobs CLI commands. CHANGELOG.md: added Minions entry to v0.10.0 (background jobs, retry, stall detection, worker daemon). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Minions v2 — agent orchestration primitives (pause/resume, inbox, tokens, replay) Adds the foundation for Minions as universal agent orchestration infrastructure. GBrain's Postgres-native job queue now supports durable, observable, steerable background agents. The OpenClaw plugin (separate repo) will consume these via library import, not MCP, for zero-latency local integration. ## New capabilities - **Concurrent worker** — Promise pool replaces sequential loop. Per-job AbortController for cooperative cancellation. Graceful shutdown waits for all in-flight jobs via Promise.allSettled. - **Pause/resume** — pauseJob clears the lock and fires AbortSignal on active jobs. Handlers check ctx.signal.aborted and exit cleanly. resumeJob returns paused jobs to waiting. Catch block skips failJob when signal.aborted. - **Inbox (separate table)** — minion_inbox table for sidechannel messages. sendMessage with sender validation (parent job or admin). readInbox is token-fenced and marks read_at atomically. Separate table avoids row bloat from rewriting JSONB on every send. - **Token accounting** — tokens_input/tokens_output/tokens_cache_read columns. updateTokens accumulates; completeJob rolls child tokens up to parent. USD cost computed at read time (no cost_usd column — pricing too volatile). - **Job replay** — replayJob clones a terminal job with optional data overrides. New job, fresh attempts, no parent link. ## Handler contract additions MinionJobContext now provides: - `signal: AbortSignal` — cooperative cancellation - `updateTokens(tokens)` — accumulate token usage - `readInbox()` — check for sidechannel messages - `log()` — now accepts string or TranscriptEntry ## MCP operations added pause_job, resume_job, replay_job, send_job_message — all auto-generate CLI commands and MCP server endpoints. ## Library exports package.json exports map adds ./minions and ./engine-factory paths so plugins can `import { MinionQueue } from 'gbrain/minions'` for direct library use. ## Instruction layer (the teaching) - skills/minion-orchestrator/SKILL.md — when/how to use Minions, decision matrix, lifecycle management, anti-patterns - skills/conventions/subagent-routing.md — cross-cutting rule: all background work goes through Minions - RESOLVER.md — trigger entries for agent orchestration - manifest.json — registered ## Schema migration v6 Additive: 3 token columns, paused status, minion_inbox table with unread index. Full Postgres + PGLite support. No backfill needed. ## Tests 65 tests (was 43): pause/resume (5), inbox (6), tokens (4), replay (4), concurrent worker context (3), plus all existing coverage. ## What's NOT in this commit Deferred to follow-up PRs: - LISTEN/NOTIFY subscribe (needs real Postgres E2E) - Resource governor (depends on concurrent worker stress testing) - Routing eval harness (needs API keys + benchmark data) - OpenClaw plugin (separate @gbrain/openclaw-minions-plugin repo) See docs/designs/MINIONS_AGENT_ORCHESTRATION.md for full CEO-approved design. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): migration v7 — agent_parity_layer schema Adds columns on minion_jobs (depth, max_children, timeout_ms, timeout_at, remove_on_complete, remove_on_fail, idempotency_key) plus the new minion_attachments table. Three partial indexes for bounded scans: idx_minion_jobs_timeout, idx_minion_jobs_parent_status, and uniq_minion_jobs_idempotency. Check constraints enforce non-negative depth and positive child cap / timeout. Additive migration — existing installs pick it up via ensureSchema on next use. No user action required. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(minions): extend types for v7 parity layer Extends MinionJob with depth/max_children/timeout_ms/timeout_at/ remove_on_complete/remove_on_fail/idempotency_key. Extends MinionJobInput with the same options plus max_spawn_depth override. Adds MinionQueueOpts (maxSpawnDepth default 5, maxAttachmentBytes default 5 MiB). Adds AttachmentInput/Attachment shapes and ChildDoneMessage in the InboxMessage union. rowToMinionJob updated to pick up the new columns. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(minions): attachments validator New module validateAttachment() gates every attachment write. Rejects empty filenames, path traversal (.., /, \), null bytes, oversized content (5 MiB default, per-queue override), invalid base64, and implausible content_type headers. Returns normalized { filename, content_type, content (Buffer), sha256, size } on success. The DB also enforces UNIQUE (job_id, filename) as defense-in-depth for concurrent addAttachment races — JS-only checks are not sufficient. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(minions): queue v7 — depth, child cap, timeouts, cascade, idempotency, child_done Wraps completeJob and failJob in engine.transaction() so parent hook invocations (resolveParent, failParent, removeChildDependency) fold into the same transaction as the child update. A process crash between child and parent can't strand the parent in waiting-children anymore. Adds v7 behaviors: - Depth tracking. add() computes depth = parent.depth + 1 and rejects past maxSpawnDepth (default 5). - Per-parent child cap. add() takes SELECT ... FOR UPDATE on the parent, counts non-terminal children, rejects when count >= max_children. NULL max_children = no cap. - Per-job wall-clock timeout. claim() populates timeout_at when timeout_ms is set. New handleTimeouts() dead-letters expired rows with error_text='timeout exceeded'. Terminal — no retry. - Cascade cancel. cancelJob() walks descendants via recursive CTE with depth-100 runaway cap. Returns the root row. Re-parented descendants (parent_job_id NULL) are naturally excluded. - Idempotency. add() uses INSERT ... ON CONFLICT (idempotency_key) DO NOTHING RETURNING; falls back to SELECT when RETURNING is empty. Same key always yields the same job id. - child_done inbox. completeJob inserts {type:'child_done', child_id, job_name, result} into the parent's inbox in the same transaction as the token rollup, guarded by EXISTS so terminal/deleted parents skip without FK violation. New readChildCompletions(parent_id, lock_token, since?) helper; token-fenced like readInbox. - removeOnComplete / removeOnFail. Deletes the row after the parent hook fires, so parent policy sees consistent state. - Attachment methods. addAttachment validates via validateAttachment then INSERTs; UNIQUE (job_id, filename) backs the JS dup check. listAttachments, getAttachment, deleteAttachment round out the API. Fixes pre-existing inverted status bug: add() now puts children in waiting/delayed (not waiting-children) and atomically flips the parent to waiting-children in the same transaction. Tests no longer need manual UPDATE workarounds. Two correctness fixes: - Sibling completion race. Under READ COMMITTED, two grandchildren completing concurrently each saw the other as still-active in the pre-commit snapshot and neither flipped the parent. Fixed by taking SELECT ... FOR UPDATE on the parent row at the start of completeJob and failJob transactions, serializing siblings on the parent lock. - JSONB double-encode. postgres.js conn.unsafe(sql, params) auto- JSON-encodes parameters. Calling JSON.stringify(obj) first stored a JSON string literal (jsonb_typeof=string) and broke payload->>'key' queries silently. Removed JSON.stringify from three call sites (child_done inbox post, updateProgress, sendMessage). PGLite tolerated both forms so unit tests missed it — real-PG E2E caught it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(minions): worker — timeout safety net + handleTimeouts tick Worker tick now calls handleStalled() first, then handleTimeouts() — stall requeue wins over timeout dead-letter when both could fire in the same cycle. handleTimeouts() guards on lock_until > now() so stalled jobs take the retryable path. launchJob schedules a per-job setTimeout(timeout_ms) that fires ctx.signal as a best-effort handler interrupt. The timer is always cleared in .finally so process exit isn't delayed by a dangling timer. Handlers that respect AbortSignal stop cleanly; handlers that ignore it still get dead-lettered by the DB-side handleTimeouts. Removed post-completeJob and post-failJob parent-hook calls from the worker — those are now inside the queue method transactions. Worker becomes simpler and crash-safer. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(minions): 33 new unit tests for v7 parity layer Covers depth cap, per-parent child cap, timeout dead-letter, cascade cancel (including the re-parent edge case), removeOnComplete / removeOnFail, idempotency (single + concurrent), child_done inbox (posted in txn + survives child removeOnComplete + since cursor), attachment validation (oversize, path traversal, null byte, duplicates, base64), AbortSignal firing on pause mid-handler, catch-block skipping failJob when aborted, worker in-flight bookkeeping, token-rollup guard when parent already terminal, and setTimeout safety-net cleanup. Existing tests updated to remove the inverted-status manual UPDATE workarounds that the add() fix made obsolete. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(e2e): Minions v7 concurrency + OpenClaw resilience coverage minions-concurrency.test.ts spins two MinionWorker instances against the test Postgres, submits 20 jobs, and asserts zero double-claims (every job runs exactly once). This is the only test that actually proves FOR UPDATE SKIP LOCKED under real concurrency — PGLite runs on a single connection and can't exercise the race. minions-resilience.test.ts covers the six OpenClaw daily pains: 1. Spawn storm caps enforce under concurrent submit. 2. Agent stall → handleStalled() requeues; handleTimeouts() skips (lock_until guard). 3. Forgotten dispatches recoverable via child_done inbox. 4. Cascade cancel stops grandchildren mid-flight. 5. Deep tree fan-in (parent → 3 children → 2 grandchildren each) completes with the full inbox chain. 6. Parent crash/recovery resumes from persisted state. helpers.ts extends ALL_TABLES with minion_attachments, minion_inbox, and minion_jobs (FK dependents first) so E2E teardown doesn't leak rows between runs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore: release v0.11.0 — Minions v7 agent orchestration primitives Bumps VERSION / package.json to 0.11.0. Adds CHANGELOG entry covering depth tracking, max_children, per-job timeouts, cascade cancel, idempotency keys, child_done inbox, removeOnComplete/Fail, attachments, migration v7, plus the two correctness fixes (sibling completion race and JSONB double-encode). TODOS.md captures the four v7 follow-ups: per-queue rate limiting, repeat/cron scheduler, worker event emitter, and waitForChildren convenience helpers. 1066 unit + 105 E2E = 1171 tests passing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(minions): unify JSONB inserts, tighten nullish coalescing Three non-blocker cleanups from post-ship review of v0.11.0: - queue.ts add() and completeJob(): pre-stringifying with JSON.stringify while other sites pass raw objects with $n::jsonb casts. postgres.js double-encodes if you stringify first — works on PGLite (text→JSONB auto-cast), fails silently on real PG. Unify on raw object + explicit $n::jsonb cast. - queue.ts readChildCompletions: since clause used sent_at > $2 relying on PG's implicit text→TIMESTAMPTZ coercion. Explicit $2::timestamptz is safer and clearer. - types.ts rowToMinionJob: parent_job_id used || which coerces 0 to null. Harmless today (SERIAL IDs start at 1) but ?? is semantically correct. All 110 unit tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(minions): updateProgress missed $1::jsonb cast in unification Residual from c502b7e — updateProgress was the only remaining JSONB write without the explicit ::jsonb cast. Not broken (implicit cast works) but breaks the convention the prior commit unified everywhere else. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * doc: Minions v7 skill count + jobs subcommands (26 skills) README: bump skill count 25 → 26, add minion-orchestrator row, add `gbrain jobs` command family block so v0.11.0's headline feature is actually discoverable from the top-level commands reference. CLAUDE.md: unit test count 48 → 49 (minions.test.ts expanded), skill count 25 → 26, add minion-orchestrator to Key files + skills categorization, expand MinionQueue one-liner to cover v7 primitives (depth/child-cap, timeouts, idempotency, child_done inbox, removeOnComplete/Fail). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: Minions adoption UX — smoke test + migration + pain-triggered routing Teach OpenClaw when to reach for Minions vs native subagents. Ship three pieces so upgrading from v0.10.x actually lands for real users: - `gbrain jobs smoke` — one-command health check that submits a `noop` job, runs a worker, verifies completion, and prints engine-aware guidance (PGLite installs get the "daemon needs Postgres, use --follow" note). Fails loud if schema's below v7 so the user knows to `gbrain init`. - `skills/migrations/v0.11.0.md` — post-upgrade migration file the auto-update agent reads. Six steps: apply schema, run smoke, ask user via AskUserQuestion which mode they want (always / pain_triggered / off), write to `~/.gbrain/preferences.json`, sanity-check handlers, mark done. Completeness scores on each option so the recommendation is explicit. - `skills/conventions/subagent-routing.md` rewritten — was a "MUST use Minions for ALL background work" mandate, now reads preferences.json on every routing decision and branches on three modes. Mode B (pain_triggered) is the default: keep subagents until gateway drops state, parallel > 3, runtime > 5min, or user expresses frustration. Then pitch the switch in-session with a specific script. Rename pass: "Minions v7" → "Minions" in README (JOBS block), TODOS.md (P1 section header + depends-on), CHANGELOG.md v0.11.0 entry. v7 stays as the internal schema version in code/migration contexts. The product name is just Minions. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * doc(readme): promote Minions — 6 OpenClaw pains + how each is fixed The one-line mention in the skills table wasn't doing the work. Added a dedicated section between "How It Works" and "Getting Data In" that leads with the six multi-agent failures every OpenClaw user hits daily (spawn storms, hung handlers, forgotten dispatches, unstructured debugging, gateway crashes, runaway grandchildren) and maps each pain to the specific Minions primitive that fixes it. Includes the smoke test command, the adoption default (pain_triggered), and a pointer to skills/minion-orchestrator for the full patterns. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(bench): add harness for Minions vs OpenClaw subagent dispatch Shared harness (openclawDispatch + minionsHandler) using matching claude-haiku-4-5 calls on both sides so the delta measures queue+ dispatch overhead on top of identical LLM work. Includes statsFromResults (p50/p95/p99) and formatStats helpers. Uses `openclaw agent --local` embedded mode; does not test gateway multi-agent fan-out (documented in the harness header). * test(bench): durability under SIGKILL — Minions vs OpenClaw --local Headline bench for the claim: when the orchestrator dies mid-dispatch, Minions rescues via PG state + stall detection; OpenClaw --local loses in-flight work outright. Minions side: seed 10 active+expired-lock rows (exact state a SIGKILLed worker leaves) then run a rescue worker. Expect 10/10 completed. OpenClaw side: spawn 10 `openclaw agent --local` in parallel, SIGKILL each at 500ms, count pre-kill delivered output. Expect 0/10 — no persistence layer, nothing to recover. Budget: ~$0 (Minions handlers sleep 10ms; OC calls die at 500ms so partial LLM billing is negligible). * test(bench): per-dispatch throughput — Minions vs OpenClaw --local 20 serial dispatches each side, identical claude-haiku-4-5 call with the same trivial prompt. p50/p95/p99 reported via statsFromResults. Serial (not parallel) so the per-dispatch cost is measured honestly and LLM token spend stays bounded (~$0.08 total). Minions: one queue, one worker, one concurrency. Submit → poll to completion before next submit. OpenClaw: N sequential `openclaw agent --local` spawns. * test(bench): fan-out — Minions 10-wide concurrency vs 10 parallel OC spawns Parent dispatches 10 children, waits for all to return. Minions uses worker concurrency=10 sharing one warm process; OpenClaw parallel `openclaw agent --local` spawns, each boots its own runtime. 3 runs × 10 children per run. Reports ok count and wall time per run plus summary. Honest caveat documented: does not test OC gateway multi-agent fan-out — that needs a custom WS client and LLM-backed parent agent. This measures what users script today. Budget: ~$0.12 LLM spend. * test(bench): memory — 10 in-flight subagents, single-proc vs 10-proc cost Measures resident memory for keeping 10 subagents in flight. Minions: one worker process, concurrency=10 with handlers that park on a promise — sample RSS of the test process via process.memoryUsage(). OpenClaw: 10 parallel `openclaw agent --local` processes, sum their RSS via `ps -o rss=`. Handlers are cheap sleeps, no LLM — we want harness memory, not LLM client state. Budget: $0. * test(bench): fan-out — don't gate on OC success rate, report numbers Initial run showed OC parallel `--local` at 10-wide hits 40% failure rate (17/30 across 3 runs). That's the finding, not a test bug — process startup stampede + LLM rate limits. Bench now prints error samples and reports the numbers instead of gating. Minions side still gates at 90% (30/30 observed in practice). * doc(benchmarks): Minions vs OpenClaw --local subagent dispatch Real numbers on four claims: durability, throughput, fan-out, memory. Same claude-haiku-4-5 call on both sides so the delta is queue+dispatch+ process cost on top of identical LLM work. Headline: Minions rescues 10/10 from a SIGKILLed worker in 458ms while OpenClaw --local loses all 10; ~10× faster per dispatch (778ms p50 vs 8086ms p50); ~21× faster at 10-wide fan-out AND 100% reliable vs OC's 43% failure rate; 2 MB vs 814 MB to keep 10 subagents in flight. Honest caveats section covers what this doesn't test (OC gateway multi-agent, load tests, other models). Fully reproducible via test/e2e/bench-vs-openclaw/. * doc(readme): inject Minions vs OpenClaw bench numbers Headline deltas now in the Minions section: 10/10 vs 0/10 on crash, ~10× faster per dispatch, ~21× faster fan-out at 10-wide with 0% failure vs 43%, ~400× less memory. Links to the full bench doc. Prose first said Minions "fixes all six pains." Now it shows the numbers that prove it. * bench: production Wintermute benchmark — Minions 753ms vs sub-agent timeout Real deployment: 45K-page brain on Render+Supabase. Task: pull 99 tweets, write brain page, commit, sync. Minions: 753ms, $0. Sub-agent: gateway timeout (>10s, couldn't even spawn under production load). Also: 19,240 tweets backfilled across 36 months in 15 min at $0. Sub-agents would cost $1.08 and fail 40% of spawns. * bench: tweet ingestion — Minions 719ms vs OpenClaw 12.5s (17×) Production benchmark with runnable test code: - test/e2e/bench-vs-openclaw/tweet-ingest.bench.ts (reusable) - docs/benchmarks/2026-04-18-tweet-ingestion.md (publishable) Task: pull 100 tweets from X API, write brain page, commit, sync. Minions: 719ms mean, $0, 100% success. OpenClaw: 12,480ms mean, $0.03/run, 60% success (gateway timeouts). At scale: 36-month backfill, 19K tweets, 15 min, $0 vs est. $1.08. * doc(benchmarks): Wintermute production data point for Minions vs OpenClaw Adds a production-environment data point to the Minions README section: one month of tweet ingest on Wintermute (Render + Supabase + 45K-page brain) ran end-to-end in 753ms for \$0.00 via Minions, while the equivalent sessions_spawn hit the 10s gateway timeout and produced nothing. Full methodology + logs in docs/benchmarks/2026-04-18-minions-vs-openclaw-production.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(core): preferences.ts + cli-util.ts — foundations for v0.11.1 Adds two foundational modules that apply-migrations (Lane A-4), the v0.11.0 orchestrator (Lane C-1), and the stopgap script (Lane C-4) all depend on. - src/core/preferences.ts: atomic-write ~/.gbrain/preferences.json (mktemp + rename, 0o600, forward-compatible for unknown keys) with validateMinionMode, loadPreferences, savePreferences. Plus appendCompletedMigration + loadCompletedMigrations for the ~/.gbrain/migrations/completed.jsonl log (tolerates malformed lines). Uses process.env.HOME || homedir() so $HOME overrides work in CI and tests; Bun's os.homedir() caches the initial value and ignores later mutations. - src/core/cli-util.ts: promptLine(prompt) helper, extracted from src/commands/init.ts:212-224. Shared so init, apply-migrations, and the v0.11.0 orchestrator's mode prompt don't each reinvent it. test/preferences.test.ts: 21 unit tests covering load/save atomicity, 0o600 perms, forward-compat for unknown keys, minion_mode validation, completed.jsonl JSONL append idempotence, auto-ts population, malformed- line tolerance in loadCompletedMigrations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(init): add --migrate-only flag (schema-only, no saveConfig) Context: v0.11.0 migration orchestrators need a safe way to re-apply the schema against an existing brain without risking a config flip. Today running bare `gbrain init` with no flags defaults to PGLite and calls saveConfig, which would silently overwrite an existing Postgres database_url — caught by Codex in the v0.11.1 plan review as a show-stopper data-loss bug. The new --migrate-only path: - loadConfig() reads the existing config (does NOT call saveConfig) - errors out with a clear "run gbrain init first" if no config exists - connects via the already-configured engine, calls engine.initSchema(), disconnects - --json emits structured success/error payloads Everything downstream in the v0.11.1 migration chain (apply-migrations, the stopgap bash script, the package.json postinstall hook) will invoke this flag rather than bare gbrain init. test/init-migrate-only.test.ts: 4 tests covering the no-config error path, --json error payload shape, happy-path with a PGLite fixture (verifies config.json content is byte-identical after the call — the real invariant), and idempotent rerun. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(migrations): TS registry replaces filesystem migration scan Context: Codex flagged that bun build --compile produces a self-contained binary, and the existing findMigrationsDir() in upgrade.ts:145 walks skills/migrations/v*.md on disk — which fails on a compiled install because the markdown files aren't bundled. The plan's fix is a TS registry: migrations are code, imported directly, visible to both source installs and compiled binaries. - src/commands/migrations/types.ts: shared Migration, OrchestratorOpts, OrchestratorResult types. - src/commands/migrations/index.ts: exports the migrations[] array, getMigration(version), and compareVersions() (semver comparator). The feature_pitch data that lived in the MD file frontmatter now lives here as a code constant on each Migration, so runPostUpgrade's post-upgrade pitch printer can consume it without a filesystem read. - src/commands/migrations/v0_11_0.ts: stub orchestrator + pitch. The full phase implementation lands in Lane C-1; for now the stub throws a clear "not yet implemented" so apply-migrations --list (Lane A-4) can still enumerate the migration. test/migrations-registry.test.ts: 9 tests covering ascending-semver ordering, feature_pitch shape invariants, getMigration lookup, and compareVersions edge cases (equal / newer / older / single-digit across major bumps). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): gbrain apply-migrations — migration runner CLI Reads ~/.gbrain/migrations/completed.jsonl, diffs against the TS migration registry, runs pending orchestrators. Resumes status:"partial" entries (the stopgap bash script writes these so v0.11.1 apply-migrations can pick up where it left off). Idempotent: rerunning when up-to-date exits 0. Flags: --list Show applied + partial + pending + future. --dry-run Print the plan; take no action. --yes / --non-interactive Skip prompts (used by runPostUpgrade + postinstall). --mode <a|p|o> Preset minion_mode (bypasses the Phase C TTY prompt). --migration vX.Y.Z Force-run one specific version. --host-dir <path> Include $PWD in host-file walk (default is $HOME/.claude + $HOME/.openclaw only). --no-autopilot-install Skip Phase F. Diff rule (Codex H9): apply when no status:"complete" entry exists AND migration.version ≤ installed VERSION. Previously proposed rule was "version > currentVersion", which would SKIP v0.11.0 when running v0.11.1; regression test in apply-migrations.test.ts pins the correct semantics. Registered in src/cli.ts CLI_ONLY Set; dispatched before connectEngine so each phase owns its own engine/subprocess lifecycle (no double-connect when the orchestrator shells out to init --migrate-only or jobs smoke). test/apply-migrations.test.ts: 18 unit tests covering parseArgs for every flag, indexCompleted/statusForVersion correctness (including stopgap-then- complete transition), and buildPlan's four buckets (applied / partial / pending / skippedFuture) with the Codex H9 regression pinned. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(upgrade): runPostUpgrade tail-calls apply-migrations; postinstall hook Closes the v0.11.0 mega-bug: migration skills never fired on upgrade. `runPostUpgrade` now does two things: 1. Cosmetic: prints feature_pitch headlines for migrations newer than the prior binary. Uses the TS registry (Codex K) instead of walking skills/migrations/*.md on disk — compiled binaries see the same list source installs do. 2. Mechanical: invokes apply-migrations --yes --non-interactive in the same process so Phase F (autopilot install) doesn't hit a subprocess timeout wall. Catches + surfaces errors without failing the upgrade. Also: - Drops the early-return on missing upgrade-state.json (Codex H8). runPostUpgrade now runs apply-migrations unconditionally; it's cheap when nothing is pending. This repairs every broken-v0.11.0 install on their next upgrade attempt. - Bumps the `gbrain post-upgrade` subprocess timeout in runUpgrade from 30s → 300s (Codex H7). A v0.11.0→v0.11.1 migration that has to schema-init + smoke + prefs + host-rewrite + launchd-install exceeds 30s trivially. - Removes now-dead findMigrationsDir + extractFeaturePitch helpers and their filesystem-reading imports (readdirSync, resolve). - src/cli.ts post-upgrade dispatch now awaits the async runPostUpgrade. apply-migrations (Lane A-4): - First-install guard: loadConfig() check at the top. No brain configured = exit silently for --yes / --non-interactive (postinstall stays quiet on fresh `bun add gbrain`); explicit message on --list / --dry-run. package.json: - New `postinstall` script: gbrain --version >/dev/null 2>&1 && gbrain apply-migrations --yes --non-interactive 2>/dev/null || true. The --version sanity check guards against a half-written binary (Codex review criticism). || true prevents `bun update gbrain` failure mid-upgrade. Manual smoke verified: fresh $HOME with no config → apply-migrations --yes silently exits 0; --dry-run prints the one-liner "No brain configured... Nothing to migrate." Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(commands): extract library-level Core functions that throw not exit Codex architecture finding #5: reusing CLI entry-point functions as Minions handler bodies is wrong. If a Minion invokes runExtract / runEmbed / runBacklinks / runLint and the handler hits a process.exit(1), the ENTIRE WORKER process dies — killing every other in-flight job. Handlers need library-level APIs that throw, and the CLI stays a thin wrapper that catches + exits. Per-command shape: - runXxxCore(opts): throws on validation errors, returns structured result. Handler-safe. - runXxx(args): arg parser; calls Core; catches; process.exit(1) on thrown errors. CLI-safe. Shipped: - runExtractCore({ mode, dir, dryRun?, jsonMode? }) → ExtractResult - runEmbedCore({ slug? | slugs? | all? | stale? }) → void - runBacklinksCore({ action, dir, dryRun? }) → BacklinksResult - runLintCore({ target, fix?, dryRun? }) → LintResult sync.ts is already correct — performSync throws; runSync wraps. No change. import.ts deferred to v0.12.0 (its one process.exit fires only on a missing dir arg; handlers always pass a dir, so worker-kill risk is zero in practice). Noted in the plan's Out-of-scope. Smoke verified: all four Core functions throw on invalid mode / missing dir / not-found target instead of exiting the process. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(jobs): Tier 1 handlers + autopilot-cycle (the killer handler) registerBuiltinHandlers now handlers every operation autopilot needs to dispatch via Minions + the single autopilot-cycle handler the autopilot loop actually submits each interval. Existing handlers (sync, embed, lint) rewired to call library-level Core functions directly instead of the CLI wrappers. CLI wrappers call process.exit(1) on validation errors; if a worker claimed a badly-formed job, the WORKER PROCESS would die — killing every in-flight job. Cores throw, so one bad job fails one job. New handlers: - extract → runExtractCore (mode: links|timeline|all, dir) - backlinks → runBacklinksCore (action: check|fix, dir) - autopilot-cycle → THE killer handler. Runs sync → extract → embed → backlinks inline. Each step wrapped in try/catch; returns { partial: true, failed_steps: [...] } when any step fails. Does NOT throw on partial failure — that would trigger Minion retry, and an intermittent extract bug would block every future cycle. Replaces the 4-job parent-child DAG proposed in early plan drafts (Codex H3/H4: parent/child is NOT a depends_on primitive in Minions). import.ts handler still uses the CLI wrapper (runImport) — import's one process.exit fires only on a missing dir arg and the handler always passes a dir; Core extraction deferred to v0.12.0 when Tier 2 refactors happen. registerBuiltinHandlers promoted from private to exported for testability. test/handlers.test.ts: 4 tests. Asserts every expected handler name registers. Asserts autopilot-cycle against a nonexistent repo returns { partial: true, failed_steps: ['sync', 'extract', 'backlinks'] } — does NOT throw. Asserts autopilot-cycle against an empty (but real) git repo returns a result with a steps map, never throws. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(autopilot): Minions dispatch + worker spawn supervisor + async shutdown Autopilot now dispatches each cycle as a single `autopilot-cycle` Minion job (with idempotency_key on the cycle slot) instead of running steps inline. A forked `gbrain jobs work` child drains the queue durably, supervised by autopilot. The user runs ONE install step (`gbrain autopilot --install`) and gets sync + extract + embed + backlinks + durable job processing, with no separate worker daemon to manage. Mode selection: - minion_mode=always OR pain_triggered (default), engine=postgres → Minions dispatch. Spawn child, submit autopilot-cycle each interval. - minion_mode=off, OR engine=pglite, OR `--inline` flag → run steps inline in-process, same as pre-v0.11.1. PGLite has an exclusive file lock that blocks a second worker process, so the inline path is the only path that works there. Worker supervision: - spawn(resolveGbrainCliPath(), ['jobs', 'work'], { stdio: 'inherit' }). stdio:'inherit' avoids pipe-buffer blocking (Codex architecture #2). - On worker exit: 10s backoff + restart. Crash counter caps at 5 → autopilot stops with a clear error. - resolveGbrainCliPath() prefers argv[1] (cli.ts / /gbrain), then process.execPath (compiled binary suffix check), then `which gbrain` (installed to $PATH). NEVER blindly uses process.execPath, which on source installs is the Bun runtime, not `gbrain` (Codex architecture #1). Shutdown: - Async SIGTERM/SIGINT handler: sends SIGTERM to worker, awaits its exit for up to 35s (the worker's own drain is 30s; we add buffer for signal-delivery latency), then SIGKILL if still alive. - Drops the old `process.on('exit')` lock-cleanup handler — its callback runs synchronously and can't wait for the worker drain. Lock file cleanup moved inside the async shutdown. Lock-file mtime refresh every cycle (Codex C) so a long-lived autopilot doesn't get declared "stale" by the next cron-fired invocation after 10 minutes. Inline fallback path calls the new Core fns (runExtractCore, runEmbedCore) instead of the CLI wrappers. That way a bad arg from inside the loop can't process.exit() the autopilot itself (matches Codex #5). test/autopilot-resolve-cli.test.ts: 3 tests covering argv[1]-as-gbrain, argv[1]-as-cli.ts, and graceful error when no path resolves. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(autopilot): env-aware install + OpenClaw bootstrap injection Expand installDaemon from 2 targets (macOS launchd, Linux crontab) to 4: - macos → launchd plist (unchanged) - linux-systemd → ~/.config/systemd/user/gbrain-autopilot.service with Restart=on-failure, RestartSec=30, and an is-system-running probe to confirm the user bus actually works (Codex architecture #7 hardened — the naive /run/systemd/system existence check was a false-positive magnet) - ephemeral-container → detects RENDER / RAILWAY_ENVIRONMENT / FLY_APP_NAME / /.dockerenv. Crontab is unreliable here (wiped on deploy), so we write ~/.gbrain/start-autopilot.sh and tell the user to source it from their agent's bootstrap - linux-cron → existing crontab path (unchanged) detectInstallTarget() + --target flag for explicit override. Also: - --inject-bootstrap / --no-inject control OpenClaw ensure-services.sh auto-injection. Default is ON when OpenClaw is detected (OPENCLAW_HOME env var, openclaw.json in CWD or $HOME, or an ensure-services.sh found). Injection adds ONE line with a `# gbrain:autopilot v0.11.0` marker and writes .bak.<ISO-timestamp> before touching the file. Idempotent — the marker check prevents double injection. uninstallDaemon mirrors all four targets. A user can now run `gbrain autopilot --uninstall` after moving hosts (macOS laptop → Linux server) and the uninstall will find + remove every artifact. writeWrapperScript now uses resolveGbrainCliPath() instead of blindly baking process.execPath into the wrapper script — on source installs that path is the Bun runtime, not gbrain (Codex architecture #1 fix propagated to the install path too). test/autopilot-install.test.ts: 4 tests covering detectInstallTarget's platform + env-var branches. Deeper E2E coverage (systemd unit file contents, ephemeral start-script contents + exec bit, OpenClaw marker injection + .bak) lives in Task 14's E2E fixture test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(migrations): v0.11.0 orchestrator — phases A through G, full implementation Replaces the stub from commit de027ce. The orchestrator runs all seven phases of the v0.11.0 Minions adoption migration idempotently, resumable from any prior status:"partial" run (the stopgap bash script writes those). Phases: A. Schema — `gbrain init --migrate-only` (NEVER bare `gbrain init`, which defaults to PGLite and clobbers existing configs — Codex H1 show-stopper). B. Smoke — `gbrain jobs smoke`. Abort loudly on non-zero. C. Mode — --mode flag wins. Preserved from prefs on resume. Non-TTY or --yes defaults pain_triggered with explicit print. Interactive: numbered 1/2/3 menu via shared promptLine. D. Prefs — savePreferences({minion_mode, set_at, set_in_version}). E. Host — AGENTS.md marker injection + cron manifest rewrites. For cron entries whose skill matches a gbrain builtin (sync/embed/lint/import/extract/backlinks/autopilot-cycle) rewrites kind:agentTurn → kind:shell with a gbrain jobs submit command. PGLite branch keeps --follow (inline execution, the only path that works without a worker daemon); Postgres branch drops --follow + adds --idempotency-key ${handler}:${slot} so long cron jobs don't stack up (same Codex fix as the autopilot-cycle dispatch). For non-builtin handlers (host-specific, like ea-inbox-sweep, frameio-scan, x-dm-triage) emits a structured TODO row to ~/.gbrain/migrations/pending-host-work.jsonl so the host agent can walk through plugin-contract work per skills/migrations/v0.11.0.md. F. Install — `gbrain autopilot --install --yes`. Best-effort (failure doesn't abort; user can run manually). G. Record — append to completed.jsonl. status:"complete" unless pending_host_work > 0, in which case status:"partial" + apply_migrations_pending: true. Safety guards (Codex code-quality tension #3: strict-skip, no rollback): - Scope: $HOME/.claude + $HOME/.openclaw only by default. --host-dir must be explicit to include $PWD or any other path. - Symlink escape: SKIP if the resolved target leaves the scoped root. - >1 MB files: SKIP with warning. - Permission denied: SKIP with warning; other files continue. - Malformed JSON manifest: SKIP with parse error logged; continue. - mtime re-check right before write: bail the file if changed between read + write; other files continue. - Every edit writes a .bak.<ISO-timestamp> sibling first (second- precision so two same-day runs don't collide). - Idempotency: `_gbrain_migrated_by: "v0.11.0"` JSON property marker on each rewritten cron entry (JSON can't have comments — Codex G); AGENTS.md marker `<!-- gbrain:subagent-routing v0.11.0 -->`. - TODO dedupe: JSONL appends deduped by (handler, manifest_path) so reruns don't grow the file. Post-run summary: when pending_host_work > 0, prints a one-liner pointing the user at the JSONL path + the v0.11.0 skill file. The skill (Lane C-3 / C-4) is the host-agent instruction manual. test/migrations-v0_11_0.test.ts: 18 tests covering: - AGENTS.md injection: happy path, .bak creation, idempotent rerun, --dry-run no-op, symlink-escape SKIP, >1MB SKIP. - Cron rewrite: builtin handlers rewrite to shell+gbrain jobs submit, non-builtins emit JSONL TODOs without touching the manifest, mixed manifests get both treatments in one pass, idempotent rerun, TODO dedupe, malformed JSON SKIP, no-entries-array SKIP, --dry-run no-op. - findAgentsMdFiles + findCronManifests: scoped walk to $HOME/.claude + $HOME/.openclaw, --host-dir opt-in for $PWD. - BUILTIN_HANDLERS frozen at the canonical 7 names. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(skill): port skillify from Wintermute, pair with check-resolvable Skillify is the "meta skill": turn any raw feature or script into a properly-skilled, tested, resolvable, evaled unit of agent-visible capability. Proven in production on Wintermute; paired with gbrain's existing `check-resolvable` it becomes a user-controllable equivalent of Hermes' auto-skill-creation — you decide when and what, the tooling keeps the checklist honest. Shipped: - skills/skillify/SKILL.md — ported from ~/git/wintermute/workspace/ skills/skillify/SKILL.md. Genericized: * /data/.openclaw/workspace → \${PROJECT_ROOT} (runtime-detected). * services/voice-agent/__tests__/ → test/ (detected from repo). * Manual `grep skills/... AGENTS.md` replaced with a reference to `gbrain check-resolvable`, which does reachability + MECE + DRY + gap detection properly instead of grep-matching a path string. - scripts/skillify-check.ts — ported from ~/git/wintermute/workspace/scripts/skillify-check.mjs. Preserves the --recent flag and --json output shape. Detects project root via package.json walkup; detects test dir (test/ → __tests__/ → tests/ → spec/). Runs the 10-item checklist per target and exits non-zero if any required item is missing. - test/skillify-check.test.ts — 4 CLI tests: happy-path against publish.ts (known-skilled), --json shape + schema, --recent smoke, bogus-target exit code. - skills/RESOLVER.md — adds the trigger row ("Skillify this", "is this a skill?", "make this proper") → skills/skillify/SKILL.md. - skills/manifest.json — adds the skillify entry so the conformance test passes. Why the pair: * Hermes auto-creates skills in the background. Fine until you don't know what the agent shipped — checklists decay silently. * gbrain ships the same capability as two user-controlled tools: /skillify builds the checklist, gbrain check-resolvable validates reachability + MECE + DRY across the whole skill tree. * Human keeps judgment. Tooling keeps the checklist honest. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(v0.11.1): cron-via-minions convention, plugin-handlers guide, minions-fix, skill updates New reference docs: - skills/conventions/cron-via-minions.md — the rewrite convention for cron manifests. Shows the Postgres (fire-and-forget + idempotency- key) vs PGLite (--follow inline) branch; explains why builtin-only auto-rewrite is safe + how host-specific handlers get the plugin contract. - docs/guides/plugin-handlers.md — the plugin contract for host- specific Minion handlers. Code-level registration via import + worker.register(), not a data file (Codex D: handlers.json was an RCE surface). Concrete TypeScript skeleton + handler contract (ctx.data, ctx.signal, ctx.inbox) + full migration flow from TODO JSONL to a rewritten cron entry. - docs/guides/minions-fix.md — user-facing troubleshooting for half-migrated v0.11.0 installs. Paste-one-liner for the stopgap, gbrain apply-migrations path for v0.11.1+, verification commands, failure-mode recipes. Rewrites + updates: - skills/migrations/v0.11.0.md — body restored as the host-agent instruction manual. Audience is the host agent reading ~/.gbrain/migrations/pending-host-work.jsonl after the CLI orchestrator has done the mechanical phases. Walks each TODO type through the 10-item skillify checklist (plugin contract, ship bootstrap, unit tests, integration tests, LLM evals, resolver trigger, trigger eval, E2E smoke, brain filing, check-resolvable). Reverses the earlier "delete the body" decision (1B) because the body serves a different audience now — host-agent, not CLI documentation. - skills/cron-scheduler/SKILL.md — Phase 4 ("Register with host scheduler") now references cron-via-minions + plugin-handlers. - skills/maintain/SKILL.md — new "Fix a half-migrated install" section with the apply-migrations recipe. - skills/setup/SKILL.md — new Phase C.5 "One-step autopilot + Minions install (v0.11.1+)" explaining the four install targets + the OpenClaw auto-injection default. - docs/GBRAIN_SKILLPACK.md — Operations section adds the three new guides + the subagent-routing and cron-routing SKILLPACK notes (v0.11.0+). All 167 related tests (conformance + resolver + skillify-check + v0_11_0 orchestrator) stay green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(v0.11.1): stopgap script + CLAUDE.md directive + README + CHANGELOG + version bump scripts/fix-v0.11.0.sh — the paste-command for broken-v0.11.0 installs. Released on the v0.11.1 tag so: curl -fsSL https://raw.githubusercontent.com/garrytan/gbrain/v0.11.1/scripts/fix-v0.11.0.sh | bash always works (master branch could be renamed). 8 steps: schema apply, smoke, mode prompt (non-TTY defaults pain_triggered), atomic write of preferences.json (0o600), append completed.jsonl with status:"partial" and apply_migrations_pending:true so the v0.11.1 apply-migrations run resumes correctly (does NOT poison the permanent migration path — Codex H2 avoidance), AGENTS.md + cron/jobs.json detection with guidance printed as text only (never auto-edits from a curl-piped script), and a closing line telling the user to run `gbrain autopilot --install` as the one-stop finisher. CLAUDE.md — new "Migration is canonical, not advisory" section pinning the design principle. Any host-repo change (AGENTS.md, cron manifests, launchctl units) is GBrain's responsibility via the migration; the exception is host-specific handler registration, which goes via the code-level plugin contract in docs/guides/plugin-handlers.md. README.md — new sections: - "v0.11.0 migration didn't fire on your upgrade?" with both repair paths (v0.11.1 binary and pre-v0.11.1 stopgap). - "Skillify + check-resolvable: user-controllable auto-skill-creation" explaining why the user-controlled pair beats Hermes-style auto generation. Includes the scripts/skillify-check.ts invocation. CHANGELOG.md — v0.11.1 entry (per CLAUDE.md voice: lead with what the user can now do that they couldn't before; frame as benefits, not files changed). Covers: mega-bug fix + apply-migrations + postinstall + stopgap, autopilot-supervises-worker + single-install-step + env-aware targets, Core fn extraction so handlers don't kill workers, skillify + check-resolvable pair, host-agnostic plugin contract replacing handlers.json (RCE concern), gbrain init --migrate-only, TS migration registry + H8/H9 diff-rule fixes, CLAUDE.md directive. All Codex hard blockers (H1, H3/H4, H5, H6, H7, H8, H9, K) + architecture issues (#1/#2/#4/#5/#7) resolved. package.json — version bump 0.11.0 → 0.11.1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(e2e): migration-flow E2E against live Postgres + Bun env quirk fix Ships test/e2e/migration-flow.test.ts — the end-to-end integration test for the v0.11.0 orchestrator. Spins up against a live Postgres (gated on DATABASE_URL per CLAUDE.md lifecycle) and exercises four scenarios: - Fresh install: schema apply (Phase A via `gbrain init --migrate-only`) → smoke (Phase B) → mode resolution (C) → prefs (D) → host rewrite (E, empty fixture) → record (G). Asserts preferences.json exists with 0o600, completed.jsonl has a v0.11.0 entry, autopilot install was skipped per --no-autopilot-install. - Idempotent rerun: second orchestrator invocation on a completed install doesn't blow up; mode stays stable. - Host rewrite mixed manifest: 4-entry cron/jobs.json with 2 gbrain- builtin handlers (sync, embed) + 2 non-builtin (ea-inbox-sweep, morning-briefing). Asserts builtins rewrite to `gbrain jobs submit` kind:shell, non-builtins are LEFT on kind:agentTurn, and 2 JSONL TODOs are emitted with correct shape. AGENTS.md gets the marker injected. Status is "partial" because pending-host-work > 0. - Resumable: stopgap writes a partial completed.jsonl row first; orchestrator re-runs successfully against it and appends a new post-orchestrator entry. 1 partial + 1 complete = 2 rows total. Critical fix surfaced by the E2E: src/commands/migrations/v0_11_0.ts's three execSync calls (gbrain init --migrate-only, gbrain jobs smoke, gbrain autopilot --install) now explicitly pass `env: process.env`. Bun's execSync default does NOT propagate post-start `process.env.PATH` mutations to subprocesses — only the initial PATH snapshot. Without the explicit env, any user-side env tweak (e.g. setting GBRAIN_DATABASE_URL in a script before calling the orchestrator) would be invisible to the orchestrator's subprocesses. This is also the reason the E2E needs a PATH shim installed at module-load time to expose the `gbrain` command. test/init-migrate-only.test.ts: subprocess env now strips DATABASE_URL and GBRAIN_DATABASE_URL. The "no config" error-path tests need loadConfig() to return null, which it won't if the env-var fallback at src/core/config.ts:30 fires. Before this fix, running the unit tests with DATABASE_URL set (e.g. during an E2E run) caused false failures because `gbrain init --migrate-only` saw the env var and succeeded. Full test totals with live Postgres: 1265 pass, 0 fail, 3497 expect calls, 67 files, ~95s. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump VERSION file to 0.11.1 Commit 5c4cf1d bumped package.json version to 0.11.1 but missed the root VERSION file. src/version.ts reads from package.json so `gbrain --version` prints 0.11.1 correctly, but any tool or script that reads the VERSION file directly (like /ship's idempotency check) saw the stale 0.11.0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(v0.11.1): doctor self-heal check + skillpack-check command for cron health reports Closes the discoverability hole from the v0.11.0 mega-bug: once a user is on v0.11.1 (or later), every `gbrain doctor` invocation immediately surfaces a half-migrated state, and `gbrain skillpack-check` gives host agents (Wintermute's morning-briefing, any OpenClaw cron) a single exit-coded JSON pipe to check from their own skills. gbrain doctor — two new checks: 1. Filesystem-only (fires on every `doctor` invocation, even --fast): if `~/.gbrain/migrations/completed.jsonl` has any status:"partial" entry with no matching status:"complete" for the same version, print `MINIONS HALF-INSTALLED (partial migration: vX.Y.Z). Run: gbrain apply-migrations --yes`. Typical cause is the stopgap wrote a partial record but nobody ran `apply-migrations` afterward. 2. DB-path: if schema version is v7+ (Minions present) AND `~/.gbrain/preferences.json` is missing, print the same banner. Catches installs that never ran the stopgap or apply-migrations at all — the classic v0.11.0 "upgrade landed, migration never fired" state. Both checks status:"fail" so doctor exits non-zero when either fires. Test `test/doctor-minions-check.test.ts` pins the five branches (partial present → FAIL, partial+complete → quiet, no-jsonl → quiet, multiple versions named correctly, human-readable banner contains the exact "MINIONS HALF-INSTALLED" phrase Wintermute's cron can grep for). gbrain skillpack-check — new command + skill: - `src/commands/skillpack-check.ts` wraps `doctor --fast --json` + `apply-migrations --list` into one JSON report with `{healthy, summary, actions[], doctor, migrations}`. Exit 0 on healthy, 1 on action-needed, 2 on determine-failure. `--quiet` flag for cron pipes that want exit-code-only behavior. - `actions[]` is the remediation list. Doctor messages of the form `... Run: <cmd>` get their command extracted (regex fixed to match the full remainder of the line, not just the first word). Pending or partial migrations push `gbrain apply-migrations --yes` to the front of actions[]. - `gbrainSpawn()` helper resolves the gbrain invocation correctly on compiled binary installs (`argv[1] = /usr/local/bin/gbrain`) AND source installs (`argv[1] = src/cli.ts`, prefix with `bun run`). Same Codex #1 fix pattern as autopilot's resolveGbrainCliPath. - `skills/skillpack-check/SKILL.md` teaches agents when to run it, what to do with the output, and anti-patterns (don't run without --quiet in a cron that emails; don't ignore exit 2). - Registered in skills/RESOLVER.md and skills/manifest.json. Test `test/skillpack-check.test.ts` (5 tests) covers healthy fresh install, half-migrated exit-1 with apply-migrations in actions[], --quiet suppresses stdout in both states, --help prints usage, summary includes top action when multiple are present. 1192 unit tests pass (+15 new). The 38 failing tests are all DATABASE_URL E2Es — same pre-existing pattern, unchanged by this commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * doc(v0.11.1): reframe README + minions-fix — v0.11.0 was never released v0.11.0 was cut but never released publicly. v0.11.1 is the first public Minions ship, and fixes the upgrade-migration mega-bug so it self-heals on every future `gbrain upgrade` + `bun update gbrain`. The README was wrongly framing the fix as a retrospective for v0.11.0 users — none exist, so remove it. README changes: - Delete the "v0.11.0 migration didn't fire on your upgrade?" section. Replace with "Health check and self-heal": the `gbrain doctor`, `gbrain skillpack-check --quiet`, and `gbrain skillpack-check | jq` recipes that ship in v0.11.1. Still links to docs/guides/minions-fix.md for deeper troubleshooting. - Promote the production benchmark to top billing. The previous section led with the lab benchmark (same LLM, localhost) and buried the production data point as a single follow-up sentence. Real deployment numbers are the stronger signal: * 753ms vs >10s gateway timeout (sub-agent couldn't even spawn) * $0.00 vs ~$0.03 per run * 100% vs 0% success rate under 19-cron production load * 36-month tweet backfill: 19,240 tweets, ~15 min, $0.00 Lab numbers stay (separate table, labeled "controlled environment") so readers can see both layers. - Add the "The routing rule" closer: Deterministic → Minions, Judgment → Sub-agents. This is the clearest framing in the production benchmark doc and belongs in the README so readers leave with the right mental model. `minion_mode: pain_triggered` automates it. docs/guides/minions-fix.md rewrite: - Reframe as: v0.11.0 never released, v0.11.1 is the first ship, `gbrain apply-migrations --yes` is canonical. Stopgap stays documented for pre-v0.11.1 branch builds (e.g. Wintermute's minions-jobs checkout before v0.11.1 tags). - Add the detection + verification commands (doctor + skillpack-check) at the top. - Cross-reference skills/skillpack-check/SKILL.md as the agent-facing health-check pattern. Zero lingering "v0.11.0 released" references in README or minions-fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(doctor): remove "schema v7+ no prefs → FAIL" check (too aggressive) CI failure in Tier 1 Mechanical E2E: (fail) E2E: Doctor Command > gbrain doctor exits 0 on healthy DB Root cause: the doctor half-migration detection added two checks. The second check (`schema v7+ AND ~/.gbrain/preferences.json missing → minions_config FAIL`) was too aggressive. It treated a valid fresh- install state as broken. `gbrain init` against Postgres applies schema v7 but doesn't write preferences.json — that's the migration orchestrator's Phase D, which only runs via `apply-migrations`. Between `init` finishing and the user running `apply-migrations`, the install is legitimately in a "schema-applied, no prefs" state. Doctor was exiting 1 on this valid state, breaking the pre-existing CI test that init's + docters a healthy DB. Fix: drop the check. The filesystem check (step 3 — partial-completed without a matching complete) is sufficient signal for genuine half- migration. Added a regression test pinning the exact CI scenario: no completed.jsonl present, no preferences.json, doctor must not fail any minions_* check. Also removes the now-unused `preferencesPaths` import. Verified against live Postgres: CI-equivalent `gbrain doctor` + `gbrain doctor --json` both pass. Full suite: 1281/1281 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * doc(readme): Minions section — lead with the story, compress the rest The previous section opened with "six daily pains" as a numbered list before the hook, buried the production numbers halfway down, and had a table explaining how each pain gets fixed. Fine for a spec doc; wrong for a README that needs to land the impact fast. Rewrite: - Lead with "your sub-agents won't drop work anymore" — the reason a reader is here. - Production numbers promoted, framed as a story: "Here's my personal OpenClaw deployment: one Render container, Supabase Postgres holding a 45,000-page brain, 19 cron jobs firing on schedule, the X Enterprise API on the wire..." Gives the reader the setup before the punchline. - The routing rule (deterministic → Minions, judgment → sub-agents) survives unchanged. It's the clearest framing in the whole section. - Lose the "how each pain gets fixed" table. Compress the six pains + their fixes into one paragraph that names the primitives by name (max_children, timeout_ms, child_done inbox, cascade cancel, idempotency keys, attachment validation). Readers who want depth click through to skills/minion-orchestrator/SKILL.md. - Close with "not incrementally better — categorically different" and the three headline numbers. - Drop the separate Lab Numbers table; the production numbers are stronger and the lab data is one click away via the link. Lines: 75 → 42. Same signal, less scroll. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * doc: scrub X Enterprise API + @garrytan references from user-facing docs User feedback: shouldn't name the specific enterprise-tier API product or the account in the README or benchmark docs. Genericize: - "X Enterprise API on the wire" → drop entirely; the 19-cron load story carries the setup without naming the vendor - "X Enterprise API ($50K/mo firehose)" → "external API" - "@garrytan tweets" → "my social posts" - "Pull ~100 @garrytan tweets" → "Pull ~100 of my social posts" - "X Enterprise API (full-archive)" env var comment → "external API bearer token" Scope: - README.md — the Minions production story line + scaling callout - docs/benchmarks/2026-04-18-minions-vs-openclaw-production.md - docs/benchmarks/2026-04-18-tweet-ingestion.md Plain "X API" references in the tweet-ingestion methodology stay — those describe which public HTTP endpoint was called, not the enterprise-tier product. Benchmark doc filenames (tweet-ingestion.md) stay to preserve inbound links; content is genericized. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * doc(readme): Skillify section — match Minions energy, land the category shift The previous section was competent but undersold what skillify actually is. Rewrite matches the Minions section's shape: lead with the hook, tell the story, land the punchline. Key changes: - Title: "your skills tree stops being a black box." Names the thing skillify actually solves. - Open with the problem: Hermes auto-creates skills as a background behavior. Six months later you have an opaque pile nobody's read or tested. Make the liability concrete. - Promote the 10 items by name (SKILL.md + script + unit tests + integration tests + LLM evals + resolver trigger + trigger eval + E2E + brain filing + check-resolvable audit). Showing the list makes the scope of the unlock visible. - New subsection "Why this is the right answer for OpenClaw" names the debugging-the-black-box pain directly. Skillify makes the tree legible: when something breaks, you know which layer (contract, test, eval, trigger, or route) to inspect. When anything goes stale, check-resolvable flags it. - Close with "compounding quality instead of compounding entropy" + "not a nice-to-have. It's the piece that makes the skills tree survive six months." - Expand the code block to include `gbrain check-resolvable` (the other half of the pair) so readers see the whole workflow. Length goes from 17 to 34 lines — still shorter than Minions, still one section. Worth the space because this is a category shift for how agent skills get built, not a feature. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: root <root@localhost>
This commit is contained in:
Garry Tan
35
src/cli.ts
35
src/cli.ts
@@ -18,7 +18,7 @@ for (const op of operations) {
|
||||
}
|
||||
|
||||
// CLI-only commands that bypass the operation layer
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot']);
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'jobs', 'apply-migrations', 'skillpack-check']);
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
@@ -248,7 +248,7 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
}
|
||||
if (command === 'post-upgrade') {
|
||||
const { runPostUpgrade } = await import('./commands/upgrade.ts');
|
||||
runPostUpgrade();
|
||||
await runPostUpgrade();
|
||||
return;
|
||||
}
|
||||
if (command === 'check-update') {
|
||||
@@ -281,6 +281,22 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
await runReport(args);
|
||||
return;
|
||||
}
|
||||
if (command === 'apply-migrations') {
|
||||
// Does not need connectEngine — each phase (schema, smoke, host-rewrite)
|
||||
// manages its own subprocess or file-layer access directly. Avoids
|
||||
// connecting a second time when the orchestrator shells out to
|
||||
// `gbrain init --migrate-only` and `gbrain jobs smoke`.
|
||||
const { runApplyMigrations } = await import('./commands/apply-migrations.ts');
|
||||
await runApplyMigrations(args);
|
||||
return;
|
||||
}
|
||||
if (command === 'skillpack-check') {
|
||||
// Agent-readable health report. Shells out to doctor + apply-migrations
|
||||
// internally; does not need its own DB connection.
|
||||
const { runSkillpackCheck } = await import('./commands/skillpack-check.ts');
|
||||
await runSkillpackCheck(args);
|
||||
return;
|
||||
}
|
||||
if (command === 'doctor') {
|
||||
// Doctor runs filesystem checks first (no DB needed), then DB checks.
|
||||
// --fast skips DB checks entirely.
|
||||
@@ -350,6 +366,11 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
await runEvalCommand(engine, args);
|
||||
break;
|
||||
}
|
||||
case 'jobs': {
|
||||
const { runJobs } = await import('./commands/jobs.ts');
|
||||
await runJobs(engine, args);
|
||||
break;
|
||||
}
|
||||
case 'sync': {
|
||||
const { runSync } = await import('./commands/sync.ts');
|
||||
await runSync(engine, args);
|
||||
@@ -474,6 +495,16 @@ TOOLS
|
||||
lint <dir|file> [--fix] Catch LLM artifacts, placeholder dates, bad frontmatter
|
||||
report --type <name> --content ... Save timestamped report to brain/reports/
|
||||
|
||||
JOBS (Minions)
|
||||
jobs submit <name> [--params JSON] Submit background job [--follow] [--dry-run]
|
||||
jobs list [--status S] [--limit N] List jobs
|
||||
jobs get <id> Job details + history
|
||||
jobs cancel <id> Cancel job
|
||||
jobs retry <id> Re-queue failed/dead job
|
||||
jobs prune [--older-than 30d] Clean old jobs
|
||||
jobs stats Job health dashboard
|
||||
jobs work [--queue Q] Start worker daemon (Postgres only)
|
||||
|
||||
ADMIN
|
||||
stats Brain statistics
|
||||
health Brain health dashboard
|
||||
|
||||
283
src/commands/apply-migrations.ts
Normal file
283
src/commands/apply-migrations.ts
Normal file
@@ -0,0 +1,283 @@
|
||||
/**
|
||||
* `gbrain apply-migrations` — migration runner CLI.
|
||||
*
|
||||
* Reads ~/.gbrain/migrations/completed.jsonl, diffs against the TS migration
|
||||
* registry, runs any pending orchestrators. Resumes `status: "partial"`
|
||||
* entries (stopgap bash script writes these). Idempotent: rerunning is
|
||||
* cheap when nothing is pending.
|
||||
*
|
||||
* Invoked from:
|
||||
* - `gbrain upgrade` → runPostUpgrade() tail (Lane A-5)
|
||||
* - package.json `postinstall` (Lane A-5)
|
||||
* - explicit user / host-agent after registering new handlers (Lane C-1)
|
||||
*/
|
||||
|
||||
import { VERSION } from '../version.ts';
|
||||
import { loadConfig } from '../core/config.ts';
|
||||
import { loadCompletedMigrations, type CompletedMigrationEntry } from '../core/preferences.ts';
|
||||
import { migrations, compareVersions, type Migration, type OrchestratorOpts } from './migrations/index.ts';
|
||||
|
||||
interface ApplyMigrationsArgs {
|
||||
list: boolean;
|
||||
dryRun: boolean;
|
||||
yes: boolean;
|
||||
nonInteractive: boolean;
|
||||
mode?: 'always' | 'pain_triggered' | 'off';
|
||||
specificMigration?: string;
|
||||
hostDir?: string;
|
||||
noAutopilotInstall: boolean;
|
||||
help: boolean;
|
||||
}
|
||||
|
||||
function parseArgs(args: string[]): ApplyMigrationsArgs {
|
||||
const has = (flag: string) => args.includes(flag);
|
||||
const val = (flag: string): string | undefined => {
|
||||
const i = args.indexOf(flag);
|
||||
return i >= 0 && i + 1 < args.length ? args[i + 1] : undefined;
|
||||
};
|
||||
const mode = val('--mode') as ApplyMigrationsArgs['mode'];
|
||||
if (mode && !['always', 'pain_triggered', 'off'].includes(mode)) {
|
||||
console.error(`Invalid --mode "${mode}". Allowed: always, pain_triggered, off.`);
|
||||
process.exit(2);
|
||||
}
|
||||
return {
|
||||
list: has('--list'),
|
||||
dryRun: has('--dry-run'),
|
||||
yes: has('--yes'),
|
||||
nonInteractive: has('--non-interactive'),
|
||||
mode,
|
||||
specificMigration: val('--migration'),
|
||||
hostDir: val('--host-dir'),
|
||||
noAutopilotInstall: has('--no-autopilot-install'),
|
||||
help: has('--help') || has('-h'),
|
||||
};
|
||||
}
|
||||
|
||||
function printHelp(): void {
|
||||
console.log(`gbrain apply-migrations — run pending migration orchestrators.
|
||||
|
||||
Usage:
|
||||
gbrain apply-migrations Run all pending migrations interactively.
|
||||
gbrain apply-migrations --yes Non-interactive; uses default mode (pain_triggered).
|
||||
gbrain apply-migrations --dry-run Print the plan; take no action.
|
||||
gbrain apply-migrations --list Show applied + pending migrations.
|
||||
gbrain apply-migrations --migration vX.Y.Z
|
||||
Force-run a specific migration by version.
|
||||
|
||||
Flags:
|
||||
--mode <always|pain_triggered|off> Set minion_mode without prompting.
|
||||
--host-dir <path> Include this directory in host-file walk
|
||||
(default scope: \$HOME/.claude + \$HOME/.openclaw).
|
||||
--no-autopilot-install Skip the Phase F autopilot install step.
|
||||
--non-interactive Equivalent to --yes; never prompt.
|
||||
|
||||
Exit codes:
|
||||
0 Success (including "nothing to do").
|
||||
1 An orchestrator failed.
|
||||
2 Invalid arguments.
|
||||
`);
|
||||
}
|
||||
|
||||
interface CompletedIndex {
|
||||
byVersion: Map<string, CompletedMigrationEntry[]>;
|
||||
}
|
||||
|
||||
function indexCompleted(entries: CompletedMigrationEntry[]): CompletedIndex {
|
||||
const byVersion = new Map<string, CompletedMigrationEntry[]>();
|
||||
for (const e of entries) {
|
||||
const list = byVersion.get(e.version) ?? [];
|
||||
list.push(e);
|
||||
byVersion.set(e.version, list);
|
||||
}
|
||||
return byVersion.size > 0
|
||||
? { byVersion }
|
||||
: { byVersion: new Map() };
|
||||
}
|
||||
|
||||
/** Returns the resolved status for a migration based on its entries. */
|
||||
function statusForVersion(
|
||||
version: string,
|
||||
idx: CompletedIndex,
|
||||
): 'complete' | 'partial' | 'pending' {
|
||||
const entries = idx.byVersion.get(version) ?? [];
|
||||
if (entries.length === 0) return 'pending';
|
||||
if (entries.some(e => e.status === 'complete')) return 'complete';
|
||||
if (entries.some(e => e.status === 'partial')) return 'partial';
|
||||
return 'pending';
|
||||
}
|
||||
|
||||
interface Plan {
|
||||
applied: Migration[];
|
||||
partial: Migration[];
|
||||
pending: Migration[];
|
||||
skippedFuture: Migration[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the run plan.
|
||||
*
|
||||
* - applied: has a `status: "complete"` entry for its version.
|
||||
* - partial: has only `status: "partial"` entries (stopgap wrote one) →
|
||||
* orchestrator runs to finish missing phases.
|
||||
* - pending: has no entries at all and migration.version ≤ installed VERSION.
|
||||
* - skippedFuture: migration.version > installed VERSION (binary is older
|
||||
* than the migration; wait for a newer install).
|
||||
*
|
||||
* Codex H9: we never compare against `current VERSION >` — that rule would
|
||||
* skip v0.11.0 when running v0.11.1. Compare against completed.jsonl.
|
||||
*/
|
||||
function buildPlan(idx: CompletedIndex, installed: string, filterVersion?: string): Plan {
|
||||
const plan: Plan = { applied: [], partial: [], pending: [], skippedFuture: [] };
|
||||
for (const m of migrations) {
|
||||
if (filterVersion && m.version !== filterVersion) continue;
|
||||
if (compareVersions(m.version, installed) > 0) {
|
||||
plan.skippedFuture.push(m);
|
||||
continue;
|
||||
}
|
||||
const status = statusForVersion(m.version, idx);
|
||||
if (status === 'complete') plan.applied.push(m);
|
||||
else if (status === 'partial') plan.partial.push(m);
|
||||
else plan.pending.push(m);
|
||||
}
|
||||
return plan;
|
||||
}
|
||||
|
||||
function printList(plan: Plan, installed: string): void {
|
||||
console.log(`Installed gbrain version: ${installed}\n`);
|
||||
console.log(' Status Version Headline');
|
||||
console.log(' ------- -------- -----------------------------------------');
|
||||
const rows: Array<{ status: string; m: Migration }> = [
|
||||
...plan.applied.map(m => ({ status: 'applied', m })),
|
||||
...plan.partial.map(m => ({ status: 'partial', m })),
|
||||
...plan.pending.map(m => ({ status: 'pending', m })),
|
||||
...plan.skippedFuture.map(m => ({ status: 'future', m })),
|
||||
];
|
||||
for (const r of rows) {
|
||||
const ver = r.m.version.padEnd(8);
|
||||
const status = r.status.padEnd(7);
|
||||
console.log(` ${status} ${ver} ${r.m.featurePitch.headline}`);
|
||||
}
|
||||
if (rows.length === 0) console.log(' (no migrations registered)');
|
||||
console.log('');
|
||||
const needsWork = plan.pending.length + plan.partial.length;
|
||||
if (needsWork === 0) {
|
||||
console.log('All migrations up to date.');
|
||||
} else {
|
||||
console.log(`${needsWork} migration(s) need action. Run \`gbrain apply-migrations --yes\` to apply.`);
|
||||
}
|
||||
}
|
||||
|
||||
function printDryRun(plan: Plan, installed: string): void {
|
||||
console.log(`Dry run — installed gbrain version: ${installed}`);
|
||||
console.log('');
|
||||
if (plan.applied.length) {
|
||||
console.log('Already applied:');
|
||||
for (const m of plan.applied) console.log(` ✓ v${m.version} — ${m.featurePitch.headline}`);
|
||||
console.log('');
|
||||
}
|
||||
if (plan.partial.length) {
|
||||
console.log('Would RESUME (previously partial):');
|
||||
for (const m of plan.partial) console.log(` ⟳ v${m.version} — ${m.featurePitch.headline}`);
|
||||
console.log('');
|
||||
}
|
||||
if (plan.pending.length) {
|
||||
console.log('Would APPLY:');
|
||||
for (const m of plan.pending) console.log(` → v${m.version} — ${m.featurePitch.headline}`);
|
||||
console.log('');
|
||||
}
|
||||
if (plan.skippedFuture.length) {
|
||||
console.log('Skipped (newer than installed binary):');
|
||||
for (const m of plan.skippedFuture) console.log(` ⧗ v${m.version}`);
|
||||
console.log('');
|
||||
}
|
||||
if (plan.pending.length + plan.partial.length === 0) {
|
||||
console.log('Nothing to do.');
|
||||
} else {
|
||||
console.log('Re-run without --dry-run to apply. Use --yes to skip prompts.');
|
||||
}
|
||||
}
|
||||
|
||||
function orchestratorOptsFrom(cli: ApplyMigrationsArgs): OrchestratorOpts {
|
||||
return {
|
||||
yes: cli.yes || cli.nonInteractive,
|
||||
mode: cli.mode,
|
||||
dryRun: cli.dryRun,
|
||||
hostDir: cli.hostDir,
|
||||
noAutopilotInstall: cli.noAutopilotInstall,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Entry point. Does not call connectEngine — each phase inside an
|
||||
* orchestrator manages its own engine / subprocess lifecycle.
|
||||
*/
|
||||
export async function runApplyMigrations(args: string[]): Promise<void> {
|
||||
const cli = parseArgs(args);
|
||||
if (cli.help) { printHelp(); return; }
|
||||
|
||||
const installed = VERSION.replace(/^v/, '').trim() || '0.0.0';
|
||||
|
||||
// First-install guard (postinstall hook calls us even on `bun add gbrain`
|
||||
// before the user has run `gbrain init`). No config = no brain = nothing
|
||||
// to migrate. Exit silently for --yes / --non-interactive so postinstall
|
||||
// stays quiet; mention the init step when invoked interactively.
|
||||
if (!loadConfig()) {
|
||||
if (cli.list) console.log('No brain configured. Run `gbrain init` to set one up.');
|
||||
else if (cli.dryRun) console.log('No brain configured (run `gbrain init` first). Nothing to migrate.');
|
||||
return;
|
||||
}
|
||||
|
||||
const completed = loadCompletedMigrations();
|
||||
const idx = indexCompleted(completed);
|
||||
const plan = buildPlan(idx, installed, cli.specificMigration);
|
||||
|
||||
if (cli.specificMigration && plan.applied.length + plan.partial.length + plan.pending.length + plan.skippedFuture.length === 0) {
|
||||
console.error(`No migration registered with version "${cli.specificMigration}". Run \`gbrain apply-migrations --list\` to see registered versions.`);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
if (cli.list) { printList(plan, installed); return; }
|
||||
if (cli.dryRun) { printDryRun(plan, installed); return; }
|
||||
|
||||
const toRun: Migration[] = [...plan.partial, ...plan.pending];
|
||||
if (toRun.length === 0) {
|
||||
console.log('All migrations up to date.');
|
||||
return;
|
||||
}
|
||||
|
||||
// Run each orchestrator in registry order. An orchestrator failure aborts
|
||||
// the rest of the chain; fixing the failure and re-running picks up where
|
||||
// we left off (per-phase idempotency markers + resume from "partial").
|
||||
let failed = false;
|
||||
for (const m of toRun) {
|
||||
console.log(`\n=== Applying migration v${m.version}: ${m.featurePitch.headline} ===`);
|
||||
try {
|
||||
const result = await m.orchestrator(orchestratorOptsFrom(cli));
|
||||
if (result.status === 'failed') {
|
||||
console.error(`Migration v${m.version} reported status=failed.`);
|
||||
failed = true;
|
||||
break;
|
||||
}
|
||||
if (result.status === 'partial') {
|
||||
console.log(`Migration v${m.version} finished as PARTIAL. Re-run \`gbrain apply-migrations --yes\` after resolving any pending host-work items.`);
|
||||
} else {
|
||||
console.log(`Migration v${m.version} complete.`);
|
||||
}
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
console.error(`Migration v${m.version} threw: ${msg}`);
|
||||
failed = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (failed) process.exit(1);
|
||||
}
|
||||
|
||||
/** Exported for unit tests only. Do not use from production code. */
|
||||
export const __testing = {
|
||||
parseArgs,
|
||||
buildPlan,
|
||||
indexCompleted,
|
||||
statusForVersion,
|
||||
};
|
||||
@@ -1,20 +1,28 @@
|
||||
/**
|
||||
* gbrain autopilot — Self-maintaining brain daemon.
|
||||
*
|
||||
* Runs: sync → extract → embed → backlinks fix in a continuous loop.
|
||||
* Health-based adaptive scheduling. Best-effort per step.
|
||||
* v0.11.1 shape:
|
||||
* - Default path (minion_mode != off AND engine == postgres): spawn a
|
||||
* `gbrain jobs work` child process, submit ONE `autopilot-cycle` job
|
||||
* per interval with an idempotency_key so slow cycles don't stack up.
|
||||
* The forked worker drains the queue durably; restart with 10s backoff
|
||||
* on crash (5-crash cap → autopilot stops with a clear error).
|
||||
* - Fallback (minion_mode=off, PGLite, or `--inline`): run sync →
|
||||
* extract → embed inline, same as pre-v0.11.1 behavior.
|
||||
*
|
||||
* Usage:
|
||||
* gbrain autopilot [--repo <path>] [--interval N] [--json]
|
||||
* gbrain autopilot [--repo <path>] [--interval N] [--json] [--inline]
|
||||
* gbrain autopilot --install [--repo <path>]
|
||||
* gbrain autopilot --uninstall
|
||||
* gbrain autopilot --status [--json]
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync, writeFileSync, mkdirSync, appendFileSync } from 'fs';
|
||||
import { existsSync, readFileSync, writeFileSync, mkdirSync, appendFileSync, utimesSync, unlinkSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { execSync } from 'child_process';
|
||||
import { execSync, spawn, type ChildProcess } from 'child_process';
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { loadPreferences } from '../core/preferences.ts';
|
||||
import { loadConfig } from '../core/config.ts';
|
||||
|
||||
function parseArg(args: string[], flag: string): string | undefined {
|
||||
const idx = args.indexOf(flag);
|
||||
@@ -33,6 +41,35 @@ function logError(phase: string, e: unknown) {
|
||||
} catch { /* best-effort */ }
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the gbrain CLI entrypoint for spawning the worker child.
|
||||
*
|
||||
* Codex caught the bug in earlier plan drafts: `process.execPath` is the
|
||||
* Bun (or Node) runtime binary on source installs, not `gbrain`. Blindly
|
||||
* using it would spawn `bun jobs work`, which does not work.
|
||||
*
|
||||
* Order of resolution:
|
||||
* 1. argv[1] if it clearly points at a gbrain entry (cli.ts or /gbrain).
|
||||
* 2. process.execPath when running as the compiled binary.
|
||||
* 3. `which gbrain` for installs where the binary is on $PATH.
|
||||
* 4. Throw — nothing on $PATH, no way to supervise the worker.
|
||||
*/
|
||||
export function resolveGbrainCliPath(): string {
|
||||
const arg1 = process.argv[1] ?? '';
|
||||
if (arg1.endsWith('/gbrain') || arg1.endsWith('/cli.ts') || arg1.endsWith('\\gbrain.exe')) {
|
||||
return arg1;
|
||||
}
|
||||
const exec = process.execPath ?? '';
|
||||
if (exec.endsWith('/gbrain') || exec.endsWith('\\gbrain.exe')) {
|
||||
return exec;
|
||||
}
|
||||
try {
|
||||
const which = execSync('which gbrain', { encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'] }).trim();
|
||||
if (which) return which;
|
||||
} catch { /* not on $PATH */ }
|
||||
throw new Error('Could not resolve the gbrain CLI path. Install gbrain so it is on $PATH, or run autopilot from the compiled binary directly.');
|
||||
}
|
||||
|
||||
export async function runAutopilot(engine: BrainEngine, args: string[]) {
|
||||
if (args.includes('--help') || args.includes('-h')) {
|
||||
console.log('Usage: gbrain autopilot [--repo <path>] [--interval N] [--json]\n gbrain autopilot --install [--repo <path>]\n gbrain autopilot --uninstall\n gbrain autopilot --status [--json]\n\nSelf-maintaining brain daemon. Runs sync + extract + embed + backlinks in a loop.');
|
||||
@@ -55,6 +92,7 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
|
||||
const repoPath = parseArg(args, '--repo') || await engine.getConfig('sync.repo_path');
|
||||
const baseInterval = parseInt(parseArg(args, '--interval') || '300', 10);
|
||||
const jsonMode = args.includes('--json');
|
||||
const forceInline = args.includes('--inline');
|
||||
|
||||
if (!repoPath) {
|
||||
console.error('No repo path. Use --repo or run gbrain sync --repo first.');
|
||||
@@ -79,12 +117,69 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
|
||||
|
||||
console.log(`Autopilot starting. Repo: ${repoPath}, interval: ${baseInterval}s`);
|
||||
|
||||
// Signal handling + lock cleanup
|
||||
// Mode resolution: Minions dispatch when the user has opted in AND the
|
||||
// worker daemon can actually run (Postgres only; PGLite's exclusive file
|
||||
// lock blocks a separate worker process).
|
||||
const mode = loadPreferences().minion_mode ?? 'pain_triggered';
|
||||
const cfg = loadConfig();
|
||||
const engineType = cfg?.engine ?? 'pglite';
|
||||
const useMinionsDispatch = mode !== 'off' && engineType === 'postgres' && !forceInline;
|
||||
|
||||
let stopping = false;
|
||||
const cleanup = () => { try { require('fs').unlinkSync(lockPath); } catch {} };
|
||||
process.on('exit', cleanup);
|
||||
process.on('SIGTERM', () => { stopping = true; console.log('Autopilot stopping (SIGTERM).'); });
|
||||
process.on('SIGINT', () => { stopping = true; console.log('Autopilot stopping (SIGINT).'); });
|
||||
let workerProc: ChildProcess | null = null;
|
||||
let crashCount = 0;
|
||||
|
||||
if (useMinionsDispatch) {
|
||||
const cliPath = resolveGbrainCliPath();
|
||||
const startWorker = () => {
|
||||
const child = spawn(cliPath, ['jobs', 'work'], { stdio: 'inherit', env: process.env });
|
||||
workerProc = child;
|
||||
console.log(`[autopilot] Minions worker spawned (pid: ${child.pid})`);
|
||||
child.on('exit', (code) => {
|
||||
workerProc = null;
|
||||
if (stopping) return;
|
||||
if (crashCount >= 5) {
|
||||
console.error('[autopilot] 5 consecutive worker crashes, giving up.');
|
||||
process.exit(1);
|
||||
}
|
||||
crashCount++;
|
||||
console.error(`[autopilot] worker exited code=${code}, restart #${crashCount} in 10s`);
|
||||
setTimeout(startWorker, 10_000);
|
||||
});
|
||||
};
|
||||
startWorker();
|
||||
} else {
|
||||
const why = mode === 'off' ? 'minion_mode=off'
|
||||
: (engineType !== 'postgres' ? 'engine=pglite' : 'flag=--inline');
|
||||
console.log(`[autopilot] running steps inline (${why})`);
|
||||
}
|
||||
|
||||
// Async shutdown with 35s drain window for the worker child. The worker
|
||||
// has its own SIGTERM handler (minions/worker.ts:79-85) that drains
|
||||
// in-flight jobs for up to 30s before exit. We give it 35s here to
|
||||
// account for signal-delivery latency, then SIGKILL as a last resort.
|
||||
//
|
||||
// No `process.on('exit')` handler — its callback runs synchronously and
|
||||
// cannot await the worker's drain.
|
||||
const shutdown = async (sig: string) => {
|
||||
if (stopping) return;
|
||||
stopping = true;
|
||||
console.log(`Autopilot stopping (${sig}).`);
|
||||
if (workerProc) {
|
||||
try { workerProc.kill('SIGTERM'); } catch { /* already dead */ }
|
||||
await Promise.race([
|
||||
new Promise<void>(r => workerProc!.once('exit', () => r())),
|
||||
new Promise<void>(r => setTimeout(() => r(), 35_000)),
|
||||
]);
|
||||
if (workerProc && !workerProc.killed) {
|
||||
try { workerProc.kill('SIGKILL'); } catch { /* already dead */ }
|
||||
}
|
||||
}
|
||||
try { unlinkSync(lockPath); } catch { /* already gone */ }
|
||||
process.exit(0);
|
||||
};
|
||||
process.on('SIGTERM', () => { void shutdown('SIGTERM'); });
|
||||
process.on('SIGINT', () => { void shutdown('SIGINT'); });
|
||||
|
||||
let consecutiveErrors = 0;
|
||||
|
||||
@@ -92,6 +187,10 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
|
||||
const cycleStart = Date.now();
|
||||
let cycleOk = true;
|
||||
|
||||
// Refresh the lock mtime so another cron-fired autopilot doesn't
|
||||
// declare the instance stale after 10 minutes (Codex C).
|
||||
try { utimesSync(lockPath, new Date(), new Date()); } catch { /* best-effort */ }
|
||||
|
||||
// DB health check (reconnect if needed)
|
||||
try {
|
||||
await engine.getConfig('version');
|
||||
@@ -102,28 +201,57 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
|
||||
} catch (e) { logError('reconnect', e); }
|
||||
}
|
||||
|
||||
// 1. Sync
|
||||
try {
|
||||
const { performSync } = await import('./sync.ts');
|
||||
const result = await performSync(engine, { repoPath, noEmbed: true });
|
||||
if (result.status === 'synced') {
|
||||
console.log(`[sync] +${result.added} ~${result.modified} -${result.deleted}`);
|
||||
}
|
||||
} catch (e) { logError('sync', e); cycleOk = false; }
|
||||
if (useMinionsDispatch) {
|
||||
// Submit ONE autopilot-cycle job per cycle slot. The idempotency key
|
||||
// dedupes overrun submissions — if a cycle's job runs longer than
|
||||
// the interval, the next submission is a no-op at the DB layer
|
||||
// (ON CONFLICT DO NOTHING on the unique partial index).
|
||||
try {
|
||||
const { MinionQueue } = await import('../core/minions/queue.ts');
|
||||
const queue = new MinionQueue(engine);
|
||||
const slotMs = Math.floor(Date.now() / (baseInterval * 1000)) * baseInterval * 1000;
|
||||
const slot = new Date(slotMs).toISOString();
|
||||
const timeoutMs = Math.max(baseInterval * 2 * 1000, 300_000);
|
||||
const job = await queue.add('autopilot-cycle',
|
||||
{ repoPath },
|
||||
{
|
||||
queue: 'default',
|
||||
idempotency_key: `autopilot-cycle:${slot}`,
|
||||
max_attempts: 2,
|
||||
timeout_ms: timeoutMs,
|
||||
},
|
||||
);
|
||||
if (jsonMode) {
|
||||
process.stderr.write(JSON.stringify({ event: 'dispatched', job_id: job.id, slot }) + '\n');
|
||||
} else {
|
||||
console.log(`[dispatch] job #${job.id} autopilot-cycle slot=${slot}`);
|
||||
}
|
||||
} catch (e) { logError('dispatch', e); cycleOk = false; }
|
||||
} else {
|
||||
// Inline fallback — same as pre-v0.11.1 behavior.
|
||||
// 1. Sync
|
||||
try {
|
||||
const { performSync } = await import('./sync.ts');
|
||||
const result = await performSync(engine, { repoPath, noEmbed: true });
|
||||
if (result.status === 'synced') {
|
||||
console.log(`[sync] +${result.added} ~${result.modified} -${result.deleted}`);
|
||||
}
|
||||
} catch (e) { logError('sync', e); cycleOk = false; }
|
||||
|
||||
// 2. Extract (full brain, incremental dedup handles repeats)
|
||||
try {
|
||||
const { runExtract } = await import('./extract.ts');
|
||||
await runExtract(engine, ['all', '--dir', repoPath]);
|
||||
} catch (e) { logError('extract', e); cycleOk = false; }
|
||||
// 2. Extract (full brain, incremental dedup handles repeats)
|
||||
try {
|
||||
const { runExtractCore } = await import('./extract.ts');
|
||||
await runExtractCore(engine, { mode: 'all', dir: repoPath });
|
||||
} catch (e) { logError('extract', e); cycleOk = false; }
|
||||
|
||||
// 3. Embed stale
|
||||
try {
|
||||
const { runEmbed } = await import('./embed.ts');
|
||||
await runEmbed(engine, ['--stale']);
|
||||
} catch (e) { logError('embed', e); cycleOk = false; }
|
||||
// 3. Embed stale
|
||||
try {
|
||||
const { runEmbedCore } = await import('./embed.ts');
|
||||
await runEmbedCore(engine, { stale: true });
|
||||
} catch (e) { logError('embed', e); cycleOk = false; }
|
||||
}
|
||||
|
||||
// 4. Health check + adaptive interval
|
||||
// 4. Health check + adaptive interval (same for both paths)
|
||||
let interval = baseInterval;
|
||||
try {
|
||||
const health = await engine.getHealth();
|
||||
@@ -147,7 +275,8 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
|
||||
consecutiveErrors++;
|
||||
if (consecutiveErrors >= 5) {
|
||||
console.error('5 consecutive cycle failures. Stopping autopilot.');
|
||||
process.exit(1);
|
||||
void shutdown('cycle-failure-cap');
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -162,22 +291,75 @@ function plistPath(): string {
|
||||
return join(process.env.HOME || '', 'Library', 'LaunchAgents', 'com.gbrain.autopilot.plist');
|
||||
}
|
||||
|
||||
async function installDaemon(engine: BrainEngine, args: string[]) {
|
||||
const repoPath = parseArg(args, '--repo') || await engine.getConfig('sync.repo_path');
|
||||
if (!repoPath) {
|
||||
console.error('No repo path. Use --repo or run gbrain sync --repo first.');
|
||||
process.exit(1);
|
||||
function systemdUnitPath(): string {
|
||||
return join(process.env.HOME || '', '.config', 'systemd', 'user', 'gbrain-autopilot.service');
|
||||
}
|
||||
|
||||
function ephemeralStartScriptPath(): string {
|
||||
return join(process.env.HOME || '', '.gbrain', 'start-autopilot.sh');
|
||||
}
|
||||
|
||||
export type InstallTarget = 'macos' | 'linux-systemd' | 'ephemeral-container' | 'linux-cron';
|
||||
|
||||
/**
|
||||
* Detect the right supervisor for this host.
|
||||
*
|
||||
* - macos → launchd (always, when platform === 'darwin').
|
||||
* - ephemeral-container → Render / Railway / Fly / Docker. Crontab is
|
||||
* unreliable here (wiped on deploy); we hand
|
||||
* the user a start script instead.
|
||||
* - linux-systemd → systemd user scope actually works (is-system-running
|
||||
* probe succeeds). Codex hardened from the naive
|
||||
* /run/systemd/system check.
|
||||
* - linux-cron → fallback.
|
||||
*/
|
||||
export function detectInstallTarget(): InstallTarget {
|
||||
if (process.platform === 'darwin') return 'macos';
|
||||
|
||||
const ephemeral = !!(
|
||||
process.env.RENDER
|
||||
|| process.env.RAILWAY_ENVIRONMENT
|
||||
|| process.env.FLY_APP_NAME
|
||||
|| existsSync('/.dockerenv')
|
||||
);
|
||||
if (ephemeral) return 'ephemeral-container';
|
||||
|
||||
if (existsSync('/run/systemd/system')) {
|
||||
try {
|
||||
execSync('systemctl --user is-system-running', { stdio: 'pipe', timeout: 3000 });
|
||||
return 'linux-systemd';
|
||||
} catch {
|
||||
// user bus not available → fall through to cron.
|
||||
}
|
||||
}
|
||||
|
||||
return 'linux-cron';
|
||||
}
|
||||
|
||||
function detectOpenClaw(): { detected: boolean; bootstrapCandidates: string[] } {
|
||||
const home = process.env.HOME || '';
|
||||
const candidates = [
|
||||
process.env.OPENCLAW_HOME ? join(process.env.OPENCLAW_HOME, 'hooks', 'bootstrap', 'ensure-services.sh') : '',
|
||||
join(process.cwd(), 'hooks', 'bootstrap', 'ensure-services.sh'),
|
||||
join(home, '.claude', 'hooks', 'bootstrap', 'ensure-services.sh'),
|
||||
].filter(Boolean) as string[];
|
||||
const existing = candidates.filter(p => existsSync(p));
|
||||
const signal = !!process.env.OPENCLAW_HOME
|
||||
|| existsSync(join(process.cwd(), 'openclaw.json'))
|
||||
|| existsSync(join(home, 'openclaw.json'))
|
||||
|| existing.length > 0;
|
||||
return { detected: signal, bootstrapCandidates: existing };
|
||||
}
|
||||
|
||||
function writeWrapperScript(repoPath: string): string {
|
||||
const home = process.env.HOME || '';
|
||||
const gbrainDir = join(home, '.gbrain');
|
||||
mkdirSync(gbrainDir, { recursive: true });
|
||||
|
||||
// Write a wrapper script that sources the user's shell profile for API keys
|
||||
// instead of baking secrets into plist/crontab (#2: no plaintext keys in config files)
|
||||
// Wrapper sources the user's shell profile for API keys so nothing is
|
||||
// baked into plist/crontab/systemd unit files (#2).
|
||||
const wrapperPath = join(gbrainDir, 'autopilot-run.sh');
|
||||
const gbrainPath = process.execPath;
|
||||
// Shell-escape values to prevent command injection (#1)
|
||||
const gbrainPath = resolveGbrainCliPath();
|
||||
const safeRepoPath = repoPath.replace(/'/g, "'\\''");
|
||||
const safeGbrainPath = gbrainPath.replace(/'/g, "'\\''");
|
||||
const wrapper = `#!/bin/bash
|
||||
@@ -187,10 +369,47 @@ source ~/.zshrc 2>/dev/null || source ~/.bashrc 2>/dev/null || true
|
||||
exec '${safeGbrainPath}' autopilot --repo '${safeRepoPath}'
|
||||
`;
|
||||
writeFileSync(wrapperPath, wrapper, { mode: 0o755 });
|
||||
return wrapperPath;
|
||||
}
|
||||
|
||||
if (process.platform === 'darwin') {
|
||||
// macOS: launchd plist — runs wrapper script (no secrets in plist)
|
||||
const plist = `<?xml version="1.0" encoding="UTF-8"?>
|
||||
async function installDaemon(engine: BrainEngine, args: string[]) {
|
||||
const repoPath = parseArg(args, '--repo') || await engine.getConfig('sync.repo_path');
|
||||
if (!repoPath) {
|
||||
console.error('No repo path. Use --repo or run gbrain sync --repo first.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const forcedTarget = parseArg(args, '--target') as InstallTarget | undefined;
|
||||
const target: InstallTarget = forcedTarget ?? detectInstallTarget();
|
||||
|
||||
const injectBootstrap = args.includes('--inject-bootstrap');
|
||||
const noInject = args.includes('--no-inject');
|
||||
|
||||
const wrapperPath = writeWrapperScript(repoPath);
|
||||
const home = process.env.HOME || '';
|
||||
|
||||
switch (target) {
|
||||
case 'macos':
|
||||
installLaunchd(wrapperPath, home, repoPath);
|
||||
break;
|
||||
case 'linux-systemd':
|
||||
installSystemd(wrapperPath, repoPath);
|
||||
break;
|
||||
case 'ephemeral-container':
|
||||
installEphemeralContainer(wrapperPath, home, repoPath, { injectBootstrap, noInject });
|
||||
break;
|
||||
case 'linux-cron':
|
||||
installCrontab(wrapperPath, home);
|
||||
break;
|
||||
default: {
|
||||
console.error(`Unknown --target "${forcedTarget}". Allowed: macos, linux-systemd, ephemeral-container, linux-cron.`);
|
||||
process.exit(2);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function installLaunchd(wrapperPath: string, home: string, repoPath: string) {
|
||||
const plist = `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
@@ -205,83 +424,255 @@ exec '${safeGbrainPath}' autopilot --repo '${safeRepoPath}'
|
||||
</dict>
|
||||
</plist>`;
|
||||
|
||||
try {
|
||||
const agentsDir = join(home, 'Library', 'LaunchAgents');
|
||||
mkdirSync(agentsDir, { recursive: true });
|
||||
writeFileSync(plistPath(), plist);
|
||||
execSync(`launchctl load "${plistPath()}"`, { stdio: 'pipe' });
|
||||
console.log(`Installed launchd service: com.gbrain.autopilot`);
|
||||
console.log(` Repo: ${repoPath}`);
|
||||
console.log(` Log: ~/.gbrain/autopilot.log`);
|
||||
console.log(` Uninstall: gbrain autopilot --uninstall`);
|
||||
} catch (e: unknown) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
if (msg.includes('EACCES') || msg.includes('Permission')) {
|
||||
console.error(`Permission denied writing plist. Try: mkdir -p ~/Library/LaunchAgents`);
|
||||
} else {
|
||||
console.error(`Failed to install: ${msg}`);
|
||||
}
|
||||
process.exit(1);
|
||||
try {
|
||||
const agentsDir = join(home, 'Library', 'LaunchAgents');
|
||||
mkdirSync(agentsDir, { recursive: true });
|
||||
writeFileSync(plistPath(), plist);
|
||||
execSync(`launchctl load "${plistPath()}"`, { stdio: 'pipe' });
|
||||
console.log('Installed launchd service: com.gbrain.autopilot');
|
||||
console.log(` Repo: ${repoPath}`);
|
||||
console.log(` Log: ~/.gbrain/autopilot.log`);
|
||||
console.log(' Uninstall: gbrain autopilot --uninstall');
|
||||
} catch (e: unknown) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
if (msg.includes('EACCES') || msg.includes('Permission')) {
|
||||
console.error('Permission denied writing plist. Try: mkdir -p ~/Library/LaunchAgents');
|
||||
} else {
|
||||
console.error(`Failed to install: ${msg}`);
|
||||
}
|
||||
} else {
|
||||
// Linux/WSL: crontab — runs wrapper script (no secrets in crontab)
|
||||
const safeWrapperPath = wrapperPath.replace(/'/g, "'\\''");
|
||||
const cronLine = `*/5 * * * * '${safeWrapperPath}' >> '${home.replace(/'/g, "'\\''")}/.gbrain/autopilot.log' 2>&1`;
|
||||
try {
|
||||
const existing = execSync('crontab -l 2>/dev/null || true', { encoding: 'utf-8' });
|
||||
if (existing.includes('gbrain autopilot') || existing.includes('autopilot-run.sh')) {
|
||||
console.log('Crontab entry already exists. Remove with: gbrain autopilot --uninstall');
|
||||
return;
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
function installSystemd(wrapperPath: string, repoPath: string) {
|
||||
const unit = `[Unit]
|
||||
Description=GBrain Autopilot
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=${wrapperPath}
|
||||
Restart=on-failure
|
||||
RestartSec=30
|
||||
StandardOutput=append:%h/.gbrain/autopilot.log
|
||||
StandardError=append:%h/.gbrain/autopilot.err
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
`;
|
||||
try {
|
||||
const unitPath = systemdUnitPath();
|
||||
mkdirSync(join(process.env.HOME || '', '.config', 'systemd', 'user'), { recursive: true });
|
||||
writeFileSync(unitPath, unit);
|
||||
execSync('systemctl --user daemon-reload', { stdio: 'pipe', timeout: 10_000 });
|
||||
execSync('systemctl --user enable --now gbrain-autopilot.service', { stdio: 'pipe', timeout: 15_000 });
|
||||
console.log('Installed systemd user service: gbrain-autopilot.service');
|
||||
console.log(` Repo: ${repoPath}`);
|
||||
console.log(' Log: ~/.gbrain/autopilot.log');
|
||||
console.log(' Uninstall: gbrain autopilot --uninstall');
|
||||
} catch (e: unknown) {
|
||||
console.error(`Failed to install systemd unit: ${e instanceof Error ? e.message : e}`);
|
||||
console.error('You may need: `loginctl enable-linger $USER` so the unit runs without a login session.');
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
function installEphemeralContainer(
|
||||
wrapperPath: string,
|
||||
home: string,
|
||||
repoPath: string,
|
||||
opts: { injectBootstrap: boolean; noInject: boolean },
|
||||
) {
|
||||
// Write a start script the agent's bootstrap can source on every container start.
|
||||
const safeWrapperPath = wrapperPath.replace(/'/g, "'\\''");
|
||||
const script = `#!/bin/bash
|
||||
# Auto-generated by gbrain autopilot --install (ephemeral-container target)
|
||||
# Ephemeral filesystems lose crontab on every deploy; source this from
|
||||
# your agent's bootstrap instead.
|
||||
nohup '${safeWrapperPath}' > ~/.gbrain/autopilot.log 2>&1 &
|
||||
echo \$! > ~/.gbrain/autopilot.pid
|
||||
`;
|
||||
const scriptPath = ephemeralStartScriptPath();
|
||||
mkdirSync(join(home, '.gbrain'), { recursive: true });
|
||||
writeFileSync(scriptPath, script, { mode: 0o755 });
|
||||
|
||||
console.log('Ephemeral container detected (Render / Railway / Fly / Docker).');
|
||||
console.log(`Repo: ${repoPath}`);
|
||||
console.log(`Start script: ${scriptPath}`);
|
||||
console.log('');
|
||||
console.log('Crontab is unreliable here (wiped on deploy). Add ONE LINE to your');
|
||||
console.log('agent bootstrap to launch autopilot on every start:');
|
||||
console.log('');
|
||||
console.log(` bash ${scriptPath}`);
|
||||
console.log('');
|
||||
|
||||
// OpenClaw detection + optional auto-injection into ensure-services.sh.
|
||||
const { detected, bootstrapCandidates } = detectOpenClaw();
|
||||
if (detected) {
|
||||
console.log(`OpenClaw detected. Bootstrap candidates found:`);
|
||||
for (const p of bootstrapCandidates) console.log(` - ${p}`);
|
||||
console.log('');
|
||||
}
|
||||
|
||||
const shouldInject = (injectOpts: { detected: boolean; injectBootstrap: boolean; noInject: boolean }) => {
|
||||
if (injectOpts.noInject) return false;
|
||||
// Auto-inject by default when OpenClaw is detected + at least one
|
||||
// candidate exists. Users can explicitly opt in with --inject-bootstrap
|
||||
// on other hosts (uncommon).
|
||||
if (injectOpts.detected && bootstrapCandidates.length > 0) return true;
|
||||
return injectOpts.injectBootstrap;
|
||||
};
|
||||
|
||||
if (shouldInject({ detected, injectBootstrap: opts.injectBootstrap, noInject: opts.noInject })) {
|
||||
for (const candidate of bootstrapCandidates) {
|
||||
try {
|
||||
const existing = readFileSync(candidate, 'utf-8');
|
||||
const marker = '# gbrain:autopilot v0.11.0';
|
||||
if (existing.includes(marker)) {
|
||||
console.log(` [skip] ${candidate} already has the gbrain marker`);
|
||||
continue;
|
||||
}
|
||||
// Backup before edit
|
||||
const stamp = new Date().toISOString().replace(/[:.]/g, '-');
|
||||
const bakPath = `${candidate}.bak.${stamp}`;
|
||||
writeFileSync(bakPath, existing);
|
||||
const snippet = `\n${marker}\nbash ${scriptPath}\n`;
|
||||
writeFileSync(candidate, existing.trimEnd() + snippet);
|
||||
console.log(` [injected] ${candidate} (.bak at ${bakPath})`);
|
||||
} catch (e) {
|
||||
console.error(` [warn] failed to inject ${candidate}: ${e instanceof Error ? e.message : e}`);
|
||||
}
|
||||
// Use a temp file instead of echo pipe to avoid shell escaping issues (#1)
|
||||
const tmpFile = join(gbrainDir, 'crontab.tmp');
|
||||
writeFileSync(tmpFile, existing.trimEnd() + '\n' + cronLine + '\n');
|
||||
execSync(`crontab '${tmpFile.replace(/'/g, "'\\''")}'`, { stdio: 'pipe' });
|
||||
try { require('fs').unlinkSync(tmpFile); } catch {}
|
||||
console.log('Installed crontab entry for gbrain autopilot (every 5 minutes)');
|
||||
console.log(` Uninstall: gbrain autopilot --uninstall`);
|
||||
} catch (e: unknown) {
|
||||
console.error(`Failed to install crontab: ${e instanceof Error ? e.message : e}`);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
console.log(' Uninstall: gbrain autopilot --uninstall');
|
||||
}
|
||||
|
||||
function installCrontab(wrapperPath: string, home: string) {
|
||||
// Linux/WSL without systemd — crontab runs the wrapper every 5 minutes.
|
||||
const safeWrapperPath = wrapperPath.replace(/'/g, "'\\''");
|
||||
const cronLine = `*/5 * * * * '${safeWrapperPath}' >> '${home.replace(/'/g, "'\\''")}/.gbrain/autopilot.log' 2>&1`;
|
||||
try {
|
||||
const existing = execSync('crontab -l 2>/dev/null || true', { encoding: 'utf-8' });
|
||||
if (existing.includes('gbrain autopilot') || existing.includes('autopilot-run.sh')) {
|
||||
console.log('Crontab entry already exists. Remove with: gbrain autopilot --uninstall');
|
||||
return;
|
||||
}
|
||||
// Use a temp file instead of echo pipe to avoid shell escaping issues (#1)
|
||||
const tmpFile = join(home, '.gbrain', 'crontab.tmp');
|
||||
writeFileSync(tmpFile, existing.trimEnd() + '\n' + cronLine + '\n');
|
||||
execSync(`crontab '${tmpFile.replace(/'/g, "'\\''")}'`, { stdio: 'pipe' });
|
||||
try { unlinkSync(tmpFile); } catch { /* best-effort */ }
|
||||
console.log('Installed crontab entry for gbrain autopilot (every 5 minutes)');
|
||||
console.log(' Uninstall: gbrain autopilot --uninstall');
|
||||
} catch (e: unknown) {
|
||||
console.error(`Failed to install crontab: ${e instanceof Error ? e.message : e}`);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
function uninstallDaemon() {
|
||||
const home = process.env.HOME || '';
|
||||
const wrapperPath = join(home, '.gbrain', 'autopilot-run.sh');
|
||||
|
||||
if (process.platform === 'darwin') {
|
||||
// Always try all four targets — the user might have run `--install` under
|
||||
// one target earlier and moved hosts (e.g. macOS laptop → Linux server).
|
||||
// Each path is idempotent (missing files = skip silently).
|
||||
|
||||
let removed = 0;
|
||||
|
||||
// macOS launchd
|
||||
if (existsSync(plistPath())) {
|
||||
try {
|
||||
execSync(`launchctl unload "${plistPath()}" 2>/dev/null || true`, { stdio: 'pipe' });
|
||||
if (existsSync(plistPath())) {
|
||||
const { unlinkSync } = require('fs');
|
||||
unlinkSync(plistPath());
|
||||
}
|
||||
if (existsSync(wrapperPath)) {
|
||||
require('fs').unlinkSync(wrapperPath);
|
||||
}
|
||||
console.log('Uninstalled launchd service: com.gbrain.autopilot');
|
||||
} catch (e: unknown) {
|
||||
console.error(`Failed to uninstall: ${e instanceof Error ? e.message : e}`);
|
||||
unlinkSync(plistPath());
|
||||
console.log('Removed launchd service: com.gbrain.autopilot');
|
||||
removed++;
|
||||
} catch (e) {
|
||||
console.error(` [warn] launchd: ${e instanceof Error ? e.message : e}`);
|
||||
}
|
||||
} else {
|
||||
}
|
||||
|
||||
// Linux systemd user unit
|
||||
if (existsSync(systemdUnitPath())) {
|
||||
try {
|
||||
const existing = execSync('crontab -l 2>/dev/null || true', { encoding: 'utf-8' });
|
||||
execSync('systemctl --user disable --now gbrain-autopilot.service 2>/dev/null || true', { stdio: 'pipe', timeout: 10_000 });
|
||||
unlinkSync(systemdUnitPath());
|
||||
try { execSync('systemctl --user daemon-reload', { stdio: 'pipe', timeout: 5_000 }); } catch { /* best-effort */ }
|
||||
console.log('Removed systemd user service: gbrain-autopilot.service');
|
||||
removed++;
|
||||
} catch (e) {
|
||||
console.error(` [warn] systemd: ${e instanceof Error ? e.message : e}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Ephemeral container start script + bootstrap marker injection
|
||||
if (existsSync(ephemeralStartScriptPath())) {
|
||||
try {
|
||||
unlinkSync(ephemeralStartScriptPath());
|
||||
console.log('Removed ephemeral start script: ~/.gbrain/start-autopilot.sh');
|
||||
removed++;
|
||||
} catch (e) {
|
||||
console.error(` [warn] start script: ${e instanceof Error ? e.message : e}`);
|
||||
}
|
||||
}
|
||||
// Remove marker-line from any OpenClaw bootstrap we previously injected.
|
||||
try {
|
||||
const { bootstrapCandidates } = detectOpenClaw();
|
||||
for (const candidate of bootstrapCandidates) {
|
||||
try {
|
||||
const content = readFileSync(candidate, 'utf-8');
|
||||
if (!content.includes('# gbrain:autopilot v0.11.0')) continue;
|
||||
const lines = content.split('\n');
|
||||
const cleaned: string[] = [];
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
if (lines[i].includes('# gbrain:autopilot v0.11.0')) {
|
||||
// Skip this marker line AND the next line (the bash start-script call).
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
cleaned.push(lines[i]);
|
||||
}
|
||||
// Backup before edit
|
||||
const stamp = new Date().toISOString().replace(/[:.]/g, '-');
|
||||
writeFileSync(`${candidate}.bak.${stamp}`, content);
|
||||
writeFileSync(candidate, cleaned.join('\n'));
|
||||
console.log(`Removed bootstrap marker from: ${candidate}`);
|
||||
removed++;
|
||||
} catch (e) {
|
||||
console.error(` [warn] bootstrap ${candidate}: ${e instanceof Error ? e.message : e}`);
|
||||
}
|
||||
}
|
||||
} catch { /* OpenClaw detection best-effort */ }
|
||||
|
||||
// Linux crontab (don't gate on platform — the user may have run `--install
|
||||
// --target linux-cron` on a different machine that now has the crontab).
|
||||
try {
|
||||
const existing = execSync('crontab -l 2>/dev/null || true', { encoding: 'utf-8' });
|
||||
if (existing.includes('gbrain autopilot') || existing.includes('autopilot-run.sh')) {
|
||||
const filtered = existing.split('\n').filter(l =>
|
||||
!l.includes('gbrain autopilot') && !l.includes('autopilot-run.sh')
|
||||
!l.includes('gbrain autopilot') && !l.includes('autopilot-run.sh'),
|
||||
).join('\n');
|
||||
const tmpFile = join(home, '.gbrain', 'crontab.tmp');
|
||||
mkdirSync(join(home, '.gbrain'), { recursive: true });
|
||||
writeFileSync(tmpFile, filtered);
|
||||
execSync(`crontab '${tmpFile.replace(/'/g, "'\\''")}' 2>/dev/null || true`, { stdio: 'pipe' });
|
||||
try { require('fs').unlinkSync(tmpFile); } catch {}
|
||||
if (existsSync(wrapperPath)) {
|
||||
require('fs').unlinkSync(wrapperPath);
|
||||
}
|
||||
try { unlinkSync(tmpFile); } catch { /* best-effort */ }
|
||||
console.log('Removed crontab entry for gbrain autopilot');
|
||||
} catch (e: unknown) {
|
||||
console.error(`Failed to uninstall: ${e instanceof Error ? e.message : e}`);
|
||||
removed++;
|
||||
}
|
||||
} catch (e) {
|
||||
console.error(` [warn] crontab: ${e instanceof Error ? e.message : e}`);
|
||||
}
|
||||
|
||||
// Wrapper script — shared by all targets
|
||||
if (existsSync(wrapperPath)) {
|
||||
try {
|
||||
unlinkSync(wrapperPath);
|
||||
} catch { /* best-effort */ }
|
||||
}
|
||||
|
||||
if (removed === 0) {
|
||||
console.log('No autopilot install found on this host. Nothing to uninstall.');
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -168,6 +168,43 @@ export function fixBacklinkGaps(brainDir: string, gaps: BacklinkGap[], dryRun: b
|
||||
return fixed;
|
||||
}
|
||||
|
||||
export interface BacklinksOpts {
|
||||
action: 'check' | 'fix';
|
||||
dir: string;
|
||||
dryRun?: boolean;
|
||||
}
|
||||
|
||||
export interface BacklinksResult {
|
||||
action: 'check' | 'fix';
|
||||
gaps_found: number;
|
||||
fixed: number;
|
||||
pages_affected: number;
|
||||
dryRun: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Library-level backlinks check/fix. Throws on validation errors; returns a
|
||||
* structured result so Minions handlers + autopilot-cycle can surface counts.
|
||||
* Safe to call from the worker — no process.exit.
|
||||
*/
|
||||
export async function runBacklinksCore(opts: BacklinksOpts): Promise<BacklinksResult> {
|
||||
if (!['check', 'fix'].includes(opts.action)) {
|
||||
throw new Error(`Invalid backlinks action "${opts.action}". Allowed: check, fix.`);
|
||||
}
|
||||
if (!existsSync(opts.dir)) {
|
||||
throw new Error(`Directory not found: ${opts.dir}`);
|
||||
}
|
||||
|
||||
const gaps = findBacklinkGaps(opts.dir);
|
||||
const pagesAffected = new Set(gaps.map(g => g.targetPage)).size;
|
||||
|
||||
if (opts.action === 'fix' && gaps.length > 0) {
|
||||
const fixed = fixBacklinkGaps(opts.dir, gaps, !!opts.dryRun);
|
||||
return { action: 'fix', gaps_found: gaps.length, fixed, pages_affected: pagesAffected, dryRun: !!opts.dryRun };
|
||||
}
|
||||
return { action: opts.action, gaps_found: gaps.length, fixed: 0, pages_affected: pagesAffected, dryRun: !!opts.dryRun };
|
||||
}
|
||||
|
||||
export async function runBacklinks(args: string[]) {
|
||||
const subcommand = args[0];
|
||||
const dirIdx = args.indexOf('--dir');
|
||||
@@ -183,19 +220,25 @@ export async function runBacklinks(args: string[]) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
if (!existsSync(brainDir)) {
|
||||
console.error(`Directory not found: ${brainDir}`);
|
||||
let result: BacklinksResult;
|
||||
try {
|
||||
result = await runBacklinksCore({
|
||||
action: subcommand as 'check' | 'fix',
|
||||
dir: brainDir,
|
||||
dryRun,
|
||||
});
|
||||
} catch (e) {
|
||||
console.error(e instanceof Error ? e.message : String(e));
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const gaps = findBacklinkGaps(brainDir);
|
||||
|
||||
if (gaps.length === 0) {
|
||||
if (result.gaps_found === 0) {
|
||||
console.log('No missing back-links found.');
|
||||
return;
|
||||
}
|
||||
|
||||
if (subcommand === 'check') {
|
||||
if (result.action === 'check') {
|
||||
// Re-walk for user-facing output (core returns counts, CLI shows detail).
|
||||
const gaps = findBacklinkGaps(brainDir);
|
||||
console.log(`Found ${gaps.length} missing back-link(s):\n`);
|
||||
for (const gap of gaps) {
|
||||
console.log(` ${gap.targetPage} <- ${gap.sourcePage}`);
|
||||
@@ -203,10 +246,9 @@ export async function runBacklinks(args: string[]) {
|
||||
}
|
||||
console.log(`\nRun 'gbrain check-backlinks fix --dir ${brainDir}' to create them.`);
|
||||
} else {
|
||||
const label = dryRun ? '(dry run) ' : '';
|
||||
const fixed = fixBacklinkGaps(brainDir, gaps, dryRun);
|
||||
console.log(`${label}Fixed ${fixed} missing back-link(s) across ${new Set(gaps.map(g => g.targetPage)).size} page(s).`);
|
||||
if (dryRun) {
|
||||
const label = result.dryRun ? '(dry run) ' : '';
|
||||
console.log(`${label}Fixed ${result.fixed} missing back-link(s) across ${result.pages_affected} page(s).`);
|
||||
if (result.dryRun) {
|
||||
console.log('\nRe-run without --dry-run to apply.');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@ import type { BrainEngine } from '../core/engine.ts';
|
||||
import * as db from '../core/db.ts';
|
||||
import { LATEST_VERSION } from '../core/migrate.ts';
|
||||
import { checkResolvable } from '../core/check-resolvable.ts';
|
||||
import { loadCompletedMigrations } from '../core/preferences.ts';
|
||||
import { join } from 'path';
|
||||
import { existsSync, readFileSync, readdirSync } from 'fs';
|
||||
|
||||
@@ -63,6 +64,39 @@ export async function runDoctor(engine: BrainEngine | null, args: string[]) {
|
||||
checks.push(conformanceResult);
|
||||
}
|
||||
|
||||
// 3. Half-migrated Minions detection (filesystem-only).
|
||||
// If completed.jsonl has any status:"partial" entry with no later
|
||||
// status:"complete" for the same version, the install is mid-migration.
|
||||
// Typical cause: v0.11.0 stopgap wrote a partial record but nobody ran
|
||||
// `gbrain apply-migrations --yes` afterward. This check fires on every
|
||||
// `gbrain doctor` invocation so Wintermute's health skill catches it.
|
||||
try {
|
||||
const completed = loadCompletedMigrations();
|
||||
const byVersion = new Map<string, { complete: boolean; partial: boolean }>();
|
||||
for (const entry of completed) {
|
||||
const seen = byVersion.get(entry.version) ?? { complete: false, partial: false };
|
||||
if (entry.status === 'complete') seen.complete = true;
|
||||
if (entry.status === 'partial') seen.partial = true;
|
||||
byVersion.set(entry.version, seen);
|
||||
}
|
||||
const stuck = Array.from(byVersion.entries())
|
||||
.filter(([, s]) => s.partial && !s.complete)
|
||||
.map(([v]) => v);
|
||||
if (stuck.length > 0) {
|
||||
checks.push({
|
||||
name: 'minions_migration',
|
||||
status: 'fail',
|
||||
message: `MINIONS HALF-INSTALLED (partial migration: ${stuck.join(', ')}). Run: gbrain apply-migrations --yes`,
|
||||
});
|
||||
}
|
||||
// Note: the "no preferences.json but schema is v7+" case is detected
|
||||
// in the DB section below (needs schema version).
|
||||
} catch (e) {
|
||||
// completed.jsonl read/parse failure is non-fatal — probably a fresh
|
||||
// install with no record yet. Don't warn here; the DB check below
|
||||
// handles the "schema v7+ but no prefs" case.
|
||||
}
|
||||
|
||||
// --- DB checks (skip if --fast or no engine) ---
|
||||
|
||||
if (fastMode || !engine) {
|
||||
@@ -120,18 +154,26 @@ export async function runDoctor(engine: BrainEngine | null, args: string[]) {
|
||||
}
|
||||
|
||||
// 6. Schema version
|
||||
let schemaVersion = 0;
|
||||
try {
|
||||
const version = await engine.getConfig('version');
|
||||
const v = parseInt(version || '0', 10);
|
||||
if (v >= LATEST_VERSION) {
|
||||
checks.push({ name: 'schema_version', status: 'ok', message: `Version ${v} (latest: ${LATEST_VERSION})` });
|
||||
schemaVersion = parseInt(version || '0', 10);
|
||||
if (schemaVersion >= LATEST_VERSION) {
|
||||
checks.push({ name: 'schema_version', status: 'ok', message: `Version ${schemaVersion} (latest: ${LATEST_VERSION})` });
|
||||
} else {
|
||||
checks.push({ name: 'schema_version', status: 'warn', message: `Version ${v}, latest is ${LATEST_VERSION}. Run gbrain init to migrate.` });
|
||||
checks.push({ name: 'schema_version', status: 'warn', message: `Version ${schemaVersion}, latest is ${LATEST_VERSION}. Run gbrain init to migrate.` });
|
||||
}
|
||||
} catch {
|
||||
checks.push({ name: 'schema_version', status: 'warn', message: 'Could not check schema version' });
|
||||
}
|
||||
|
||||
// Note: we intentionally DO NOT fail on "schema v7+ but no preferences.json".
|
||||
// That's a valid fresh-install state after `gbrain init` — the migration
|
||||
// orchestrator writes preferences, but `init` alone doesn't run it. The
|
||||
// partial-completed.jsonl check in the filesystem section (step 3) is
|
||||
// the canonical half-migration signal and fires when the stopgap ran
|
||||
// but `apply-migrations` didn't follow up.
|
||||
|
||||
// 7. Embedding health
|
||||
try {
|
||||
const health = await engine.getHealth();
|
||||
|
||||
@@ -3,29 +3,66 @@ import { embedBatch } from '../core/embedding.ts';
|
||||
import type { ChunkInput } from '../core/types.ts';
|
||||
import { chunkText } from '../core/chunkers/recursive.ts';
|
||||
|
||||
export interface EmbedOpts {
|
||||
/** Embed ALL pages (every chunk). */
|
||||
all?: boolean;
|
||||
/** Embed only stale chunks (missing embedding). */
|
||||
stale?: boolean;
|
||||
/** Embed specific pages by slug. */
|
||||
slugs?: string[];
|
||||
/** Embed a single page. */
|
||||
slug?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Library-level embed. Throws on validation errors; per-page embed failures
|
||||
* are logged to stderr but do not throw (matches the existing CLI semantics
|
||||
* for batch runs). Safe to call from Minions handlers — no process.exit.
|
||||
*/
|
||||
export async function runEmbedCore(engine: BrainEngine, opts: EmbedOpts): Promise<void> {
|
||||
if (opts.slugs && opts.slugs.length > 0) {
|
||||
for (const s of opts.slugs) {
|
||||
try { await embedPage(engine, s); } catch (e: unknown) {
|
||||
console.error(` Error embedding ${s}: ${e instanceof Error ? e.message : e}`);
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (opts.all || opts.stale) {
|
||||
await embedAll(engine, !!opts.stale);
|
||||
return;
|
||||
}
|
||||
if (opts.slug) {
|
||||
await embedPage(engine, opts.slug);
|
||||
return;
|
||||
}
|
||||
throw new Error('No embed target specified. Pass { slug }, { slugs }, { all }, or { stale }.');
|
||||
}
|
||||
|
||||
export async function runEmbed(engine: BrainEngine, args: string[]) {
|
||||
const slugsIdx = args.indexOf('--slugs');
|
||||
const all = args.includes('--all');
|
||||
const stale = args.includes('--stale');
|
||||
|
||||
let opts: EmbedOpts;
|
||||
if (slugsIdx >= 0) {
|
||||
// --slugs slug1 slug2 ... (embed specific pages)
|
||||
const slugs = args.slice(slugsIdx + 1).filter(a => !a.startsWith('--'));
|
||||
for (const s of slugs) {
|
||||
try { await embedPage(engine, s); } catch (e: unknown) {
|
||||
console.error(` Error embedding ${s}: ${e instanceof Error ? e.message : e}`);
|
||||
}
|
||||
}
|
||||
opts = { slugs: args.slice(slugsIdx + 1).filter(a => !a.startsWith('--')) };
|
||||
} else if (all || stale) {
|
||||
await embedAll(engine, stale);
|
||||
opts = { all, stale };
|
||||
} else {
|
||||
const slug = args.find(a => !a.startsWith('--'));
|
||||
if (slug) {
|
||||
await embedPage(engine, slug);
|
||||
} else {
|
||||
if (!slug) {
|
||||
console.error('Usage: gbrain embed [<slug>|--all|--stale|--slugs s1 s2 ...]');
|
||||
process.exit(1);
|
||||
}
|
||||
opts = { slug };
|
||||
}
|
||||
|
||||
try {
|
||||
await runEmbedCore(engine, opts);
|
||||
} catch (e) {
|
||||
console.error(e instanceof Error ? e.message : String(e));
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -174,6 +174,49 @@ export function extractTimelineFromContent(content: string, slug: string): Extra
|
||||
|
||||
// --- Main command ---
|
||||
|
||||
export interface ExtractOpts {
|
||||
/** What to extract: 'links' (wiki-style refs), 'timeline' (date entries), or 'all'. */
|
||||
mode: 'links' | 'timeline' | 'all';
|
||||
/** Brain directory to walk. */
|
||||
dir: string;
|
||||
/** Report what would change without writing. */
|
||||
dryRun?: boolean;
|
||||
/** Emit JSON (progress to stderr, result to stdout) instead of human text. */
|
||||
jsonMode?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Library-level extract. Throws on error; prints nothing unless jsonMode or
|
||||
* explicit output is warranted. Safe to call from Minions handlers because it
|
||||
* never calls process.exit — a bad mode or missing dir throws through, which
|
||||
* the handler wrapper turns into a failed job (NOT a killed worker).
|
||||
*/
|
||||
export async function runExtractCore(engine: BrainEngine, opts: ExtractOpts): Promise<ExtractResult> {
|
||||
if (!['links', 'timeline', 'all'].includes(opts.mode)) {
|
||||
throw new Error(`Invalid extract mode "${opts.mode}". Allowed: links, timeline, all.`);
|
||||
}
|
||||
if (!existsSync(opts.dir)) {
|
||||
throw new Error(`Directory not found: ${opts.dir}`);
|
||||
}
|
||||
|
||||
const dryRun = !!opts.dryRun;
|
||||
const jsonMode = !!opts.jsonMode;
|
||||
const result: ExtractResult = { links_created: 0, timeline_entries_created: 0, pages_processed: 0 };
|
||||
|
||||
if (opts.mode === 'links' || opts.mode === 'all') {
|
||||
const r = await extractLinksFromDir(engine, opts.dir, dryRun, jsonMode);
|
||||
result.links_created = r.created;
|
||||
result.pages_processed = r.pages;
|
||||
}
|
||||
if (opts.mode === 'timeline' || opts.mode === 'all') {
|
||||
const r = await extractTimelineFromDir(engine, opts.dir, dryRun, jsonMode);
|
||||
result.timeline_entries_created = r.created;
|
||||
result.pages_processed = Math.max(result.pages_processed, r.pages);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
export async function runExtract(engine: BrainEngine, args: string[]) {
|
||||
const subcommand = args[0];
|
||||
const dirIdx = args.indexOf('--dir');
|
||||
@@ -186,24 +229,19 @@ export async function runExtract(engine: BrainEngine, args: string[]) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
if (!existsSync(brainDir)) {
|
||||
console.error(`Directory not found: ${brainDir}`);
|
||||
let result: ExtractResult;
|
||||
try {
|
||||
result = await runExtractCore(engine, {
|
||||
mode: subcommand as 'links' | 'timeline' | 'all',
|
||||
dir: brainDir,
|
||||
dryRun,
|
||||
jsonMode,
|
||||
});
|
||||
} catch (e) {
|
||||
console.error(e instanceof Error ? e.message : String(e));
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const result: ExtractResult = { links_created: 0, timeline_entries_created: 0, pages_processed: 0 };
|
||||
|
||||
if (subcommand === 'links' || subcommand === 'all') {
|
||||
const r = await extractLinksFromDir(engine, brainDir, dryRun, jsonMode);
|
||||
result.links_created = r.created;
|
||||
result.pages_processed = r.pages;
|
||||
}
|
||||
if (subcommand === 'timeline' || subcommand === 'all') {
|
||||
const r = await extractTimelineFromDir(engine, brainDir, dryRun, jsonMode);
|
||||
result.timeline_entries_created = r.created;
|
||||
result.pages_processed = Math.max(result.pages_processed, r.pages);
|
||||
}
|
||||
|
||||
if (jsonMode) {
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
} else if (!dryRun) {
|
||||
|
||||
@@ -6,13 +6,14 @@ import { homedir } from 'os';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
import { saveConfig, type GBrainConfig } from '../core/config.ts';
|
||||
import { saveConfig, loadConfig, toEngineConfig, type GBrainConfig } from '../core/config.ts';
|
||||
import { createEngine } from '../core/engine-factory.ts';
|
||||
|
||||
export async function runInit(args: string[]) {
|
||||
const isSupabase = args.includes('--supabase');
|
||||
const isPGLite = args.includes('--pglite');
|
||||
const isNonInteractive = args.includes('--non-interactive');
|
||||
const isMigrateOnly = args.includes('--migrate-only');
|
||||
const jsonOutput = args.includes('--json');
|
||||
const urlIndex = args.indexOf('--url');
|
||||
const manualUrl = urlIndex !== -1 ? args[urlIndex + 1] : null;
|
||||
@@ -21,6 +22,15 @@ export async function runInit(args: string[]) {
|
||||
const pathIndex = args.indexOf('--path');
|
||||
const customPath = pathIndex !== -1 ? args[pathIndex + 1] : null;
|
||||
|
||||
// Schema-only path: apply initSchema against the already-configured engine
|
||||
// without ever calling saveConfig. Used by apply-migrations, the stopgap
|
||||
// script, and the postinstall hook. Bare `gbrain init` defaults to PGLite
|
||||
// and overwrites any existing Postgres config — we must never take that
|
||||
// branch from a migration orchestrator.
|
||||
if (isMigrateOnly) {
|
||||
return initMigrateOnly({ jsonOutput });
|
||||
}
|
||||
|
||||
// Explicit PGLite mode
|
||||
if (isPGLite || (!isSupabase && !manualUrl && !isNonInteractive)) {
|
||||
// Smart detection: scan for .md files unless --pglite flag forces it
|
||||
@@ -59,6 +69,39 @@ export async function runInit(args: string[]) {
|
||||
return initPostgres({ databaseUrl, jsonOutput, apiKey });
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply the schema against the already-configured engine. No saveConfig.
|
||||
* No PGLite fallback when no config exists. Used by migration orchestrators
|
||||
* to bump an existing brain's schema to the latest version without
|
||||
* clobbering the user's chosen engine.
|
||||
*/
|
||||
async function initMigrateOnly(opts: { jsonOutput: boolean }) {
|
||||
const config = loadConfig();
|
||||
if (!config) {
|
||||
const msg = 'No brain configured. Run `gbrain init` (interactive) or `gbrain init --pglite` / `gbrain init --supabase` first.';
|
||||
if (opts.jsonOutput) {
|
||||
console.log(JSON.stringify({ status: 'error', reason: 'no_config', message: msg }));
|
||||
} else {
|
||||
console.error(msg);
|
||||
}
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const engine = await createEngine(toEngineConfig(config));
|
||||
try {
|
||||
await engine.connect(toEngineConfig(config));
|
||||
await engine.initSchema();
|
||||
} finally {
|
||||
try { await engine.disconnect(); } catch { /* best-effort */ }
|
||||
}
|
||||
|
||||
if (opts.jsonOutput) {
|
||||
console.log(JSON.stringify({ status: 'success', engine: config.engine, mode: 'migrate-only' }));
|
||||
} else {
|
||||
console.log(`Schema up to date (engine: ${config.engine}).`);
|
||||
}
|
||||
}
|
||||
|
||||
async function initPGLite(opts: { jsonOutput: boolean; apiKey: string | null; customPath: string | null }) {
|
||||
const dbPath = opts.customPath || join(homedir(), '.gbrain', 'brain.pglite');
|
||||
console.log(`Setting up local brain with PGLite (no server needed)...`);
|
||||
|
||||
473
src/commands/jobs.ts
Normal file
473
src/commands/jobs.ts
Normal file
@@ -0,0 +1,473 @@
|
||||
/**
|
||||
* CLI handler for `gbrain jobs` subcommands.
|
||||
* Thin wrapper around MinionQueue and MinionWorker.
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { MinionQueue } from '../core/minions/queue.ts';
|
||||
import { MinionWorker } from '../core/minions/worker.ts';
|
||||
import type { MinionJob, MinionJobStatus } from '../core/minions/types.ts';
|
||||
|
||||
function parseFlag(args: string[], flag: string): string | undefined {
|
||||
const idx = args.indexOf(flag);
|
||||
return idx >= 0 && idx + 1 < args.length ? args[idx + 1] : undefined;
|
||||
}
|
||||
|
||||
function hasFlag(args: string[], flag: string): boolean {
|
||||
return args.includes(flag);
|
||||
}
|
||||
|
||||
function formatJob(job: MinionJob): string {
|
||||
const dur = job.finished_at && job.started_at
|
||||
? `${((job.finished_at.getTime() - job.started_at.getTime()) / 1000).toFixed(1)}s`
|
||||
: '—';
|
||||
const stalled = job.status === 'active' && job.lock_until && job.lock_until < new Date()
|
||||
? ' (stalled?)' : '';
|
||||
return ` ${String(job.id).padEnd(6)} ${job.name.padEnd(14)} ${(job.status + stalled).padEnd(20)} ${job.queue.padEnd(10)} ${dur.padEnd(8)} ${job.created_at.toISOString().slice(0, 19)}`;
|
||||
}
|
||||
|
||||
function formatJobDetail(job: MinionJob): string {
|
||||
const lines = [
|
||||
`Job #${job.id}: ${job.name} (${job.status.toUpperCase()}${job.status === 'dead' ? ` after ${job.attempts_made} attempts` : ''})`,
|
||||
` Queue: ${job.queue} | Priority: ${job.priority}`,
|
||||
` Attempts: ${job.attempts_made}/${job.max_attempts} (started: ${job.attempts_started})`,
|
||||
` Backoff: ${job.backoff_type} ${job.backoff_delay}ms (jitter: ${job.backoff_jitter})`,
|
||||
];
|
||||
if (job.started_at) lines.push(` Started: ${job.started_at.toISOString()}`);
|
||||
if (job.finished_at) lines.push(` Finished: ${job.finished_at.toISOString()}`);
|
||||
if (job.lock_token) lines.push(` Lock: ${job.lock_token} (until ${job.lock_until?.toISOString()})`);
|
||||
if (job.delay_until) lines.push(` Delayed until: ${job.delay_until.toISOString()}`);
|
||||
if (job.parent_job_id) lines.push(` Parent: job #${job.parent_job_id} (on_child_fail: ${job.on_child_fail})`);
|
||||
if (job.error_text) lines.push(` Error: ${job.error_text}`);
|
||||
if (job.stacktrace.length > 0) {
|
||||
lines.push(` History:`);
|
||||
for (const entry of job.stacktrace) lines.push(` - ${entry}`);
|
||||
}
|
||||
if (job.progress != null) lines.push(` Progress: ${JSON.stringify(job.progress)}`);
|
||||
if (job.result != null) lines.push(` Result: ${JSON.stringify(job.result)}`);
|
||||
lines.push(` Data: ${JSON.stringify(job.data)}`);
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
export async function runJobs(engine: BrainEngine, args: string[]): Promise<void> {
|
||||
const sub = args[0];
|
||||
|
||||
if (!sub || sub === '--help' || sub === '-h') {
|
||||
console.log(`gbrain jobs — Minions job queue
|
||||
|
||||
USAGE
|
||||
gbrain jobs submit <name> [--params JSON] [--follow] [--priority N]
|
||||
[--delay Nms] [--max-attempts N] [--queue Q]
|
||||
[--dry-run]
|
||||
gbrain jobs list [--status S] [--queue Q] [--limit N]
|
||||
gbrain jobs get <id>
|
||||
gbrain jobs cancel <id>
|
||||
gbrain jobs retry <id>
|
||||
gbrain jobs prune [--older-than 30d]
|
||||
gbrain jobs delete <id>
|
||||
gbrain jobs stats
|
||||
gbrain jobs smoke
|
||||
gbrain jobs work [--queue Q] [--concurrency N]
|
||||
`);
|
||||
return;
|
||||
}
|
||||
|
||||
const queue = new MinionQueue(engine);
|
||||
|
||||
switch (sub) {
|
||||
case 'submit': {
|
||||
const name = args[1];
|
||||
if (!name) {
|
||||
console.error('Error: job name required. Usage: gbrain jobs submit <name>');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const paramsStr = parseFlag(args, '--params');
|
||||
let data: Record<string, unknown> = {};
|
||||
if (paramsStr) {
|
||||
try { data = JSON.parse(paramsStr); }
|
||||
catch { console.error('Error: --params must be valid JSON'); process.exit(1); }
|
||||
}
|
||||
|
||||
const priority = parseInt(parseFlag(args, '--priority') ?? '0', 10);
|
||||
const delay = parseInt(parseFlag(args, '--delay') ?? '0', 10);
|
||||
const maxAttempts = parseInt(parseFlag(args, '--max-attempts') ?? '3', 10);
|
||||
const queueName = parseFlag(args, '--queue') ?? 'default';
|
||||
const dryRun = hasFlag(args, '--dry-run');
|
||||
const follow = hasFlag(args, '--follow');
|
||||
|
||||
if (dryRun) {
|
||||
console.log(`[DRY RUN] Would submit job:`);
|
||||
console.log(` Name: ${name}`);
|
||||
console.log(` Queue: ${queueName}`);
|
||||
console.log(` Priority: ${priority}`);
|
||||
console.log(` Max attempts: ${maxAttempts}`);
|
||||
if (delay > 0) console.log(` Delay: ${delay}ms`);
|
||||
console.log(` Data: ${JSON.stringify(data)}`);
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
await queue.ensureSchema();
|
||||
} catch (e) {
|
||||
console.error(e instanceof Error ? e.message : String(e));
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const job = await queue.add(name, data, {
|
||||
priority,
|
||||
delay: delay > 0 ? delay : undefined,
|
||||
max_attempts: maxAttempts,
|
||||
queue: queueName,
|
||||
});
|
||||
|
||||
if (follow) {
|
||||
console.log(`Job #${job.id} submitted (${name}). Executing inline...`);
|
||||
// Inline execution: run the job in this process
|
||||
const worker = new MinionWorker(engine, { queue: queueName, pollInterval: 100 });
|
||||
|
||||
// Register built-in handlers
|
||||
await registerBuiltinHandlers(worker, engine);
|
||||
|
||||
if (!worker.registeredNames.includes(name)) {
|
||||
console.error(`Error: Unknown job type '${name}'.`);
|
||||
console.error(`Available types: ${worker.registeredNames.join(', ')}`);
|
||||
console.error(`Register custom types with worker.register('${name}', handler).`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Run worker for one job then stop
|
||||
const startTime = Date.now();
|
||||
const workerPromise = worker.start();
|
||||
// Poll until this job completes
|
||||
const pollInterval = setInterval(async () => {
|
||||
const updated = await queue.getJob(job.id);
|
||||
if (updated && ['completed', 'failed', 'dead', 'cancelled'].includes(updated.status)) {
|
||||
worker.stop();
|
||||
clearInterval(pollInterval);
|
||||
}
|
||||
}, 200);
|
||||
await workerPromise;
|
||||
clearInterval(pollInterval);
|
||||
|
||||
const final = await queue.getJob(job.id);
|
||||
const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
|
||||
if (final?.status === 'completed') {
|
||||
console.log(`Job #${job.id} completed in ${elapsed}s`);
|
||||
if (final.result) console.log(`Result: ${JSON.stringify(final.result)}`);
|
||||
} else {
|
||||
console.error(`Job #${job.id} ${final?.status}: ${final?.error_text}`);
|
||||
process.exit(1);
|
||||
}
|
||||
} else {
|
||||
console.log(JSON.stringify(job, null, 2));
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case 'list': {
|
||||
const status = parseFlag(args, '--status') as MinionJobStatus | undefined;
|
||||
const queueName = parseFlag(args, '--queue');
|
||||
const limit = parseInt(parseFlag(args, '--limit') ?? '20', 10);
|
||||
|
||||
try { await queue.ensureSchema(); }
|
||||
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
|
||||
|
||||
const jobs = await queue.getJobs({ status, queue: queueName, limit });
|
||||
|
||||
if (jobs.length === 0) {
|
||||
console.log('No jobs found.');
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(` ${'ID'.padEnd(6)} ${'Name'.padEnd(14)} ${'Status'.padEnd(20)} ${'Queue'.padEnd(10)} ${'Time'.padEnd(8)} Created`);
|
||||
console.log(' ' + '─'.repeat(80));
|
||||
for (const job of jobs) console.log(formatJob(job));
|
||||
console.log(`\n ${jobs.length} jobs shown`);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'get': {
|
||||
const id = parseInt(args[1], 10);
|
||||
if (isNaN(id)) { console.error('Error: job ID required. Usage: gbrain jobs get <id>'); process.exit(1); }
|
||||
|
||||
try { await queue.ensureSchema(); }
|
||||
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
|
||||
|
||||
const job = await queue.getJob(id);
|
||||
if (!job) { console.error(`Job #${id} not found.`); process.exit(1); }
|
||||
console.log(formatJobDetail(job));
|
||||
break;
|
||||
}
|
||||
|
||||
case 'cancel': {
|
||||
const id = parseInt(args[1], 10);
|
||||
if (isNaN(id)) { console.error('Error: job ID required.'); process.exit(1); }
|
||||
|
||||
try { await queue.ensureSchema(); }
|
||||
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
|
||||
|
||||
const cancelled = await queue.cancelJob(id);
|
||||
if (cancelled) {
|
||||
console.log(`Job #${id} cancelled.`);
|
||||
} else {
|
||||
console.error(`Could not cancel job #${id} (may already be completed/dead).`);
|
||||
process.exit(1);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case 'retry': {
|
||||
const id = parseInt(args[1], 10);
|
||||
if (isNaN(id)) { console.error('Error: job ID required.'); process.exit(1); }
|
||||
|
||||
try { await queue.ensureSchema(); }
|
||||
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
|
||||
|
||||
const retried = await queue.retryJob(id);
|
||||
if (retried) {
|
||||
console.log(`Job #${id} re-queued for retry.`);
|
||||
} else {
|
||||
console.error(`Could not retry job #${id} (must be failed or dead).`);
|
||||
process.exit(1);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case 'delete': {
|
||||
const id = parseInt(args[1], 10);
|
||||
if (isNaN(id)) { console.error('Error: job ID required.'); process.exit(1); }
|
||||
|
||||
try { await queue.ensureSchema(); }
|
||||
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
|
||||
|
||||
const removed = await queue.removeJob(id);
|
||||
if (removed) {
|
||||
console.log(`Job #${id} deleted.`);
|
||||
} else {
|
||||
console.error(`Could not delete job #${id} (must be in a terminal status).`);
|
||||
process.exit(1);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case 'prune': {
|
||||
const olderThanStr = parseFlag(args, '--older-than') ?? '30d';
|
||||
const days = parseInt(olderThanStr, 10);
|
||||
if (isNaN(days) || days <= 0) {
|
||||
console.error('Error: --older-than must be a positive number (days). Example: --older-than 30d');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
try { await queue.ensureSchema(); }
|
||||
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
|
||||
|
||||
const count = await queue.prune({ olderThan: new Date(Date.now() - days * 86400000) });
|
||||
console.log(`Pruned ${count} jobs older than ${days} days.`);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'stats': {
|
||||
try { await queue.ensureSchema(); }
|
||||
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
|
||||
|
||||
const stats = await queue.getStats();
|
||||
|
||||
console.log('Job Stats (last 24h):');
|
||||
if (stats.by_type.length > 0) {
|
||||
console.log(` ${'Type'.padEnd(14)} ${'Total'.padEnd(7)} ${'Done'.padEnd(7)} ${'Failed'.padEnd(8)} ${'Dead'.padEnd(6)} Avg Time`);
|
||||
for (const t of stats.by_type) {
|
||||
const avgTime = t.avg_duration_ms != null ? `${(t.avg_duration_ms / 1000).toFixed(1)}s` : '—';
|
||||
console.log(` ${t.name.padEnd(14)} ${String(t.total).padEnd(7)} ${String(t.completed).padEnd(7)} ${String(t.failed).padEnd(8)} ${String(t.dead).padEnd(6)} ${avgTime}`);
|
||||
}
|
||||
} else {
|
||||
console.log(' No jobs in the last 24 hours.');
|
||||
}
|
||||
console.log(`\n Queue health: ${stats.queue_health.waiting} waiting, ${stats.queue_health.active} active, ${stats.queue_health.stalled} stalled`);
|
||||
break;
|
||||
}
|
||||
|
||||
case 'smoke': {
|
||||
const startTime = Date.now();
|
||||
try { await queue.ensureSchema(); }
|
||||
catch (e) {
|
||||
console.error(`SMOKE FAIL — schema init: ${e instanceof Error ? e.message : String(e)}`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const worker = new MinionWorker(engine, { queue: 'smoke', pollInterval: 100 });
|
||||
worker.register('noop', async () => ({ ok: true, at: new Date().toISOString() }));
|
||||
|
||||
const job = await queue.add('noop', {}, { queue: 'smoke', max_attempts: 1 });
|
||||
const workerPromise = worker.start();
|
||||
|
||||
const timeoutMs = 15000;
|
||||
let final: MinionJob | null = null;
|
||||
for (let elapsed = 0; elapsed < timeoutMs; elapsed += 100) {
|
||||
await new Promise(r => setTimeout(r, 100));
|
||||
final = await queue.getJob(job.id);
|
||||
if (final && ['completed', 'failed', 'dead', 'cancelled'].includes(final.status)) break;
|
||||
}
|
||||
worker.stop();
|
||||
await workerPromise;
|
||||
|
||||
const elapsedSec = ((Date.now() - startTime) / 1000).toFixed(2);
|
||||
if (final?.status === 'completed') {
|
||||
const cfg = (await import('../core/config.ts')).loadConfig();
|
||||
const engineLabel = cfg?.engine ?? 'unknown';
|
||||
console.log(`SMOKE PASS — Minions healthy in ${elapsedSec}s (engine: ${engineLabel})`);
|
||||
if (engineLabel === 'pglite') {
|
||||
console.log('Note: the `gbrain jobs work` daemon requires Postgres. PGLite');
|
||||
console.log('supports inline execution only (`submit --follow`).');
|
||||
}
|
||||
try { await queue.removeJob(job.id); } catch { /* non-fatal cleanup */ }
|
||||
process.exit(0);
|
||||
} else {
|
||||
console.error(`SMOKE FAIL — job #${job.id} status: ${final?.status ?? 'timeout'} (${elapsedSec}s elapsed)`);
|
||||
if (final?.error_text) console.error(` Error: ${final.error_text}`);
|
||||
process.exit(1);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case 'work': {
|
||||
// Check if PGLite
|
||||
const config = (await import('../core/config.ts')).loadConfig();
|
||||
if (config?.engine === 'pglite') {
|
||||
console.error('Error: Worker daemon requires Postgres. PGLite uses an exclusive file lock that blocks other processes.');
|
||||
console.error('Use --follow for inline execution: gbrain jobs submit <name> --follow');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const queueName = parseFlag(args, '--queue') ?? 'default';
|
||||
const concurrency = parseInt(parseFlag(args, '--concurrency') ?? '1', 10);
|
||||
|
||||
try { await queue.ensureSchema(); }
|
||||
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
|
||||
|
||||
const worker = new MinionWorker(engine, { queue: queueName, concurrency });
|
||||
await registerBuiltinHandlers(worker, engine);
|
||||
|
||||
console.log(`Minion worker started (queue: ${queueName}, concurrency: ${concurrency})`);
|
||||
console.log(`Registered handlers: ${worker.registeredNames.join(', ')}`);
|
||||
await worker.start();
|
||||
break;
|
||||
}
|
||||
|
||||
default:
|
||||
console.error(`Unknown subcommand: ${sub}. Run 'gbrain jobs --help' for usage.`);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Register built-in job handlers.
|
||||
*
|
||||
* Handlers call library-level Core functions (runSyncCore via performSync,
|
||||
* runExtractCore, runEmbedCore, runBacklinksCore) directly — NOT the CLI
|
||||
* wrappers. CLI wrappers call process.exit(1) on validation errors; if a
|
||||
* worker claimed a badly-formed job and ran one, the WORKER PROCESS would
|
||||
* die and every in-flight job would go stalled. Library Cores throw
|
||||
* instead, so one bad job fails one job — not the worker.
|
||||
*
|
||||
* Per the v0.11.1 plan (Codex architecture #5 — tension 3).
|
||||
*/
|
||||
export async function registerBuiltinHandlers(worker: MinionWorker, engine: BrainEngine): Promise<void> {
|
||||
worker.register('sync', async (job) => {
|
||||
const { performSync } = await import('./sync.ts');
|
||||
const repoPath = typeof job.data.repoPath === 'string' ? job.data.repoPath : undefined;
|
||||
const noPull = !!job.data.noPull;
|
||||
const noEmbed = job.data.noEmbed !== false;
|
||||
const result = await performSync(engine, { repoPath, noPull, noEmbed });
|
||||
return result;
|
||||
});
|
||||
|
||||
worker.register('embed', async (job) => {
|
||||
const { runEmbedCore } = await import('./embed.ts');
|
||||
await runEmbedCore(engine, {
|
||||
slug: typeof job.data.slug === 'string' ? job.data.slug : undefined,
|
||||
slugs: Array.isArray(job.data.slugs) ? (job.data.slugs as string[]) : undefined,
|
||||
all: !!job.data.all,
|
||||
stale: job.data.all ? false : (job.data.stale !== false),
|
||||
});
|
||||
return { embedded: true };
|
||||
});
|
||||
|
||||
worker.register('lint', async (job) => {
|
||||
const { runLintCore } = await import('./lint.ts');
|
||||
const target = typeof job.data.dir === 'string' ? job.data.dir : '.';
|
||||
const result = await runLintCore({ target, fix: !!job.data.fix, dryRun: !!job.data.dryRun });
|
||||
return result;
|
||||
});
|
||||
|
||||
worker.register('import', async (job) => {
|
||||
// import.ts Core extraction deferred to v0.12.0 (import has parallel
|
||||
// workers + checkpointing). Keep the CLI wrapper call but note the
|
||||
// worker-kill risk is bounded: import's only process.exit fires on
|
||||
// a missing dir arg, which this handler always passes.
|
||||
const { runImport } = await import('./import.ts');
|
||||
const importArgs: string[] = [];
|
||||
if (job.data.dir) importArgs.push(String(job.data.dir));
|
||||
if (job.data.noEmbed) importArgs.push('--no-embed');
|
||||
await runImport(engine, importArgs);
|
||||
return { imported: true };
|
||||
});
|
||||
|
||||
worker.register('extract', async (job) => {
|
||||
const { runExtractCore } = await import('./extract.ts');
|
||||
const mode = (typeof job.data.mode === 'string' && ['links', 'timeline', 'all'].includes(job.data.mode))
|
||||
? (job.data.mode as 'links' | 'timeline' | 'all')
|
||||
: 'all';
|
||||
const dir = typeof job.data.dir === 'string'
|
||||
? job.data.dir
|
||||
: (await engine.getConfig('sync.repo_path')) ?? '.';
|
||||
return await runExtractCore(engine, { mode, dir, dryRun: !!job.data.dryRun });
|
||||
});
|
||||
|
||||
worker.register('backlinks', async (job) => {
|
||||
const { runBacklinksCore } = await import('./backlinks.ts');
|
||||
const action: 'check' | 'fix' = job.data.action === 'check' ? 'check' : 'fix';
|
||||
const dir = typeof job.data.dir === 'string'
|
||||
? job.data.dir
|
||||
: (await engine.getConfig('sync.repo_path')) ?? '.';
|
||||
return await runBacklinksCore({ action, dir, dryRun: !!job.data.dryRun });
|
||||
});
|
||||
|
||||
// The killer handler. Autopilot submits ONE `autopilot-cycle` per cycle
|
||||
// (idempotency_key on cycle slot) instead of a 4-job parent-child DAG,
|
||||
// because Minions' parent/child is NOT a depends_on primitive (Codex
|
||||
// H3/H4). Each step is wrapped in its own try/catch; the handler returns
|
||||
// `{ partial: true, failed_steps: [...] }` when any step fails. It does
|
||||
// NOT throw on partial failure — that would cause the Minion to retry,
|
||||
// and an intermittent extract bug would block every future cycle.
|
||||
worker.register('autopilot-cycle', async (job) => {
|
||||
const { performSync } = await import('./sync.ts');
|
||||
const { runExtractCore } = await import('./extract.ts');
|
||||
const { runEmbedCore } = await import('./embed.ts');
|
||||
const { runBacklinksCore } = await import('./backlinks.ts');
|
||||
|
||||
const repoPath = typeof job.data.repoPath === 'string'
|
||||
? job.data.repoPath
|
||||
: (await engine.getConfig('sync.repo_path')) ?? '.';
|
||||
|
||||
const steps: Record<string, unknown> = {};
|
||||
const failed: string[] = [];
|
||||
|
||||
try { steps.sync = await performSync(engine, { repoPath, noEmbed: true }); }
|
||||
catch (e) { steps.sync = { error: e instanceof Error ? e.message : String(e) }; failed.push('sync'); }
|
||||
|
||||
try { steps.extract = await runExtractCore(engine, { mode: 'all', dir: repoPath }); }
|
||||
catch (e) { steps.extract = { error: e instanceof Error ? e.message : String(e) }; failed.push('extract'); }
|
||||
|
||||
try { await runEmbedCore(engine, { stale: true }); steps.embed = { embedded: true }; }
|
||||
catch (e) { steps.embed = { error: e instanceof Error ? e.message : String(e) }; failed.push('embed'); }
|
||||
|
||||
try { steps.backlinks = await runBacklinksCore({ action: 'fix', dir: repoPath }); }
|
||||
catch (e) { steps.backlinks = { error: e instanceof Error ? e.message : String(e) }; failed.push('backlinks'); }
|
||||
|
||||
if (failed.length > 0) {
|
||||
return { partial: true, failed_steps: failed, steps };
|
||||
}
|
||||
return { partial: false, steps };
|
||||
});
|
||||
}
|
||||
@@ -181,6 +181,71 @@ function collectPages(dir: string): string[] {
|
||||
return pages.sort();
|
||||
}
|
||||
|
||||
export interface LintOpts {
|
||||
target: string;
|
||||
fix?: boolean;
|
||||
dryRun?: boolean;
|
||||
}
|
||||
|
||||
export interface LintResult {
|
||||
pages_scanned: number;
|
||||
pages_with_issues: number;
|
||||
total_issues: number;
|
||||
total_fixed: number;
|
||||
dryRun: boolean;
|
||||
applied_fix: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Library-level lint. Throws on validation errors (missing target, target
|
||||
* not found); lints otherwise. Does NOT print human-readable details (the
|
||||
* CLI wrapper handles that) — returns counts so Minions handlers can
|
||||
* report structured results. Safe from the worker — no process.exit.
|
||||
*/
|
||||
export async function runLintCore(opts: LintOpts): Promise<LintResult> {
|
||||
if (!opts.target) {
|
||||
throw new Error('lint: target (dir|file.md) required');
|
||||
}
|
||||
if (!existsSync(opts.target)) {
|
||||
throw new Error(`Not found: ${opts.target}`);
|
||||
}
|
||||
|
||||
const isSingleFile = statSync(opts.target).isFile();
|
||||
const pages = isSingleFile ? [opts.target] : collectPages(opts.target);
|
||||
|
||||
let totalIssues = 0;
|
||||
let totalFixed = 0;
|
||||
let pagesWithIssues = 0;
|
||||
|
||||
for (const page of pages) {
|
||||
const content = readFileSync(page, 'utf-8');
|
||||
const issues = lintContent(content, isSingleFile ? page : relative(opts.target, page));
|
||||
if (issues.length === 0) continue;
|
||||
pagesWithIssues++;
|
||||
totalIssues += issues.length;
|
||||
|
||||
if (opts.fix && issues.some(i => i.fixable)) {
|
||||
const fixed = fixContent(content);
|
||||
if (fixed !== content) {
|
||||
const fixCount = issues.filter(i => i.fixable).length;
|
||||
totalFixed += fixCount;
|
||||
if (!opts.dryRun) {
|
||||
writeFileSync(page, fixed);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
pages_scanned: pages.length,
|
||||
pages_with_issues: pagesWithIssues,
|
||||
total_issues: totalIssues,
|
||||
total_fixed: totalFixed,
|
||||
dryRun: !!opts.dryRun,
|
||||
applied_fix: !!opts.fix,
|
||||
};
|
||||
}
|
||||
|
||||
export async function runLint(args: string[]) {
|
||||
const target = args.find(a => !a.startsWith('--'));
|
||||
const doFix = args.includes('--fix');
|
||||
@@ -198,22 +263,16 @@ export async function runLint(args: string[]) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Single file or directory
|
||||
// Single file or directory — print human detail as we go, then rely on
|
||||
// Core for the aggregate numbers at the end.
|
||||
const isSingleFile = statSync(target).isFile();
|
||||
const pages = isSingleFile ? [target] : collectPages(target);
|
||||
|
||||
let totalIssues = 0;
|
||||
let totalFixed = 0;
|
||||
let pagesWithIssues = 0;
|
||||
|
||||
for (const page of pages) {
|
||||
const content = readFileSync(page, 'utf-8');
|
||||
const relPath = isSingleFile ? page : relative(target, page);
|
||||
const issues = lintContent(content, relPath);
|
||||
|
||||
if (issues.length === 0) continue;
|
||||
pagesWithIssues++;
|
||||
totalIssues += issues.length;
|
||||
|
||||
console.log(`\n${relPath}:`);
|
||||
for (const issue of issues) {
|
||||
@@ -221,12 +280,10 @@ export async function runLint(args: string[]) {
|
||||
console.log(` L${issue.line} ${issue.rule}: ${issue.message}${fixLabel}`);
|
||||
}
|
||||
|
||||
// Auto-fix if requested
|
||||
if (doFix && issues.some(i => i.fixable)) {
|
||||
const fixed = fixContent(content);
|
||||
if (fixed !== content) {
|
||||
const fixCount = issues.filter(i => i.fixable).length;
|
||||
totalFixed += fixCount;
|
||||
if (!dryRun) {
|
||||
writeFileSync(page, fixed);
|
||||
}
|
||||
@@ -235,11 +292,13 @@ export async function runLint(args: string[]) {
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`\n${pages.length} pages scanned. ${totalIssues} issue(s) in ${pagesWithIssues} page(s).`);
|
||||
// Re-run core for the aggregate counts (cheap; re-parses contents but
|
||||
// produces canonical numbers for the summary line).
|
||||
const result = await runLintCore({ target, fix: doFix, dryRun });
|
||||
console.log(`\n${result.pages_scanned} pages scanned. ${result.total_issues} issue(s) in ${result.pages_with_issues} page(s).`);
|
||||
if (doFix) {
|
||||
console.log(`${dryRun ? '(dry run) ' : ''}${totalFixed} auto-fixed.`);
|
||||
} else if (totalIssues > 0) {
|
||||
const fixable = totalIssues; // rough estimate
|
||||
console.log(`${dryRun ? '(dry run) ' : ''}${result.total_fixed} auto-fixed.`);
|
||||
} else if (result.total_issues > 0) {
|
||||
console.log(`Run with --fix to auto-fix fixable issues.`);
|
||||
}
|
||||
}
|
||||
|
||||
42
src/commands/migrations/index.ts
Normal file
42
src/commands/migrations/index.ts
Normal file
@@ -0,0 +1,42 @@
|
||||
/**
|
||||
* TS migration registry. Compiled into the gbrain binary so migration
|
||||
* discovery works on both source installs and `bun build --compile`
|
||||
* distributions without reading `skills/migrations/*.md` from disk.
|
||||
*
|
||||
* Each migration module exports a `Migration` object. Add new migrations
|
||||
* to the `migrations` array in chronological (semver) order. The registry
|
||||
* is the runtime source of truth; the markdown file at
|
||||
* `skills/migrations/vX.Y.Z.md` remains as the host-agent instruction
|
||||
* manual (read on demand when pending-host-work.jsonl is non-empty).
|
||||
*/
|
||||
|
||||
import type { Migration } from './types.ts';
|
||||
import { v0_11_0 } from './v0_11_0.ts';
|
||||
|
||||
export const migrations: Migration[] = [
|
||||
v0_11_0,
|
||||
];
|
||||
|
||||
/** Look up a migration by exact version string. */
|
||||
export function getMigration(version: string): Migration | null {
|
||||
return migrations.find(m => m.version === version) ?? null;
|
||||
}
|
||||
|
||||
export type { Migration, FeaturePitch, OrchestratorOpts, OrchestratorResult } from './types.ts';
|
||||
|
||||
/**
|
||||
* Compare two semver strings (MAJOR.MINOR.PATCH). Returns -1 / 0 / 1.
|
||||
* Extracted from src/commands/upgrade.ts#isNewerThan for shared use across
|
||||
* the migration runner + post-upgrade pitch path.
|
||||
*/
|
||||
export function compareVersions(a: string, b: string): -1 | 0 | 1 {
|
||||
const va = a.split('.').map(n => parseInt(n, 10) || 0);
|
||||
const vb = b.split('.').map(n => parseInt(n, 10) || 0);
|
||||
for (let i = 0; i < 3; i++) {
|
||||
const da = va[i] ?? 0;
|
||||
const db = vb[i] ?? 0;
|
||||
if (da > db) return 1;
|
||||
if (da < db) return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
60
src/commands/migrations/types.ts
Normal file
60
src/commands/migrations/types.ts
Normal file
@@ -0,0 +1,60 @@
|
||||
/**
|
||||
* Shared types for the migration registry + orchestrators.
|
||||
*
|
||||
* Each migration is a module that exports a `Migration` object; the registry
|
||||
* at `./index.ts` lists them in version order. Compiled binaries ship the
|
||||
* registry directly — no filesystem walk of `skills/migrations/*.md` is
|
||||
* needed at runtime.
|
||||
*/
|
||||
|
||||
export interface FeaturePitch {
|
||||
/** One-line headline printed post-upgrade. */
|
||||
headline: string;
|
||||
/** Optional multi-line description. */
|
||||
description?: string;
|
||||
/** Optional integration recipe name printed as a follow-up. */
|
||||
recipe?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Options passed to every orchestrator. The orchestrator must be idempotent:
|
||||
* re-running after a partial run must complete missed phases without
|
||||
* duplicating side-effects.
|
||||
*/
|
||||
export interface OrchestratorOpts {
|
||||
/** Non-interactive: skip prompts, use defaults with explicit print. */
|
||||
yes: boolean;
|
||||
/** Explicit minion_mode override (bypasses the Phase C prompt). */
|
||||
mode?: 'always' | 'pain_triggered' | 'off';
|
||||
/** Dry-run: print intended actions, take no side effects. */
|
||||
dryRun: boolean;
|
||||
/** Include $PWD in host-file walk (default: $HOME/.claude + $HOME/.openclaw). */
|
||||
hostDir?: string;
|
||||
/** Skip autopilot install (Phase F). */
|
||||
noAutopilotInstall: boolean;
|
||||
}
|
||||
|
||||
export interface OrchestratorPhaseResult {
|
||||
name: string;
|
||||
status: 'complete' | 'skipped' | 'failed';
|
||||
detail?: string;
|
||||
}
|
||||
|
||||
export interface OrchestratorResult {
|
||||
version: string;
|
||||
status: 'complete' | 'partial' | 'failed';
|
||||
phases: OrchestratorPhaseResult[];
|
||||
files_rewritten?: number;
|
||||
autopilot_installed?: boolean;
|
||||
install_target?: string;
|
||||
pending_host_work?: number;
|
||||
}
|
||||
|
||||
export interface Migration {
|
||||
/** Semver string, e.g. "0.11.0". */
|
||||
version: string;
|
||||
/** Agent-readable feature pitch printed by runPostUpgrade. */
|
||||
featurePitch: FeaturePitch;
|
||||
/** Run the migration. Must be idempotent. */
|
||||
orchestrator: (opts: OrchestratorOpts) => Promise<OrchestratorResult>;
|
||||
}
|
||||
510
src/commands/migrations/v0_11_0.ts
Normal file
510
src/commands/migrations/v0_11_0.ts
Normal file
@@ -0,0 +1,510 @@
|
||||
/**
|
||||
* v0.11.0 migration orchestrator — GBrain Minions adoption.
|
||||
*
|
||||
* Phases (all idempotent; resumable from a prior status:"partial" run):
|
||||
* A. Schema — gbrain init --migrate-only (never bare init — that
|
||||
* defaults to PGLite and clobbers existing configs).
|
||||
* B. Smoke — gbrain jobs smoke. Fail loudly on non-zero.
|
||||
* C. Mode — resolve minion_mode (flag / default / TTY prompt).
|
||||
* D. Prefs — write ~/.gbrain/preferences.json.
|
||||
* E. Host — detect AGENTS.md + cron manifests. Inject the subagent-
|
||||
* routing convention marker into each AGENTS.md. Rewrite
|
||||
* cron entries for GBRAIN-BUILTIN handler names only.
|
||||
* For non-builtin handlers (host-specific, like
|
||||
* ea-inbox-sweep) emit structured TODO rows to
|
||||
* ~/.gbrain/migrations/pending-host-work.jsonl so the host
|
||||
* agent can walk through its plugin-contract work per
|
||||
* skills/migrations/v0.11.0.md.
|
||||
* F. Install — gbrain autopilot --install (env-aware).
|
||||
* G. Record — append completed.jsonl (status: complete unless any
|
||||
* pending-host-work items remain).
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync, writeFileSync, mkdirSync, appendFileSync, lstatSync, statSync, realpathSync } from 'fs';
|
||||
import { join, resolve, dirname } from 'path';
|
||||
import { execSync } from 'child_process';
|
||||
import type { Migration, OrchestratorOpts, OrchestratorResult, OrchestratorPhaseResult } from './types.ts';
|
||||
import { savePreferences, loadPreferences, appendCompletedMigration } from '../../core/preferences.ts';
|
||||
import { promptLine } from '../../core/cli-util.ts';
|
||||
import { VERSION } from '../../version.ts';
|
||||
|
||||
const BUILTIN_HANDLERS = new Set(['sync', 'embed', 'lint', 'import', 'extract', 'backlinks', 'autopilot-cycle']);
|
||||
const AGENTS_MD_MARKER = '<!-- gbrain:subagent-routing v0.11.0 -->';
|
||||
const CRON_MIGRATED_PROPERTY = '_gbrain_migrated_by';
|
||||
const MAX_HOST_FILE_BYTES = 1_000_000;
|
||||
|
||||
function home(): string { return process.env.HOME || ''; }
|
||||
function gbrainDir(): string { return join(home(), '.gbrain'); }
|
||||
function pendingHostWorkPath(): string { return join(gbrainDir(), 'migrations', 'pending-host-work.jsonl'); }
|
||||
|
||||
export interface PendingHostWorkEntry {
|
||||
type: 'cron-handler-needs-host-registration' | 'agents-md-dispatcher-needs-host-review';
|
||||
status: 'pending' | 'complete';
|
||||
detected_at: string;
|
||||
/** For cron-handler type. */
|
||||
handler?: string;
|
||||
cron_schedule?: string;
|
||||
manifest_path?: string;
|
||||
current_cmd?: string;
|
||||
/** For agents-md type. */
|
||||
file?: string;
|
||||
detected_patterns?: string[];
|
||||
recommendation: string;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Phase A — Schema
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
function phaseASchema(opts: OrchestratorOpts): OrchestratorPhaseResult {
|
||||
if (opts.dryRun) return { name: 'schema', status: 'skipped', detail: 'dry-run' };
|
||||
try {
|
||||
execSync('gbrain init --migrate-only', { stdio: 'inherit', timeout: 60_000, env: process.env });
|
||||
return { name: 'schema', status: 'complete' };
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
return { name: 'schema', status: 'failed', detail: msg };
|
||||
}
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Phase B — Smoke
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
function phaseBSmoke(opts: OrchestratorOpts): OrchestratorPhaseResult {
|
||||
if (opts.dryRun) return { name: 'smoke', status: 'skipped', detail: 'dry-run' };
|
||||
try {
|
||||
execSync('gbrain jobs smoke', { stdio: 'inherit', timeout: 30_000, env: process.env });
|
||||
return { name: 'smoke', status: 'complete' };
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
return { name: 'smoke', status: 'failed', detail: msg };
|
||||
}
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Phase C — Mode resolution
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
async function phaseCMode(opts: OrchestratorOpts): Promise<{
|
||||
phase: OrchestratorPhaseResult;
|
||||
mode: 'always' | 'pain_triggered' | 'off';
|
||||
}> {
|
||||
// Explicit flag wins.
|
||||
if (opts.mode) {
|
||||
return { phase: { name: 'mode', status: 'complete', detail: `mode=${opts.mode}` }, mode: opts.mode };
|
||||
}
|
||||
// If already set in preferences (resume from a partial run), respect it.
|
||||
const existing = loadPreferences();
|
||||
if (existing.minion_mode) {
|
||||
return { phase: { name: 'mode', status: 'complete', detail: `mode=${existing.minion_mode} (preserved)` }, mode: existing.minion_mode };
|
||||
}
|
||||
|
||||
// --yes / non-TTY: explicit pain_triggered default with a visible print.
|
||||
if (opts.yes || !process.stdin.isTTY) {
|
||||
console.log('Defaulting minion_mode=pain_triggered (non-interactive). Change with `gbrain config set minion_mode <always|off>`.');
|
||||
return { phase: { name: 'mode', status: 'complete', detail: 'mode=pain_triggered (default)' }, mode: 'pain_triggered' };
|
||||
}
|
||||
|
||||
// Interactive: numbered menu via the shared promptLine helper.
|
||||
console.log('');
|
||||
console.log('How should your agent use GBrain Minions?');
|
||||
console.log(' [1] always — route every background agent task through Minions (most durable)');
|
||||
console.log(' [2] pain_triggered — default to native subagents, switch to Minions when pain signals fire (recommended)');
|
||||
console.log(' [3] off — disable Minions; keep native subagents');
|
||||
console.log('');
|
||||
const answer = (await promptLine('Choice [2]: ')).trim() || '2';
|
||||
const mode = answer === '1' ? 'always' : answer === '3' ? 'off' : 'pain_triggered';
|
||||
return { phase: { name: 'mode', status: 'complete', detail: `mode=${mode}` }, mode };
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Phase D — Preferences
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
function phaseDPrefs(mode: 'always' | 'pain_triggered' | 'off', opts: OrchestratorOpts): OrchestratorPhaseResult {
|
||||
if (opts.dryRun) return { name: 'prefs', status: 'skipped', detail: `would write mode=${mode}` };
|
||||
try {
|
||||
savePreferences({
|
||||
minion_mode: mode,
|
||||
set_at: new Date().toISOString(),
|
||||
set_in_version: VERSION.replace(/^v/, '').trim() || '0.11.0',
|
||||
});
|
||||
return { name: 'prefs', status: 'complete' };
|
||||
} catch (e) {
|
||||
return { name: 'prefs', status: 'failed', detail: e instanceof Error ? e.message : String(e) };
|
||||
}
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Phase E — Host manifest rewrites + JSONL TODOs
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
function hostScopes(opts: OrchestratorOpts): string[] {
|
||||
const scopes = [join(home(), '.claude'), join(home(), '.openclaw')];
|
||||
if (opts.hostDir) scopes.push(resolve(opts.hostDir));
|
||||
return scopes.filter(p => existsSync(p));
|
||||
}
|
||||
|
||||
function safeReadHostFile(path: string): { content: string; skipReason?: string } {
|
||||
try {
|
||||
const stats = lstatSync(path);
|
||||
if (stats.isSymbolicLink()) {
|
||||
const resolved = realpathSync(path);
|
||||
// Skip if the symlink target escapes the scoped roots.
|
||||
const scopedRoots = [join(home(), '.claude'), join(home(), '.openclaw')];
|
||||
if (!scopedRoots.some(root => resolved.startsWith(root))) {
|
||||
return { content: '', skipReason: `symlink target outside scoped root: ${resolved}` };
|
||||
}
|
||||
}
|
||||
const fileStats = statSync(path);
|
||||
if (fileStats.size > MAX_HOST_FILE_BYTES) {
|
||||
return { content: '', skipReason: `file > ${MAX_HOST_FILE_BYTES} bytes` };
|
||||
}
|
||||
return { content: readFileSync(path, 'utf-8') };
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
if (msg.includes('EACCES') || msg.includes('permission')) {
|
||||
return { content: '', skipReason: `permission denied` };
|
||||
}
|
||||
return { content: '', skipReason: `read failed: ${msg}` };
|
||||
}
|
||||
}
|
||||
|
||||
function injectAgentsMdMarker(path: string, opts: OrchestratorOpts): { injected: boolean; skipReason?: string } {
|
||||
const { content, skipReason } = safeReadHostFile(path);
|
||||
if (skipReason) return { injected: false, skipReason };
|
||||
if (content.includes(AGENTS_MD_MARKER)) return { injected: false, skipReason: 'already has marker' };
|
||||
|
||||
if (opts.dryRun) return { injected: true, skipReason: 'dry-run' };
|
||||
|
||||
// mtime re-check immediately before write.
|
||||
const beforeMtime = statSync(path).mtimeMs;
|
||||
const stamp = new Date().toISOString().replace(/[:.]/g, '-');
|
||||
const bakPath = `${path}.bak.${stamp}`;
|
||||
try {
|
||||
writeFileSync(bakPath, content);
|
||||
const snippet = `\n\n${AGENTS_MD_MARKER}\n## Subagent routing (gbrain v0.11.0)\n\nSee \`skills/conventions/subagent-routing.md\` for the runtime routing convention.\n\`~/.gbrain/preferences.json\` controls \`minion_mode\` (always / pain_triggered / off).\n`;
|
||||
// Re-check mtime
|
||||
const nowMtime = statSync(path).mtimeMs;
|
||||
if (nowMtime !== beforeMtime) {
|
||||
return { injected: false, skipReason: 'file modified between read and write — skipping; re-run to retry' };
|
||||
}
|
||||
writeFileSync(path, content.trimEnd() + snippet);
|
||||
return { injected: true };
|
||||
} catch (e) {
|
||||
return { injected: false, skipReason: `write failed: ${e instanceof Error ? e.message : e}` };
|
||||
}
|
||||
}
|
||||
|
||||
function findAgentsMdFiles(opts: OrchestratorOpts): string[] {
|
||||
const found: string[] = [];
|
||||
for (const scope of hostScopes(opts)) {
|
||||
const candidate = join(scope, 'AGENTS.md');
|
||||
if (existsSync(candidate)) found.push(candidate);
|
||||
}
|
||||
// Also check $HOME/AGENTS.md and $PWD/AGENTS.md when --host-dir passed.
|
||||
if (opts.hostDir) {
|
||||
const c = join(resolve(opts.hostDir), 'AGENTS.md');
|
||||
if (existsSync(c) && !found.includes(c)) found.push(c);
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
function findCronManifests(opts: OrchestratorOpts): string[] {
|
||||
const found: string[] = [];
|
||||
for (const scope of hostScopes(opts)) {
|
||||
const candidates = [
|
||||
join(scope, 'cron', 'jobs.json'),
|
||||
join(scope, 'cron.json'),
|
||||
];
|
||||
for (const c of candidates) if (existsSync(c)) found.push(c);
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
function rewriteCronManifest(
|
||||
path: string,
|
||||
opts: OrchestratorOpts,
|
||||
): { rewritten: number; todos_emitted: number; skipReason?: string } {
|
||||
const { content, skipReason } = safeReadHostFile(path);
|
||||
if (skipReason) return { rewritten: 0, todos_emitted: 0, skipReason };
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(content);
|
||||
} catch (e) {
|
||||
return { rewritten: 0, todos_emitted: 0, skipReason: `malformed JSON: ${e instanceof Error ? e.message : e}` };
|
||||
}
|
||||
|
||||
const entries = Array.isArray(parsed) ? parsed : (parsed as { jobs?: unknown[] }).jobs;
|
||||
if (!Array.isArray(entries)) {
|
||||
return { rewritten: 0, todos_emitted: 0, skipReason: 'no entries array (expected Array or { jobs: [...] })' };
|
||||
}
|
||||
|
||||
const pendingEntries: PendingHostWorkEntry[] = [];
|
||||
let rewritten = 0;
|
||||
let changed = false;
|
||||
|
||||
// Detect engine for --follow branch (PGLite needs --follow because its
|
||||
// worker daemon can't run; Postgres drops --follow + uses idempotency key).
|
||||
// We load config lazily to avoid a hard dep.
|
||||
let enginePglite = false;
|
||||
try {
|
||||
const cfg = JSON.parse(readFileSync(join(gbrainDir(), 'config.json'), 'utf-8'));
|
||||
enginePglite = cfg?.engine === 'pglite';
|
||||
} catch { /* best-effort */ }
|
||||
|
||||
for (const rawEntry of entries) {
|
||||
if (!rawEntry || typeof rawEntry !== 'object') continue;
|
||||
const entry = rawEntry as Record<string, unknown>;
|
||||
if ((entry as any)[CRON_MIGRATED_PROPERTY]) continue; // idempotency
|
||||
|
||||
const kind = typeof entry.kind === 'string' ? entry.kind : undefined;
|
||||
const handler = (typeof entry.skill === 'string' ? entry.skill : undefined)
|
||||
|| (typeof entry.handler === 'string' ? entry.handler : undefined)
|
||||
|| (typeof entry.name === 'string' ? entry.name : undefined);
|
||||
const schedule = typeof entry.schedule === 'string' ? entry.schedule : (typeof entry.cron === 'string' ? entry.cron : '<unknown>');
|
||||
|
||||
if (kind !== 'agentTurn' && kind !== 'session' && kind !== 'skill') continue;
|
||||
if (!handler) continue;
|
||||
|
||||
if (BUILTIN_HANDLERS.has(handler)) {
|
||||
// Rewrite to shell + gbrain jobs submit.
|
||||
let cmd: string;
|
||||
if (enginePglite) {
|
||||
cmd = `gbrain jobs submit ${handler} --params '{}' --follow`;
|
||||
} else {
|
||||
// slot computed via date(1). Host scheduler evaluates shell.
|
||||
cmd = `gbrain jobs submit ${handler} --params '{"slot":"$(date -u +%Y-%m-%dT%H:%M)"}' --idempotency-key ${handler}:$(date -u +%Y-%m-%dT%H:%M)`;
|
||||
}
|
||||
entry.kind = 'shell';
|
||||
entry.cmd = cmd;
|
||||
(entry as any)[CRON_MIGRATED_PROPERTY] = 'v0.11.0';
|
||||
rewritten++;
|
||||
changed = true;
|
||||
} else {
|
||||
// Non-builtin handler → emit pending-host-work TODO.
|
||||
pendingEntries.push({
|
||||
type: 'cron-handler-needs-host-registration',
|
||||
handler,
|
||||
cron_schedule: schedule,
|
||||
manifest_path: path,
|
||||
current_cmd: `agentTurn ${handler}`,
|
||||
recommendation: `Add a handler registration for \`${handler}\` in your host worker bootstrap per docs/guides/plugin-handlers.md. Once registered, re-run \`gbrain apply-migrations\` to auto-rewrite this entry.`,
|
||||
detected_at: new Date().toISOString(),
|
||||
status: 'pending',
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (changed && !opts.dryRun) {
|
||||
const beforeMtime = statSync(path).mtimeMs;
|
||||
const stamp = new Date().toISOString().replace(/[:.]/g, '-');
|
||||
try {
|
||||
writeFileSync(`${path}.bak.${stamp}`, content);
|
||||
const nowMtime = statSync(path).mtimeMs;
|
||||
if (nowMtime !== beforeMtime) {
|
||||
return { rewritten: 0, todos_emitted: 0, skipReason: 'file modified mid-rewrite — skipping' };
|
||||
}
|
||||
const output = Array.isArray(parsed) ? parsed : { ...(parsed as object), jobs: entries };
|
||||
writeFileSync(path, JSON.stringify(output, null, 2) + '\n');
|
||||
} catch (e) {
|
||||
return { rewritten: 0, todos_emitted: 0, skipReason: `write failed: ${e instanceof Error ? e.message : e}` };
|
||||
}
|
||||
}
|
||||
|
||||
// Emit TODOs (deduped by handler + manifest_path).
|
||||
let todosEmitted = 0;
|
||||
if (pendingEntries.length > 0 && !opts.dryRun) {
|
||||
const existingTodos = loadPendingHostWork();
|
||||
const seen = new Set<string>(existingTodos.map(t => `${t.handler}::${t.manifest_path}`));
|
||||
for (const todo of pendingEntries) {
|
||||
const key = `${todo.handler}::${todo.manifest_path}`;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
appendPendingHostWork(todo);
|
||||
todosEmitted++;
|
||||
}
|
||||
}
|
||||
|
||||
return { rewritten, todos_emitted: todosEmitted };
|
||||
}
|
||||
|
||||
export function loadPendingHostWork(): PendingHostWorkEntry[] {
|
||||
const path = pendingHostWorkPath();
|
||||
if (!existsSync(path)) return [];
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const out: PendingHostWorkEntry[] = [];
|
||||
for (const line of raw.split('\n')) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) continue;
|
||||
try { out.push(JSON.parse(trimmed) as PendingHostWorkEntry); }
|
||||
catch { /* skip malformed line */ }
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
export function appendPendingHostWork(entry: PendingHostWorkEntry): void {
|
||||
mkdirSync(dirname(pendingHostWorkPath()), { recursive: true });
|
||||
appendFileSync(pendingHostWorkPath(), JSON.stringify(entry) + '\n');
|
||||
}
|
||||
|
||||
async function phaseEHost(opts: OrchestratorOpts): Promise<{
|
||||
phase: OrchestratorPhaseResult;
|
||||
files_rewritten: number;
|
||||
pending_host_work: number;
|
||||
}> {
|
||||
let filesTouched = 0;
|
||||
let todosEmitted = 0;
|
||||
const warnings: string[] = [];
|
||||
|
||||
// AGENTS.md marker injection.
|
||||
for (const path of findAgentsMdFiles(opts)) {
|
||||
const { injected, skipReason } = injectAgentsMdMarker(path, opts);
|
||||
if (injected) filesTouched++;
|
||||
if (skipReason && skipReason !== 'already has marker' && skipReason !== 'dry-run') {
|
||||
warnings.push(`${path}: ${skipReason}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Cron manifest rewrites.
|
||||
for (const path of findCronManifests(opts)) {
|
||||
const { rewritten, todos_emitted, skipReason } = rewriteCronManifest(path, opts);
|
||||
filesTouched += rewritten;
|
||||
todosEmitted += todos_emitted;
|
||||
if (skipReason) warnings.push(`${path}: ${skipReason}`);
|
||||
}
|
||||
|
||||
if (warnings.length > 0) {
|
||||
console.warn('[host-rewrite] warnings:');
|
||||
for (const w of warnings) console.warn(` ${w}`);
|
||||
}
|
||||
|
||||
return {
|
||||
phase: { name: 'host', status: 'complete', detail: `rewrote ${filesTouched} entries; ${todosEmitted} host-work TODOs emitted` },
|
||||
files_rewritten: filesTouched,
|
||||
pending_host_work: todosEmitted,
|
||||
};
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Phase F — Autopilot install
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
function phaseFInstall(opts: OrchestratorOpts): OrchestratorPhaseResult {
|
||||
if (opts.dryRun) return { name: 'install', status: 'skipped', detail: 'dry-run' };
|
||||
if (opts.noAutopilotInstall) return { name: 'install', status: 'skipped', detail: '--no-autopilot-install' };
|
||||
try {
|
||||
execSync('gbrain autopilot --install --yes', { stdio: 'inherit', timeout: 60_000, env: process.env });
|
||||
return { name: 'install', status: 'complete' };
|
||||
} catch (e) {
|
||||
// Install is best-effort — log but don't fail the whole migration. User
|
||||
// can re-run `gbrain autopilot --install` manually.
|
||||
return { name: 'install', status: 'failed', detail: e instanceof Error ? e.message : String(e) };
|
||||
}
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Orchestrator
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
async function orchestrator(opts: OrchestratorOpts): Promise<OrchestratorResult> {
|
||||
const phases: OrchestratorPhaseResult[] = [];
|
||||
|
||||
const a = phaseASchema(opts);
|
||||
phases.push(a);
|
||||
if (a.status === 'failed') {
|
||||
console.error(`Phase A (schema) failed: ${a.detail}. Aborting; re-run after fixing.`);
|
||||
return { version: '0.11.0', status: 'failed', phases };
|
||||
}
|
||||
|
||||
const b = phaseBSmoke(opts);
|
||||
phases.push(b);
|
||||
if (b.status === 'failed') {
|
||||
console.error(`Phase B (smoke) failed: ${b.detail}. Aborting; re-run after fixing.`);
|
||||
return { version: '0.11.0', status: 'failed', phases };
|
||||
}
|
||||
|
||||
const { phase: c, mode } = await phaseCMode(opts);
|
||||
phases.push(c);
|
||||
|
||||
const d = phaseDPrefs(mode, opts);
|
||||
phases.push(d);
|
||||
if (d.status === 'failed') {
|
||||
console.error(`Phase D (prefs) failed: ${d.detail}.`);
|
||||
return { version: '0.11.0', status: 'failed', phases };
|
||||
}
|
||||
|
||||
const { phase: e, files_rewritten, pending_host_work } = await phaseEHost(opts);
|
||||
phases.push(e);
|
||||
|
||||
const f = phaseFInstall(opts);
|
||||
phases.push(f);
|
||||
|
||||
// Phase G: record in completed.jsonl. Status depends on whether any
|
||||
// host work remains pending AND whether the install phase succeeded.
|
||||
const status: 'complete' | 'partial' = (pending_host_work > 0) ? 'partial' : 'complete';
|
||||
|
||||
if (!opts.dryRun) {
|
||||
appendCompletedMigration({
|
||||
version: '0.11.0',
|
||||
status,
|
||||
mode,
|
||||
files_rewritten,
|
||||
autopilot_installed: f.status === 'complete',
|
||||
install_target: undefined, // install target is decided inside autopilot --install
|
||||
...(status === 'partial' ? { apply_migrations_pending: true } : {}),
|
||||
});
|
||||
}
|
||||
phases.push({ name: 'record', status: opts.dryRun ? 'skipped' : 'complete', detail: `status=${status}` });
|
||||
|
||||
// Post-run: print pending-host-work summary if anything needs host action.
|
||||
if (pending_host_work > 0) {
|
||||
console.log('');
|
||||
console.log(`${pending_host_work} host-specific item(s) need your agent's attention before the Minions migration is complete.`);
|
||||
console.log('');
|
||||
console.log('Next: run your host agent and have it read:');
|
||||
console.log(` ${pendingHostWorkPath()}`);
|
||||
console.log(` skills/migrations/v0.11.0.md`);
|
||||
console.log('');
|
||||
console.log('The skill walks the host through each item using GBrain\'s plugin contract.');
|
||||
console.log('Re-run `gbrain apply-migrations --yes` after each batch to auto-rewrite newly-');
|
||||
console.log('registerable crons and mark items done.');
|
||||
}
|
||||
|
||||
return {
|
||||
version: '0.11.0',
|
||||
status,
|
||||
phases,
|
||||
files_rewritten,
|
||||
autopilot_installed: f.status === 'complete',
|
||||
pending_host_work,
|
||||
};
|
||||
}
|
||||
|
||||
export const v0_11_0: Migration = {
|
||||
version: '0.11.0',
|
||||
featurePitch: {
|
||||
headline: 'GBrain Minions — durable background agents',
|
||||
description:
|
||||
'Turn any long-running agent task into a durable job that survives gateway ' +
|
||||
'restarts, streams progress, and can be paused, resumed, or steered mid-flight. ' +
|
||||
'Postgres-native, zero infra beyond your existing brain. Replaces flaky ' +
|
||||
'subagent spawns for multi-step work, parallel fan-out, and anything the ' +
|
||||
'user might ask about later.',
|
||||
},
|
||||
orchestrator,
|
||||
};
|
||||
|
||||
/** Exported for unit tests. */
|
||||
export const __testing = {
|
||||
injectAgentsMdMarker,
|
||||
rewriteCronManifest,
|
||||
phaseEHost,
|
||||
findAgentsMdFiles,
|
||||
findCronManifests,
|
||||
BUILTIN_HANDLERS,
|
||||
AGENTS_MD_MARKER,
|
||||
loadPendingHostWork,
|
||||
pendingHostWorkPath,
|
||||
};
|
||||
225
src/commands/skillpack-check.ts
Normal file
225
src/commands/skillpack-check.ts
Normal file
@@ -0,0 +1,225 @@
|
||||
/**
|
||||
* `gbrain skillpack-check` — agent-readable health report.
|
||||
*
|
||||
* Wraps `gbrain doctor --json` + `gbrain apply-migrations --list` into a
|
||||
* single JSON blob a host agent (Wintermute's morning-briefing, any
|
||||
* OpenClaw cron) can consume without parsing two subcommands.
|
||||
*
|
||||
* Usage:
|
||||
* gbrain skillpack-check # pretty-printed JSON + exit code
|
||||
* gbrain skillpack-check --quiet # only exits with status; no output
|
||||
* gbrain skillpack-check --help
|
||||
*
|
||||
* Exit codes:
|
||||
* 0 — Healthy. Nothing needs action.
|
||||
* 1 — Action needed (partial migration, half-install, or doctor FAIL).
|
||||
* 2 — Could not determine (missing binary / crashed).
|
||||
*/
|
||||
|
||||
import { execFileSync } from 'child_process';
|
||||
import { VERSION } from '../version.ts';
|
||||
|
||||
/**
|
||||
* Resolve the gbrain binary + args for spawning subcommands from
|
||||
* within skillpack-check. Handles three install cases:
|
||||
* - Running the compiled binary (argv[1] ends in /gbrain): re-exec it.
|
||||
* - Running via `bun run src/cli.ts` (argv[1] is a .ts file): prefix with `bun run`.
|
||||
* - Anything else: fall back to `which gbrain` on $PATH.
|
||||
*/
|
||||
function gbrainSpawn(): { cmd: string; prefix: string[] } {
|
||||
const arg1 = process.argv[1] ?? '';
|
||||
if (arg1.endsWith('/gbrain') || arg1.endsWith('\\gbrain.exe')) {
|
||||
return { cmd: arg1, prefix: [] };
|
||||
}
|
||||
if (arg1.endsWith('.ts') || arg1.endsWith('.mjs') || arg1.endsWith('.js')) {
|
||||
return { cmd: 'bun', prefix: ['run', arg1] };
|
||||
}
|
||||
const execPath = process.execPath ?? '';
|
||||
if (execPath.endsWith('/gbrain') || execPath.endsWith('\\gbrain.exe')) {
|
||||
return { cmd: execPath, prefix: [] };
|
||||
}
|
||||
return { cmd: 'gbrain', prefix: [] };
|
||||
}
|
||||
|
||||
interface DoctorCheck {
|
||||
name: string;
|
||||
status: 'ok' | 'warn' | 'fail';
|
||||
message: string;
|
||||
issues?: unknown[];
|
||||
}
|
||||
|
||||
interface SkillpackReport {
|
||||
version: string;
|
||||
ts: string;
|
||||
healthy: boolean;
|
||||
/** One-line summary for an agent to quote in a briefing. */
|
||||
summary: string;
|
||||
/** Every recommended action the user/agent should take. */
|
||||
actions: string[];
|
||||
/** Full doctor output, machine-readable. */
|
||||
doctor: {
|
||||
exit_code: number;
|
||||
checks: DoctorCheck[];
|
||||
} | { error: string };
|
||||
/** apply-migrations --list output, parsed. */
|
||||
migrations: {
|
||||
pending_count: number;
|
||||
partial_count: number;
|
||||
applied_count: number;
|
||||
stdout: string;
|
||||
} | { error: string };
|
||||
}
|
||||
|
||||
function runDoctor(): SkillpackReport['doctor'] {
|
||||
const { cmd, prefix } = gbrainSpawn();
|
||||
try {
|
||||
// --fast avoids DB dependency; the filesystem half-migration checks
|
||||
// we care about most run in the fast path.
|
||||
const stdout = execFileSync(cmd, [...prefix, 'doctor', '--fast', '--json'], {
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
env: process.env,
|
||||
});
|
||||
// doctor emits a JSON object on success; on FAIL it exits non-zero
|
||||
// but still prints JSON. Parse either way.
|
||||
const parsed = JSON.parse(stdout) as { checks: DoctorCheck[] };
|
||||
return { exit_code: 0, checks: parsed.checks };
|
||||
} catch (err: any) {
|
||||
// execFileSync throws on non-zero exit; stdout is still on the error.
|
||||
const stdout = err.stdout?.toString?.() ?? '';
|
||||
try {
|
||||
const parsed = JSON.parse(stdout) as { checks: DoctorCheck[] };
|
||||
return { exit_code: err.status ?? 1, checks: parsed.checks };
|
||||
} catch {
|
||||
return { error: `doctor failed: ${err.message ?? String(err)}` };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function runMigrationsList(): SkillpackReport['migrations'] {
|
||||
const { cmd, prefix } = gbrainSpawn();
|
||||
try {
|
||||
const stdout = execFileSync(cmd, [...prefix, 'apply-migrations', '--list'], {
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
env: process.env,
|
||||
});
|
||||
|
||||
// Count rows by status word. Output shape from apply-migrations:
|
||||
// Installed gbrain version: 0.11.1
|
||||
//
|
||||
// Status Version Headline
|
||||
// ------- -------- ...
|
||||
// applied 0.11.0 ...
|
||||
// pending 0.11.1 ...
|
||||
// partial 0.10.0 ...
|
||||
const lines = stdout.split('\n');
|
||||
let applied = 0;
|
||||
let pending = 0;
|
||||
let partial = 0;
|
||||
for (const line of lines) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed || trimmed.startsWith('Status') || trimmed.startsWith('---')) continue;
|
||||
const first = trimmed.split(/\s+/)[0];
|
||||
if (first === 'applied') applied++;
|
||||
else if (first === 'pending') pending++;
|
||||
else if (first === 'partial') partial++;
|
||||
}
|
||||
return { applied_count: applied, pending_count: pending, partial_count: partial, stdout };
|
||||
} catch (err: any) {
|
||||
return { error: `apply-migrations --list failed: ${err.message ?? String(err)}` };
|
||||
}
|
||||
}
|
||||
|
||||
function buildReport(): SkillpackReport {
|
||||
const doctor = runDoctor();
|
||||
const migrations = runMigrationsList();
|
||||
|
||||
const actions: string[] = [];
|
||||
let healthy = true;
|
||||
|
||||
// Gather actions from doctor failures.
|
||||
if ('checks' in doctor) {
|
||||
for (const check of doctor.checks) {
|
||||
if (check.status === 'fail') {
|
||||
healthy = false;
|
||||
// Extract remediation command from check message if it follows
|
||||
// the `... Run: <cmd>` convention. Otherwise include the whole
|
||||
// message so the agent has enough to reason.
|
||||
const runMatch = check.message.match(/Run:\s*(.+)$/);
|
||||
if (runMatch) actions.push(runMatch[1].trim());
|
||||
else actions.push(`[${check.name}] ${check.message}`);
|
||||
} else if (check.status === 'warn') {
|
||||
// Warnings don't fail the report but surface as informational
|
||||
// actions the agent can decide about.
|
||||
const runMatch = check.message.match(/Run:\s*(.+)$/);
|
||||
if (runMatch && !actions.includes(runMatch[1].trim())) actions.push(runMatch[1].trim());
|
||||
}
|
||||
}
|
||||
} else {
|
||||
healthy = false;
|
||||
actions.push('Investigate doctor failure: ' + doctor.error);
|
||||
}
|
||||
|
||||
// Gather actions from pending/partial migrations.
|
||||
if ('applied_count' in migrations) {
|
||||
if (migrations.partial_count > 0 || migrations.pending_count > 0) {
|
||||
healthy = false;
|
||||
const action = 'gbrain apply-migrations --yes';
|
||||
if (!actions.includes(action)) actions.unshift(action);
|
||||
}
|
||||
} else {
|
||||
healthy = false;
|
||||
actions.push('Investigate apply-migrations failure: ' + migrations.error);
|
||||
}
|
||||
|
||||
const summary = healthy
|
||||
? 'gbrain skillpack healthy'
|
||||
: `gbrain skillpack needs attention: ${actions.length} action(s) — ${actions[0]}`;
|
||||
|
||||
return {
|
||||
version: VERSION,
|
||||
ts: new Date().toISOString(),
|
||||
healthy,
|
||||
summary,
|
||||
actions,
|
||||
doctor,
|
||||
migrations,
|
||||
};
|
||||
}
|
||||
|
||||
export async function runSkillpackCheck(args: string[]): Promise<void> {
|
||||
if (args.includes('--help') || args.includes('-h')) {
|
||||
console.log(`gbrain skillpack-check — agent-readable health report.
|
||||
|
||||
Wraps doctor + apply-migrations --list into one JSON blob. Cron-friendly:
|
||||
zero interactive prompts, non-zero exit on any needed action.
|
||||
|
||||
Usage:
|
||||
gbrain skillpack-check Pretty JSON to stdout, exit 0/1/2.
|
||||
gbrain skillpack-check --quiet Exit code only, no output.
|
||||
|
||||
Exit codes:
|
||||
0 healthy (no action needed)
|
||||
1 action needed (see JSON.actions[])
|
||||
2 could not determine (binary or subcommand crash)
|
||||
`);
|
||||
return;
|
||||
}
|
||||
|
||||
const quiet = args.includes('--quiet');
|
||||
const report = buildReport();
|
||||
|
||||
if (!quiet) {
|
||||
console.log(JSON.stringify(report, null, 2));
|
||||
}
|
||||
|
||||
// Determine exit code.
|
||||
if ('error' in report.doctor || 'error' in report.migrations) {
|
||||
process.exit(2);
|
||||
}
|
||||
process.exit(report.healthy ? 0 : 1);
|
||||
}
|
||||
|
||||
/** Exported for unit tests. */
|
||||
export const __testing = { buildReport, runDoctor, runMigrationsList };
|
||||
@@ -1,6 +1,6 @@
|
||||
import { execSync } from 'child_process';
|
||||
import { existsSync, readFileSync, readdirSync, writeFileSync, mkdirSync } from 'fs';
|
||||
import { join, resolve } from 'path';
|
||||
import { existsSync, readFileSync, writeFileSync, mkdirSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { VERSION } from '../version.ts';
|
||||
|
||||
export async function runUpgrade(args: string[]) {
|
||||
@@ -55,9 +55,12 @@ export async function runUpgrade(args: string[]) {
|
||||
const newVersion = verifyUpgrade();
|
||||
// Save old version for post-upgrade migration detection
|
||||
saveUpgradeState(oldVersion, newVersion);
|
||||
// Run post-upgrade feature discovery (reads migration files from the NEW binary)
|
||||
// Run post-upgrade feature discovery (reads migration files from the NEW binary).
|
||||
// Timeout bumped 30s → 300s because runPostUpgrade now tail-calls
|
||||
// apply-migrations, which can do long work (schema, smoke, host-rewrite,
|
||||
// autopilot install) on a v0.11.0→v0.11.1 jump. Codex H7.
|
||||
try {
|
||||
execSync('gbrain post-upgrade', { stdio: 'inherit', timeout: 30_000 });
|
||||
execSync('gbrain post-upgrade', { stdio: 'inherit', timeout: 300_000 });
|
||||
} catch {
|
||||
// post-upgrade is best-effort, don't fail the upgrade
|
||||
}
|
||||
@@ -101,78 +104,71 @@ function saveUpgradeState(oldVersion: string, newVersion: string) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Post-upgrade feature discovery. Reads migration files between old and new version,
|
||||
* prints feature pitches from YAML frontmatter. Called by `gbrain post-upgrade` which
|
||||
* runs the NEW binary after upgrade completes.
|
||||
* Post-upgrade feature discovery + migration application.
|
||||
*
|
||||
* Two responsibilities:
|
||||
* 1. Print feature_pitch headlines for migrations newer than the prior
|
||||
* binary (cosmetic; runs only when upgrade-state.json is readable and
|
||||
* has a from/to pair).
|
||||
* 2. Invoke `gbrain apply-migrations --yes` so the mechanical side of
|
||||
* every outstanding migration actually executes (schema, smoke, prefs,
|
||||
* host rewrites, autopilot install). This is the Codex H8 fix:
|
||||
* previously runPostUpgrade early-returned when upgrade-state.json
|
||||
* was missing, which meant every broken-v0.11.0 install stayed broken.
|
||||
* apply-migrations now runs unconditionally (idempotent; cheap when
|
||||
* nothing is pending).
|
||||
*
|
||||
* Migration enumeration uses the TS registry at
|
||||
* src/commands/migrations/index.ts (Codex K) — no filesystem walk of
|
||||
* skills/migrations/*.md, so compiled binaries see the same set source
|
||||
* installs do.
|
||||
*/
|
||||
export function runPostUpgrade() {
|
||||
export async function runPostUpgrade(): Promise<void> {
|
||||
// Cosmetic: print feature pitches for migrations newer than the prior binary.
|
||||
try {
|
||||
const statePath = join(process.env.HOME || '', '.gbrain', 'upgrade-state.json');
|
||||
if (!existsSync(statePath)) return;
|
||||
const state = JSON.parse(readFileSync(statePath, 'utf-8'));
|
||||
const lastUpgrade = state.last_upgrade;
|
||||
if (!lastUpgrade?.from || !lastUpgrade?.to) return;
|
||||
|
||||
// Find migration files in version range
|
||||
const migrationsDir = findMigrationsDir();
|
||||
if (!migrationsDir) return;
|
||||
|
||||
const files = readdirSync(migrationsDir)
|
||||
.filter(f => f.match(/^v\d+\.\d+\.\d+\.md$/))
|
||||
.sort();
|
||||
|
||||
for (const file of files) {
|
||||
const version = file.replace(/^v/, '').replace(/\.md$/, '');
|
||||
if (isNewerThan(version, lastUpgrade.from)) {
|
||||
const content = readFileSync(join(migrationsDir, file), 'utf-8');
|
||||
const pitch = extractFeaturePitch(content);
|
||||
if (pitch) {
|
||||
console.log('');
|
||||
console.log(`NEW: ${pitch.headline}`);
|
||||
if (pitch.description) console.log(pitch.description);
|
||||
if (pitch.recipe) {
|
||||
console.log(`Run \`gbrain integrations show ${pitch.recipe}\` to set it up.`);
|
||||
if (existsSync(statePath)) {
|
||||
const state = JSON.parse(readFileSync(statePath, 'utf-8'));
|
||||
const from = state?.last_upgrade?.from;
|
||||
if (from) {
|
||||
const { migrations } = await import('./migrations/index.ts');
|
||||
for (const m of migrations) {
|
||||
if (isNewerThan(m.version, from)) {
|
||||
console.log('');
|
||||
console.log(`NEW: ${m.featurePitch.headline}`);
|
||||
if (m.featurePitch.description) console.log(m.featurePitch.description);
|
||||
if (m.featurePitch.recipe) {
|
||||
console.log(`Run \`gbrain integrations show ${m.featurePitch.recipe}\` to set it up.`);
|
||||
}
|
||||
console.log('');
|
||||
}
|
||||
console.log('');
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// post-upgrade is best-effort
|
||||
// Pitch printing is cosmetic — don't gate migrations on it.
|
||||
}
|
||||
|
||||
// Mechanical: run every outstanding migration. Idempotent; exits 0 quickly
|
||||
// when nothing is pending. Stays inside the same process so a long Phase F
|
||||
// (autopilot install) doesn't hit a subprocess boundary.
|
||||
try {
|
||||
const { runApplyMigrations } = await import('./apply-migrations.ts');
|
||||
await runApplyMigrations(['--yes', '--non-interactive']);
|
||||
} catch (e) {
|
||||
// Surface the error but don't throw — post-upgrade is best-effort.
|
||||
// Users can re-run `gbrain apply-migrations` manually if they want
|
||||
// to retry.
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
console.error(`\napply-migrations failed: ${msg}`);
|
||||
console.error('Run `gbrain apply-migrations --yes` manually to retry.');
|
||||
}
|
||||
}
|
||||
|
||||
function findMigrationsDir(): string | null {
|
||||
// Try relative to this file (source install)
|
||||
const candidates = [
|
||||
resolve(__dirname, '../../skills/migrations'),
|
||||
resolve(process.cwd(), 'skills/migrations'),
|
||||
resolve(process.cwd(), 'node_modules/gbrain/skills/migrations'),
|
||||
];
|
||||
for (const dir of candidates) {
|
||||
if (existsSync(dir)) return dir;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function extractFeaturePitch(content: string): { headline: string; description?: string; recipe?: string } | null {
|
||||
// Parse YAML frontmatter for feature_pitch
|
||||
const fmMatch = content.match(/^---\n([\s\S]*?)\n---/);
|
||||
if (!fmMatch) return null;
|
||||
const fm = fmMatch[1];
|
||||
|
||||
const headlineMatch = fm.match(/headline:\s*["']?(.+?)["']?\s*$/m);
|
||||
if (!headlineMatch) return null;
|
||||
|
||||
const descMatch = fm.match(/description:\s*["']?(.+?)["']?\s*$/m);
|
||||
const recipeMatch = fm.match(/recipe:\s*["']?(.+?)["']?\s*$/m);
|
||||
|
||||
return {
|
||||
headline: headlineMatch[1],
|
||||
description: descMatch?.[1],
|
||||
recipe: recipeMatch?.[1],
|
||||
};
|
||||
}
|
||||
// findMigrationsDir + extractFeaturePitch removed in v0.11.1: migration data
|
||||
// now lives in the TS registry at src/commands/migrations/index.ts so
|
||||
// compiled binaries don't depend on filesystem skills/migrations/*.md
|
||||
// (Codex K).
|
||||
|
||||
function isNewerThan(version: string, baseline: string): boolean {
|
||||
const v = version.split('.').map(Number);
|
||||
|
||||
16
src/core/cli-util.ts
Normal file
16
src/core/cli-util.ts
Normal file
@@ -0,0 +1,16 @@
|
||||
/**
|
||||
* Prompt on stdout, read one line from stdin, return trimmed string.
|
||||
* Shared helper used by interactive CLI flows (init, apply-migrations, etc.).
|
||||
*/
|
||||
export function promptLine(prompt: string): Promise<string> {
|
||||
return new Promise((resolve) => {
|
||||
process.stdout.write(prompt);
|
||||
process.stdin.setEncoding('utf-8');
|
||||
process.stdin.once('data', (chunk) => {
|
||||
const data = chunk.toString().trim();
|
||||
process.stdin.pause();
|
||||
resolve(data);
|
||||
});
|
||||
process.stdin.resume();
|
||||
});
|
||||
}
|
||||
@@ -89,4 +89,7 @@ export interface BrainEngine {
|
||||
// Migration support
|
||||
runMigration(version: number, sql: string): Promise<void>;
|
||||
getChunksWithEmbeddings(slug: string): Promise<Chunk[]>;
|
||||
|
||||
// Raw SQL (for Minions job queue and other internal modules)
|
||||
executeRaw<T = Record<string, unknown>>(sql: string, params?: unknown[]): Promise<T[]>;
|
||||
}
|
||||
|
||||
@@ -82,6 +82,143 @@ const MIGRATIONS: Migration[] = [
|
||||
);
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 5,
|
||||
name: 'minion_jobs_table',
|
||||
sql: `
|
||||
CREATE TABLE IF NOT EXISTS minion_jobs (
|
||||
id SERIAL PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
queue TEXT NOT NULL DEFAULT 'default',
|
||||
status TEXT NOT NULL DEFAULT 'waiting',
|
||||
priority INTEGER NOT NULL DEFAULT 0,
|
||||
data JSONB NOT NULL DEFAULT '{}',
|
||||
max_attempts INTEGER NOT NULL DEFAULT 3,
|
||||
attempts_made INTEGER NOT NULL DEFAULT 0,
|
||||
attempts_started INTEGER NOT NULL DEFAULT 0,
|
||||
backoff_type TEXT NOT NULL DEFAULT 'exponential',
|
||||
backoff_delay INTEGER NOT NULL DEFAULT 1000,
|
||||
backoff_jitter REAL NOT NULL DEFAULT 0.2,
|
||||
stalled_counter INTEGER NOT NULL DEFAULT 0,
|
||||
max_stalled INTEGER NOT NULL DEFAULT 1,
|
||||
lock_token TEXT,
|
||||
lock_until TIMESTAMPTZ,
|
||||
delay_until TIMESTAMPTZ,
|
||||
parent_job_id INTEGER REFERENCES minion_jobs(id) ON DELETE SET NULL,
|
||||
on_child_fail TEXT NOT NULL DEFAULT 'fail_parent',
|
||||
result JSONB,
|
||||
progress JSONB,
|
||||
error_text TEXT,
|
||||
stacktrace JSONB DEFAULT '[]',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
started_at TIMESTAMPTZ,
|
||||
finished_at TIMESTAMPTZ,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT chk_status CHECK (status IN ('waiting','active','completed','failed','delayed','dead','cancelled','waiting-children')),
|
||||
CONSTRAINT chk_backoff_type CHECK (backoff_type IN ('fixed','exponential')),
|
||||
CONSTRAINT chk_on_child_fail CHECK (on_child_fail IN ('fail_parent','remove_dep','ignore','continue')),
|
||||
CONSTRAINT chk_jitter_range CHECK (backoff_jitter >= 0.0 AND backoff_jitter <= 1.0),
|
||||
CONSTRAINT chk_attempts_order CHECK (attempts_made <= attempts_started),
|
||||
CONSTRAINT chk_nonnegative CHECK (attempts_made >= 0 AND attempts_started >= 0 AND stalled_counter >= 0 AND max_attempts >= 1 AND max_stalled >= 0)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_claim ON minion_jobs (queue, priority ASC, created_at ASC) WHERE status = 'waiting';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_status ON minion_jobs(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_stalled ON minion_jobs (lock_until) WHERE status = 'active';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_delayed ON minion_jobs (delay_until) WHERE status = 'delayed';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_parent ON minion_jobs(parent_job_id);
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 6,
|
||||
name: 'agent_orchestration_primitives',
|
||||
sql: `
|
||||
-- Token accounting columns
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS tokens_input INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS tokens_output INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS tokens_cache_read INTEGER NOT NULL DEFAULT 0;
|
||||
|
||||
-- Update status constraint to include 'paused'
|
||||
ALTER TABLE minion_jobs DROP CONSTRAINT IF EXISTS chk_status;
|
||||
ALTER TABLE minion_jobs ADD CONSTRAINT chk_status
|
||||
CHECK (status IN ('waiting','active','completed','failed','delayed','dead','cancelled','waiting-children','paused'));
|
||||
|
||||
-- Inbox table (separate from job row for clean concurrency)
|
||||
CREATE TABLE IF NOT EXISTS minion_inbox (
|
||||
id SERIAL PRIMARY KEY,
|
||||
job_id INTEGER NOT NULL REFERENCES minion_jobs(id) ON DELETE CASCADE,
|
||||
sender TEXT NOT NULL,
|
||||
payload JSONB NOT NULL,
|
||||
sent_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
read_at TIMESTAMPTZ
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_inbox_unread ON minion_inbox (job_id) WHERE read_at IS NULL;
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 7,
|
||||
name: 'agent_parity_layer',
|
||||
sql: `
|
||||
-- Subagent primitives + BullMQ parity columns
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS depth INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS max_children INTEGER;
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS timeout_ms INTEGER;
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS timeout_at TIMESTAMPTZ;
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS remove_on_complete BOOLEAN NOT NULL DEFAULT FALSE;
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS remove_on_fail BOOLEAN NOT NULL DEFAULT FALSE;
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS idempotency_key TEXT;
|
||||
|
||||
-- Tighten constraints (drop-then-add for idempotency)
|
||||
ALTER TABLE minion_jobs DROP CONSTRAINT IF EXISTS chk_depth_nonnegative;
|
||||
ALTER TABLE minion_jobs ADD CONSTRAINT chk_depth_nonnegative CHECK (depth >= 0);
|
||||
ALTER TABLE minion_jobs DROP CONSTRAINT IF EXISTS chk_max_children_positive;
|
||||
ALTER TABLE minion_jobs ADD CONSTRAINT chk_max_children_positive CHECK (max_children IS NULL OR max_children > 0);
|
||||
ALTER TABLE minion_jobs DROP CONSTRAINT IF EXISTS chk_timeout_positive;
|
||||
ALTER TABLE minion_jobs ADD CONSTRAINT chk_timeout_positive CHECK (timeout_ms IS NULL OR timeout_ms > 0);
|
||||
|
||||
-- Bounded scan for handleTimeouts
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_timeout ON minion_jobs (timeout_at)
|
||||
WHERE status = 'active' AND timeout_at IS NOT NULL;
|
||||
|
||||
-- O(children) child-count check in add()
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_parent_status ON minion_jobs (parent_job_id, status)
|
||||
WHERE parent_job_id IS NOT NULL;
|
||||
|
||||
-- Idempotency: enforce "only one job per key" at the DB layer
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS uniq_minion_jobs_idempotency ON minion_jobs (idempotency_key)
|
||||
WHERE idempotency_key IS NOT NULL;
|
||||
|
||||
-- Fast lookup of child_done messages for readChildCompletions
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_inbox_child_done ON minion_inbox (job_id, sent_at)
|
||||
WHERE (payload->>'type') = 'child_done';
|
||||
|
||||
-- Attachment manifest (BYTEA inline + forward-compat storage_uri)
|
||||
CREATE TABLE IF NOT EXISTS minion_attachments (
|
||||
id SERIAL PRIMARY KEY,
|
||||
job_id INTEGER NOT NULL REFERENCES minion_jobs(id) ON DELETE CASCADE,
|
||||
filename TEXT NOT NULL,
|
||||
content_type TEXT NOT NULL,
|
||||
content BYTEA,
|
||||
storage_uri TEXT,
|
||||
size_bytes INTEGER NOT NULL,
|
||||
sha256 TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT uniq_minion_attachments_job_filename UNIQUE (job_id, filename),
|
||||
CONSTRAINT chk_attachment_storage CHECK (content IS NOT NULL OR storage_uri IS NOT NULL),
|
||||
CONSTRAINT chk_attachment_size CHECK (size_bytes >= 0)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_attachments_job ON minion_attachments (job_id);
|
||||
|
||||
-- TOAST tuning: store attachment bytes out-of-line, skip compression.
|
||||
-- Attachments are usually already-compressed formats; compression burns CPU for no win.
|
||||
DO $$
|
||||
BEGIN
|
||||
ALTER TABLE minion_attachments ALTER COLUMN content SET STORAGE EXTERNAL;
|
||||
EXCEPTION WHEN OTHERS THEN
|
||||
-- PGLite may not support SET STORAGE EXTERNAL. Storage tuning is an optimization, not correctness.
|
||||
NULL;
|
||||
END $$;
|
||||
`,
|
||||
},
|
||||
];
|
||||
|
||||
export const LATEST_VERSION = MIGRATIONS.length > 0
|
||||
|
||||
99
src/core/minions/attachments.ts
Normal file
99
src/core/minions/attachments.ts
Normal file
@@ -0,0 +1,99 @@
|
||||
/**
|
||||
* Attachment validation for Minions.
|
||||
*
|
||||
* Decoupled from queue.ts so it can be unit-tested without a DB.
|
||||
* Pure function: takes input + opts, returns ok-or-error.
|
||||
*
|
||||
* The DB UNIQUE (job_id, filename) constraint is the authoritative duplicate
|
||||
* fence; the in-memory `existingFilenames` check just gives a faster, clearer
|
||||
* error before the round-trip.
|
||||
*/
|
||||
|
||||
import { createHash } from 'node:crypto';
|
||||
import type { AttachmentInput } from './types.ts';
|
||||
|
||||
export interface AttachmentValidationOpts {
|
||||
maxBytes: number;
|
||||
existingFilenames?: Set<string>;
|
||||
}
|
||||
|
||||
export interface NormalizedAttachment {
|
||||
filename: string;
|
||||
content_type: string;
|
||||
bytes: Buffer;
|
||||
size_bytes: number;
|
||||
sha256: string;
|
||||
}
|
||||
|
||||
export type ValidationResult =
|
||||
| { ok: true; normalized: NormalizedAttachment }
|
||||
| { ok: false; error: string };
|
||||
|
||||
const BASE64_RE = /^[A-Za-z0-9+/]*={0,2}$/;
|
||||
const CONTENT_TYPE_RE = /^[A-Za-z0-9!#$&^_.+\-]+\/[A-Za-z0-9!#$&^_.+\-]+(;\s*[A-Za-z0-9!#$&^_.+\-]+=[A-Za-z0-9!#$&^_.+\-"]+)*$/;
|
||||
|
||||
export function validateAttachment(input: AttachmentInput, opts: AttachmentValidationOpts): ValidationResult {
|
||||
if (!input.filename || input.filename.trim() === '') {
|
||||
return { ok: false, error: 'filename is required' };
|
||||
}
|
||||
const filename = input.filename;
|
||||
|
||||
// Reject path traversal, separators, null bytes. Filenames are leaves only.
|
||||
if (
|
||||
filename.includes('/') ||
|
||||
filename.includes('\\') ||
|
||||
filename.includes('..') ||
|
||||
filename.includes('\0')
|
||||
) {
|
||||
return { ok: false, error: `filename contains invalid characters: ${JSON.stringify(filename)}` };
|
||||
}
|
||||
|
||||
if (!input.content_type || !CONTENT_TYPE_RE.test(input.content_type)) {
|
||||
return { ok: false, error: 'content_type missing or malformed' };
|
||||
}
|
||||
|
||||
if (input.content_base64 == null || input.content_base64 === '') {
|
||||
return { ok: false, error: 'content_base64 is empty' };
|
||||
}
|
||||
|
||||
// Strict base64: only A-Z a-z 0-9 + / and trailing =. Reject whitespace and
|
||||
// line breaks so callers normalize before sending (no silent corruption).
|
||||
if (!BASE64_RE.test(input.content_base64)) {
|
||||
return { ok: false, error: 'content_base64 contains invalid characters' };
|
||||
}
|
||||
|
||||
let bytes: Buffer;
|
||||
try {
|
||||
bytes = Buffer.from(input.content_base64, 'base64');
|
||||
} catch (e) {
|
||||
return { ok: false, error: `base64 decode failed: ${(e as Error).message}` };
|
||||
}
|
||||
|
||||
if (bytes.length === 0) {
|
||||
return { ok: false, error: 'attachment content is empty after base64 decode' };
|
||||
}
|
||||
|
||||
if (bytes.length > opts.maxBytes) {
|
||||
return {
|
||||
ok: false,
|
||||
error: `attachment size ${bytes.length} exceeds maxBytes ${opts.maxBytes}`,
|
||||
};
|
||||
}
|
||||
|
||||
if (opts.existingFilenames?.has(filename)) {
|
||||
return { ok: false, error: `filename already exists for this job: ${filename}` };
|
||||
}
|
||||
|
||||
const sha256 = createHash('sha256').update(bytes).digest('hex');
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
normalized: {
|
||||
filename,
|
||||
content_type: input.content_type,
|
||||
bytes,
|
||||
size_bytes: bytes.length,
|
||||
sha256,
|
||||
},
|
||||
};
|
||||
}
|
||||
26
src/core/minions/backoff.ts
Normal file
26
src/core/minions/backoff.ts
Normal file
@@ -0,0 +1,26 @@
|
||||
/**
|
||||
* Backoff calculation for job retries.
|
||||
* Exponential: 2^(attempts-1) * delay, with jitter.
|
||||
* Fixed: constant delay, with jitter.
|
||||
* From Sidekiq's formula, with BullMQ-style jitter parameter.
|
||||
*/
|
||||
|
||||
import type { MinionJob } from './types.ts';
|
||||
|
||||
export function calculateBackoff(job: Pick<MinionJob, 'backoff_type' | 'backoff_delay' | 'backoff_jitter' | 'attempts_made'>): number {
|
||||
const { backoff_type, backoff_delay, backoff_jitter, attempts_made } = job;
|
||||
|
||||
let delay: number;
|
||||
if (backoff_type === 'exponential') {
|
||||
delay = Math.pow(2, Math.max(attempts_made - 1, 0)) * backoff_delay;
|
||||
} else {
|
||||
delay = backoff_delay;
|
||||
}
|
||||
|
||||
if (backoff_jitter > 0) {
|
||||
const jitterRange = delay * backoff_jitter;
|
||||
delay += Math.random() * jitterRange * 2 - jitterRange;
|
||||
}
|
||||
|
||||
return Math.max(delay, 0);
|
||||
}
|
||||
9
src/core/minions/index.ts
Normal file
9
src/core/minions/index.ts
Normal file
@@ -0,0 +1,9 @@
|
||||
export { MinionQueue } from './queue.ts';
|
||||
export { MinionWorker } from './worker.ts';
|
||||
export { calculateBackoff } from './backoff.ts';
|
||||
export { UnrecoverableError, rowToMinionJob, rowToInboxMessage } from './types.ts';
|
||||
export type {
|
||||
MinionJob, MinionJobInput, MinionJobStatus, MinionJobContext,
|
||||
MinionHandler, MinionWorkerOpts, BackoffType, ChildFailPolicy,
|
||||
InboxMessage, TokenUpdate, AgentProgress, TranscriptEntry,
|
||||
} from './types.ts';
|
||||
953
src/core/minions/queue.ts
Normal file
953
src/core/minions/queue.ts
Normal file
@@ -0,0 +1,953 @@
|
||||
/**
|
||||
* MinionQueue — Postgres-native job queue inspired by BullMQ.
|
||||
*
|
||||
* Usage:
|
||||
* const queue = new MinionQueue(engine);
|
||||
* const job = await queue.add('sync', { full: true });
|
||||
* const status = await queue.getJob(job.id);
|
||||
* await queue.prune({ olderThan: new Date(Date.now() - 30 * 86400000) });
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import type {
|
||||
MinionJob, MinionJobInput, MinionJobStatus, InboxMessage, TokenUpdate,
|
||||
MinionQueueOpts, ChildDoneMessage, Attachment, AttachmentInput,
|
||||
} from './types.ts';
|
||||
import { rowToMinionJob, rowToInboxMessage, rowToAttachment } from './types.ts';
|
||||
import { validateAttachment } from './attachments.ts';
|
||||
|
||||
const MIGRATION_VERSION = 7;
|
||||
|
||||
const DEFAULT_MAX_SPAWN_DEPTH = 5;
|
||||
const DEFAULT_MAX_ATTACHMENT_BYTES = 5 * 1024 * 1024; // 5 MiB
|
||||
|
||||
const TERMINAL_STATUSES = ['completed', 'failed', 'dead', 'cancelled'] as const;
|
||||
|
||||
export class MinionQueue {
|
||||
readonly maxSpawnDepth: number;
|
||||
readonly maxAttachmentBytes: number;
|
||||
|
||||
constructor(private engine: BrainEngine, opts: MinionQueueOpts = {}) {
|
||||
this.maxSpawnDepth = opts.maxSpawnDepth ?? DEFAULT_MAX_SPAWN_DEPTH;
|
||||
this.maxAttachmentBytes = opts.maxAttachmentBytes ?? DEFAULT_MAX_ATTACHMENT_BYTES;
|
||||
}
|
||||
|
||||
/** Verify minion_jobs table exists (migration v5+). Call before first operation. */
|
||||
async ensureSchema(): Promise<void> {
|
||||
const ver = await this.engine.getConfig('version');
|
||||
const current = parseInt(ver || '1', 10);
|
||||
if (current < MIGRATION_VERSION) {
|
||||
throw new Error(
|
||||
`minion_jobs table not found (schema version ${current}, need ${MIGRATION_VERSION}). Run 'gbrain init' to apply migrations.`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Submit a new job.
|
||||
*
|
||||
* Wrapped in engine.transaction(): when parent_job_id is set, takes
|
||||
* SELECT ... FOR UPDATE on the parent so concurrent submissions serialize
|
||||
* on the cap check. Without this, two concurrent submissions could both
|
||||
* see count = N-1 and both insert, blowing max_children.
|
||||
*
|
||||
* Child status is 'waiting' (or 'delayed') — claimable. Parent is flipped
|
||||
* to 'waiting-children' atomically. Idempotency_key dedups via PG unique
|
||||
* partial index; same key returns the existing row (no second insert).
|
||||
*/
|
||||
async add(name: string, data?: Record<string, unknown>, opts?: Partial<MinionJobInput>): Promise<MinionJob> {
|
||||
if (!name || name.trim().length === 0) {
|
||||
throw new Error('Job name cannot be empty');
|
||||
}
|
||||
await this.ensureSchema();
|
||||
|
||||
const childStatus: MinionJobStatus = opts?.delay ? 'delayed' : 'waiting';
|
||||
const delayUntil = opts?.delay ? new Date(Date.now() + opts.delay) : null;
|
||||
const maxSpawnDepth = opts?.max_spawn_depth ?? this.maxSpawnDepth;
|
||||
|
||||
return this.engine.transaction(async (tx) => {
|
||||
// 1. Idempotency fast path — if a row already exists for this key, return it
|
||||
// without doing any other work. The unique partial index guarantees
|
||||
// no second row can be inserted with the same non-null key.
|
||||
if (opts?.idempotency_key) {
|
||||
const existing = await tx.executeRaw<Record<string, unknown>>(
|
||||
`SELECT * FROM minion_jobs WHERE idempotency_key = $1`,
|
||||
[opts.idempotency_key]
|
||||
);
|
||||
if (existing.length > 0) return rowToMinionJob(existing[0]);
|
||||
}
|
||||
|
||||
// 2. Parent lock + depth/cap validation
|
||||
let depth = 0;
|
||||
if (opts?.parent_job_id) {
|
||||
const parentRows = await tx.executeRaw<Record<string, unknown>>(
|
||||
`SELECT * FROM minion_jobs WHERE id = $1 FOR UPDATE`,
|
||||
[opts.parent_job_id]
|
||||
);
|
||||
if (parentRows.length === 0) {
|
||||
throw new Error(`parent_job_id ${opts.parent_job_id} not found`);
|
||||
}
|
||||
const parent = rowToMinionJob(parentRows[0]);
|
||||
|
||||
depth = parent.depth + 1;
|
||||
if (depth > maxSpawnDepth) {
|
||||
throw new Error(`spawn depth ${depth} exceeds maxSpawnDepth ${maxSpawnDepth}`);
|
||||
}
|
||||
|
||||
if (parent.max_children !== null) {
|
||||
const countRows = await tx.executeRaw<{ count: string }>(
|
||||
`SELECT count(*)::text as count FROM minion_jobs
|
||||
WHERE parent_job_id = $1 AND status NOT IN ('completed','failed','dead','cancelled')`,
|
||||
[opts.parent_job_id]
|
||||
);
|
||||
const live = parseInt(countRows[0]?.count ?? '0', 10);
|
||||
if (live >= parent.max_children) {
|
||||
throw new Error(`parent ${opts.parent_job_id} already has ${live} live children (max_children=${parent.max_children})`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Insert child. Use ON CONFLICT for idempotency; if a concurrent submit
|
||||
// raced past the fast-path SELECT, the unique index catches it here.
|
||||
const insertSql = opts?.idempotency_key
|
||||
? `INSERT INTO minion_jobs (name, queue, status, priority, data, max_attempts, backoff_type,
|
||||
backoff_delay, backoff_jitter, delay_until, parent_job_id, on_child_fail,
|
||||
depth, max_children, timeout_ms, remove_on_complete, remove_on_fail, idempotency_key)
|
||||
VALUES ($1, $2, $3, $4, $5::jsonb, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18)
|
||||
ON CONFLICT (idempotency_key) WHERE idempotency_key IS NOT NULL DO NOTHING
|
||||
RETURNING *`
|
||||
: `INSERT INTO minion_jobs (name, queue, status, priority, data, max_attempts, backoff_type,
|
||||
backoff_delay, backoff_jitter, delay_until, parent_job_id, on_child_fail,
|
||||
depth, max_children, timeout_ms, remove_on_complete, remove_on_fail, idempotency_key)
|
||||
VALUES ($1, $2, $3, $4, $5::jsonb, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18)
|
||||
RETURNING *`;
|
||||
|
||||
const params = [
|
||||
name.trim(),
|
||||
opts?.queue ?? 'default',
|
||||
childStatus,
|
||||
opts?.priority ?? 0,
|
||||
data ?? {},
|
||||
opts?.max_attempts ?? 3,
|
||||
opts?.backoff_type ?? 'exponential',
|
||||
opts?.backoff_delay ?? 1000,
|
||||
opts?.backoff_jitter ?? 0.2,
|
||||
delayUntil?.toISOString() ?? null,
|
||||
opts?.parent_job_id ?? null,
|
||||
opts?.on_child_fail ?? 'fail_parent',
|
||||
depth,
|
||||
opts?.max_children ?? null,
|
||||
opts?.timeout_ms ?? null,
|
||||
opts?.remove_on_complete ?? false,
|
||||
opts?.remove_on_fail ?? false,
|
||||
opts?.idempotency_key ?? null,
|
||||
];
|
||||
|
||||
const inserted = await tx.executeRaw<Record<string, unknown>>(insertSql, params);
|
||||
|
||||
// ON CONFLICT DO NOTHING returns 0 rows — fall back to SELECT to fetch the
|
||||
// existing row that won the race.
|
||||
if (inserted.length === 0 && opts?.idempotency_key) {
|
||||
const existing = await tx.executeRaw<Record<string, unknown>>(
|
||||
`SELECT * FROM minion_jobs WHERE idempotency_key = $1`,
|
||||
[opts.idempotency_key]
|
||||
);
|
||||
if (existing.length === 0) {
|
||||
throw new Error(`idempotency_key ${opts.idempotency_key} insert returned no row and no existing row found`);
|
||||
}
|
||||
return rowToMinionJob(existing[0]);
|
||||
}
|
||||
|
||||
const child = rowToMinionJob(inserted[0]);
|
||||
|
||||
// 4. Flip parent to waiting-children if this is a fresh child insert.
|
||||
// Only transition from non-terminal, non-already-waiting-children states.
|
||||
if (opts?.parent_job_id) {
|
||||
await tx.executeRaw(
|
||||
`UPDATE minion_jobs SET status = 'waiting-children', updated_at = now()
|
||||
WHERE id = $1 AND status IN ('waiting','active','delayed')`,
|
||||
[opts.parent_job_id]
|
||||
);
|
||||
}
|
||||
|
||||
return child;
|
||||
});
|
||||
}
|
||||
|
||||
/** Get a job by ID. Returns null if not found. */
|
||||
async getJob(id: number): Promise<MinionJob | null> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
'SELECT * FROM minion_jobs WHERE id = $1',
|
||||
[id]
|
||||
);
|
||||
return rows.length > 0 ? rowToMinionJob(rows[0]) : null;
|
||||
}
|
||||
|
||||
/** List jobs with optional filters. */
|
||||
async getJobs(opts?: {
|
||||
status?: MinionJobStatus;
|
||||
queue?: string;
|
||||
name?: string;
|
||||
limit?: number;
|
||||
offset?: number;
|
||||
}): Promise<MinionJob[]> {
|
||||
const conditions: string[] = [];
|
||||
const params: unknown[] = [];
|
||||
let idx = 1;
|
||||
|
||||
if (opts?.status) {
|
||||
conditions.push(`status = $${idx++}`);
|
||||
params.push(opts.status);
|
||||
}
|
||||
if (opts?.queue) {
|
||||
conditions.push(`queue = $${idx++}`);
|
||||
params.push(opts.queue);
|
||||
}
|
||||
if (opts?.name) {
|
||||
conditions.push(`name = $${idx++}`);
|
||||
params.push(opts.name);
|
||||
}
|
||||
|
||||
const where = conditions.length > 0 ? `WHERE ${conditions.join(' AND ')}` : '';
|
||||
const limit = opts?.limit ?? 50;
|
||||
const offset = opts?.offset ?? 0;
|
||||
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`SELECT * FROM minion_jobs ${where} ORDER BY created_at DESC LIMIT $${idx++} OFFSET $${idx}`,
|
||||
[...params, limit, offset]
|
||||
);
|
||||
return rows.map(rowToMinionJob);
|
||||
}
|
||||
|
||||
/** Remove a job. Only terminal statuses can be removed. */
|
||||
async removeJob(id: number): Promise<boolean> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`DELETE FROM minion_jobs WHERE id = $1 AND status IN ('completed', 'dead', 'cancelled', 'failed') RETURNING id`,
|
||||
[id]
|
||||
);
|
||||
return rows.length > 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Cancel a job and cascade-kill all descendants in one statement.
|
||||
*
|
||||
* Honest scope: this is BullMQ-style best-effort cancel. The recursive CTE
|
||||
* snapshots the parent_job_id chain at statement start. A descendant
|
||||
* re-parented BEFORE the cancel call is excluded; one re-parented DURING
|
||||
* the call may still get cancelled (cancel wins if seen in the snapshot).
|
||||
* Re-parented descendants whose parent_job_id is NULL'd by
|
||||
* removeChildDependency naturally fall out of the recursive walk.
|
||||
*
|
||||
* Active descendants get lock_token = NULL — same path pause uses, so the
|
||||
* worker's renewLock will fail next tick and AbortController fires.
|
||||
*
|
||||
* Returns the *root* (the job matching id), not an arbitrary descendant.
|
||||
*/
|
||||
async cancelJob(id: number): Promise<MinionJob | null> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`WITH RECURSIVE descendants AS (
|
||||
SELECT id, 0 AS d FROM minion_jobs WHERE id = $1
|
||||
UNION ALL
|
||||
SELECT m.id, descendants.d + 1
|
||||
FROM minion_jobs m
|
||||
JOIN descendants ON m.parent_job_id = descendants.id
|
||||
WHERE descendants.d < 100
|
||||
)
|
||||
UPDATE minion_jobs SET
|
||||
status = 'cancelled',
|
||||
lock_token = NULL,
|
||||
lock_until = NULL,
|
||||
finished_at = now(),
|
||||
updated_at = now()
|
||||
WHERE id IN (SELECT id FROM descendants)
|
||||
AND status IN ('waiting','active','delayed','waiting-children','paused')
|
||||
RETURNING *`,
|
||||
[id]
|
||||
);
|
||||
if (rows.length === 0) return null;
|
||||
const root = rows.find(r => (r.id as number) === id);
|
||||
return root ? rowToMinionJob(root) : null;
|
||||
}
|
||||
|
||||
/** Re-queue a failed or dead job for retry. */
|
||||
async retryJob(id: number): Promise<MinionJob | null> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET status = 'waiting', error_text = NULL,
|
||||
lock_token = NULL, lock_until = NULL, delay_until = NULL,
|
||||
finished_at = NULL, updated_at = now()
|
||||
WHERE id = $1 AND status IN ('failed', 'dead')
|
||||
RETURNING *`,
|
||||
[id]
|
||||
);
|
||||
return rows.length > 0 ? rowToMinionJob(rows[0]) : null;
|
||||
}
|
||||
|
||||
/** Prune old jobs in terminal statuses. Returns count of deleted rows. */
|
||||
async prune(opts?: { olderThan?: Date; status?: MinionJobStatus[] }): Promise<number> {
|
||||
const statuses = opts?.status ?? ['completed', 'dead', 'cancelled'];
|
||||
const olderThan = opts?.olderThan ?? new Date(Date.now() - 30 * 86400000);
|
||||
|
||||
const rows = await this.engine.executeRaw<{ count: string }>(
|
||||
`WITH pruned AS (
|
||||
DELETE FROM minion_jobs
|
||||
WHERE status = ANY($1) AND updated_at < $2
|
||||
RETURNING id
|
||||
)
|
||||
SELECT count(*)::text as count FROM pruned`,
|
||||
[statuses, olderThan.toISOString()]
|
||||
);
|
||||
return parseInt(rows[0]?.count ?? '0', 10);
|
||||
}
|
||||
|
||||
/** Get job statistics. */
|
||||
async getStats(opts?: { since?: Date }): Promise<{
|
||||
by_status: Record<string, number>;
|
||||
by_type: Array<{ name: string; total: number; completed: number; failed: number; dead: number; avg_duration_ms: number | null }>;
|
||||
queue_health: { waiting: number; active: number; stalled: number };
|
||||
}> {
|
||||
const since = opts?.since ?? new Date(Date.now() - 86400000);
|
||||
|
||||
// Status counts
|
||||
const statusRows = await this.engine.executeRaw<{ status: string; count: string }>(
|
||||
`SELECT status, count(*)::text as count FROM minion_jobs GROUP BY status`
|
||||
);
|
||||
const by_status: Record<string, number> = {};
|
||||
for (const r of statusRows) by_status[r.status] = parseInt(r.count, 10);
|
||||
|
||||
// Type breakdown (within time window)
|
||||
const typeRows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`SELECT name,
|
||||
count(*)::text as total,
|
||||
count(*) FILTER (WHERE status = 'completed')::text as completed,
|
||||
count(*) FILTER (WHERE status = 'failed')::text as failed,
|
||||
count(*) FILTER (WHERE status = 'dead')::text as dead,
|
||||
avg(EXTRACT(EPOCH FROM (finished_at - started_at)) * 1000) FILTER (WHERE finished_at IS NOT NULL AND started_at IS NOT NULL) as avg_duration_ms
|
||||
FROM minion_jobs WHERE created_at >= $1
|
||||
GROUP BY name ORDER BY total DESC`,
|
||||
[since.toISOString()]
|
||||
);
|
||||
const by_type = typeRows.map(r => ({
|
||||
name: r.name as string,
|
||||
total: parseInt(r.total as string, 10),
|
||||
completed: parseInt(r.completed as string, 10),
|
||||
failed: parseInt(r.failed as string, 10),
|
||||
dead: parseInt(r.dead as string, 10),
|
||||
avg_duration_ms: r.avg_duration_ms != null ? Math.round(r.avg_duration_ms as number) : null,
|
||||
}));
|
||||
|
||||
// Queue health: stalled = active with expired lock
|
||||
const stalledRows = await this.engine.executeRaw<{ count: string }>(
|
||||
`SELECT count(*)::text as count FROM minion_jobs WHERE status = 'active' AND lock_until < now()`
|
||||
);
|
||||
const stalled = parseInt(stalledRows[0]?.count ?? '0', 10);
|
||||
|
||||
return {
|
||||
by_status,
|
||||
by_type,
|
||||
queue_health: {
|
||||
waiting: by_status['waiting'] ?? 0,
|
||||
active: by_status['active'] ?? 0,
|
||||
stalled,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Claim the next waiting job for a worker. Token-fenced, filters by registered names.
|
||||
*
|
||||
* Sets timeout_at = now() + timeout_ms when the job has a per-job deadline,
|
||||
* so handleTimeouts() can dead-letter expired jobs without rereading timeout_ms.
|
||||
*/
|
||||
async claim(lockToken: string, lockDurationMs: number, queue: string, registeredNames: string[]): Promise<MinionJob | null> {
|
||||
if (registeredNames.length === 0) return null;
|
||||
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET
|
||||
status = 'active',
|
||||
lock_token = $1,
|
||||
lock_until = now() + ($2::double precision * interval '1 millisecond'),
|
||||
timeout_at = CASE WHEN timeout_ms IS NOT NULL
|
||||
THEN now() + (timeout_ms::double precision * interval '1 millisecond')
|
||||
ELSE NULL END,
|
||||
attempts_started = attempts_started + 1,
|
||||
started_at = COALESCE(started_at, now()),
|
||||
updated_at = now()
|
||||
WHERE id = (
|
||||
SELECT id FROM minion_jobs
|
||||
WHERE queue = $3 AND status = 'waiting' AND name = ANY($4)
|
||||
ORDER BY priority ASC, created_at ASC
|
||||
FOR UPDATE SKIP LOCKED
|
||||
LIMIT 1
|
||||
)
|
||||
RETURNING *`,
|
||||
[lockToken, lockDurationMs, queue, registeredNames]
|
||||
);
|
||||
return rows.length > 0 ? rowToMinionJob(rows[0]) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Dead-letter active jobs whose timeout_at has passed.
|
||||
*
|
||||
* The lock_until > now() guard is critical: a stalled job (lock_until < now)
|
||||
* is being requeued by handleStalled, NOT timed out terminally. Stall →
|
||||
* retry, timeout → dead. Order in worker loop: handleStalled() before
|
||||
* handleTimeouts() to give stall recovery first crack.
|
||||
*
|
||||
* Honest scope: 1-tick TOCTOU window remains. A job whose lock_until
|
||||
* expires between handleStalled and handleTimeouts may miss this tick
|
||||
* but will be caught the next one (after re-claim). Never double-handled.
|
||||
*/
|
||||
async handleTimeouts(): Promise<MinionJob[]> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET
|
||||
status = 'dead',
|
||||
error_text = 'timeout exceeded',
|
||||
lock_token = NULL,
|
||||
lock_until = NULL,
|
||||
finished_at = now(),
|
||||
updated_at = now()
|
||||
WHERE status = 'active'
|
||||
AND timeout_at IS NOT NULL
|
||||
AND timeout_at < now()
|
||||
AND lock_until > now()
|
||||
RETURNING *`
|
||||
);
|
||||
return rows.map(rowToMinionJob);
|
||||
}
|
||||
|
||||
/**
|
||||
* Complete a job (token-fenced). All side effects atomic in one transaction:
|
||||
* 1. UPDATE child to 'completed' with result
|
||||
* 2. Roll up token counts to parent (skipped if parent is terminal)
|
||||
* 3. Insert child_done message into parent's inbox (skipped if parent terminal)
|
||||
* 4. Resolve parent (flip waiting-children → waiting if all kids done)
|
||||
* 5. If remove_on_complete, DELETE the child row (cascades inbox + attachments)
|
||||
*
|
||||
* Returns the completed job (the in-memory snapshot before any delete), or
|
||||
* null if the lock_token mismatched (e.g., reclaimed mid-completion).
|
||||
*
|
||||
* The fold-in of resolveParent eliminates the crash window where a process
|
||||
* died between completeJob and worker's prior post-call resolveParent,
|
||||
* stranding the parent in waiting-children forever.
|
||||
*/
|
||||
async completeJob(id: number, lockToken: string, result?: Record<string, unknown>): Promise<MinionJob | null> {
|
||||
return this.engine.transaction(async (tx) => {
|
||||
// Peek at parent_job_id before the UPDATE so we can lock the parent row
|
||||
// FIRST. Without this SELECT FOR UPDATE, two siblings completing
|
||||
// concurrently each see the other as still active (pre-commit snapshot
|
||||
// under read-committed), neither flips the parent, and the parent is
|
||||
// stuck in waiting-children forever.
|
||||
const peek = await tx.executeRaw<{ parent_job_id: number | null }>(
|
||||
`SELECT parent_job_id FROM minion_jobs WHERE id = $1`,
|
||||
[id]
|
||||
);
|
||||
const parentId = peek[0]?.parent_job_id ?? null;
|
||||
if (parentId) {
|
||||
await tx.executeRaw(
|
||||
`SELECT id FROM minion_jobs WHERE id = $1 FOR UPDATE`,
|
||||
[parentId]
|
||||
);
|
||||
}
|
||||
|
||||
const rows = await tx.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET status = 'completed', result = $1::jsonb,
|
||||
finished_at = now(), lock_token = NULL, lock_until = NULL, updated_at = now()
|
||||
WHERE id = $2 AND status = 'active' AND lock_token = $3
|
||||
RETURNING *`,
|
||||
[result ?? null, id, lockToken]
|
||||
);
|
||||
if (rows.length === 0) return null;
|
||||
|
||||
const completed = rowToMinionJob(rows[0]);
|
||||
|
||||
if (completed.parent_job_id) {
|
||||
// Roll up token counts. Guarded against parent already being terminal.
|
||||
if (completed.tokens_input > 0 || completed.tokens_output > 0 || completed.tokens_cache_read > 0) {
|
||||
await tx.executeRaw(
|
||||
`UPDATE minion_jobs SET
|
||||
tokens_input = tokens_input + $1,
|
||||
tokens_output = tokens_output + $2,
|
||||
tokens_cache_read = tokens_cache_read + $3,
|
||||
updated_at = now()
|
||||
WHERE id = $4 AND status NOT IN ('completed', 'failed', 'dead', 'cancelled')`,
|
||||
[completed.tokens_input, completed.tokens_output, completed.tokens_cache_read, completed.parent_job_id]
|
||||
);
|
||||
}
|
||||
|
||||
// Auto-post child_done into parent's inbox. EXISTS guard skips if parent
|
||||
// was deleted or hit a terminal state mid-flight (no FK violation, no
|
||||
// contradiction with the token rollup guard).
|
||||
const childDone: ChildDoneMessage = {
|
||||
type: 'child_done',
|
||||
child_id: completed.id,
|
||||
job_name: completed.name,
|
||||
result: result ?? null,
|
||||
};
|
||||
await tx.executeRaw(
|
||||
`INSERT INTO minion_inbox (job_id, sender, payload)
|
||||
SELECT $1, 'minions', $2::jsonb
|
||||
WHERE EXISTS (
|
||||
SELECT 1 FROM minion_jobs
|
||||
WHERE id = $1 AND status NOT IN ('completed','failed','dead','cancelled')
|
||||
)`,
|
||||
[completed.parent_job_id, childDone]
|
||||
);
|
||||
|
||||
// Fold-in resolveParent: flip parent to waiting once all children done.
|
||||
await tx.executeRaw(
|
||||
`UPDATE minion_jobs SET status = 'waiting', updated_at = now()
|
||||
WHERE id = $1 AND status = 'waiting-children'
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM minion_jobs
|
||||
WHERE parent_job_id = $1
|
||||
AND status NOT IN ('completed', 'dead', 'cancelled')
|
||||
)`,
|
||||
[completed.parent_job_id]
|
||||
);
|
||||
}
|
||||
|
||||
// remove_on_complete cleanup AFTER all parent-side bookkeeping.
|
||||
// The child_done we just inserted lives in the *parent's* inbox row,
|
||||
// so it survives the child cascade-delete.
|
||||
if (completed.remove_on_complete) {
|
||||
await tx.executeRaw(
|
||||
`DELETE FROM minion_jobs WHERE id = $1`,
|
||||
[completed.id]
|
||||
);
|
||||
}
|
||||
|
||||
return completed;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Fail a job (token-fenced). All side effects atomic in one transaction:
|
||||
* 1. UPDATE child to 'delayed' (retry) | 'failed' | 'dead'
|
||||
* 2. If terminal AND parent_job_id, run on_child_fail policy:
|
||||
* - 'fail_parent' → mark parent 'failed' (via failParent SQL)
|
||||
* - 'remove_dep' → null out parent_job_id (via removeChildDependency SQL)
|
||||
* - 'ignore' / 'continue' → no parent action
|
||||
* 3. If remove_on_fail AND terminal, DELETE the child row (parent hook
|
||||
* already ran in this txn using in-memory state, so child deletion is safe)
|
||||
*
|
||||
* Folding the parent hook into this transaction eliminates the crash window
|
||||
* where a process died between failJob and worker's prior post-call hook,
|
||||
* leaving the parent stuck in waiting-children.
|
||||
*/
|
||||
async failJob(
|
||||
id: number,
|
||||
lockToken: string,
|
||||
errorText: string,
|
||||
newStatus: 'delayed' | 'failed' | 'dead',
|
||||
backoffMs?: number
|
||||
): Promise<MinionJob | null> {
|
||||
return this.engine.transaction(async (tx) => {
|
||||
// Lock the parent row first so concurrent sibling completions/failures
|
||||
// serialize on the parent — same race fix as completeJob.
|
||||
const peek = await tx.executeRaw<{ parent_job_id: number | null }>(
|
||||
`SELECT parent_job_id FROM minion_jobs WHERE id = $1`,
|
||||
[id]
|
||||
);
|
||||
const parentId = peek[0]?.parent_job_id ?? null;
|
||||
if (parentId) {
|
||||
await tx.executeRaw(
|
||||
`SELECT id FROM minion_jobs WHERE id = $1 FOR UPDATE`,
|
||||
[parentId]
|
||||
);
|
||||
}
|
||||
|
||||
const rows = await tx.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET
|
||||
status = $1, error_text = $2, attempts_made = attempts_made + 1,
|
||||
stacktrace = COALESCE(stacktrace, '[]'::jsonb) || to_jsonb($3::text),
|
||||
delay_until = CASE WHEN $1 = 'delayed' THEN now() + ($4::double precision * interval '1 millisecond') ELSE NULL END,
|
||||
finished_at = CASE WHEN $1 IN ('failed', 'dead') THEN now() ELSE NULL END,
|
||||
lock_token = NULL, lock_until = NULL, updated_at = now()
|
||||
WHERE id = $5 AND status = 'active' AND lock_token = $6
|
||||
RETURNING *`,
|
||||
[newStatus, errorText, errorText, backoffMs ?? 0, id, lockToken]
|
||||
);
|
||||
if (rows.length === 0) return null;
|
||||
|
||||
const failed = rowToMinionJob(rows[0]);
|
||||
const terminal = newStatus === 'failed' || newStatus === 'dead';
|
||||
|
||||
// Parent hook on terminal failure.
|
||||
if (terminal && failed.parent_job_id) {
|
||||
if (failed.on_child_fail === 'fail_parent') {
|
||||
await tx.executeRaw(
|
||||
`UPDATE minion_jobs SET status = 'failed',
|
||||
error_text = $1, finished_at = now(), updated_at = now()
|
||||
WHERE id = $2 AND status = 'waiting-children'`,
|
||||
[`child job ${failed.id} failed: ${errorText}`, failed.parent_job_id]
|
||||
);
|
||||
} else if (failed.on_child_fail === 'remove_dep') {
|
||||
await tx.executeRaw(
|
||||
`UPDATE minion_jobs SET parent_job_id = NULL, updated_at = now() WHERE id = $1`,
|
||||
[failed.id]
|
||||
);
|
||||
// After dropping the dep, try to resolve the parent if all OTHER kids are done.
|
||||
await tx.executeRaw(
|
||||
`UPDATE minion_jobs SET status = 'waiting', updated_at = now()
|
||||
WHERE id = $1 AND status = 'waiting-children'
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM minion_jobs
|
||||
WHERE parent_job_id = $1
|
||||
AND status NOT IN ('completed', 'dead', 'cancelled')
|
||||
)`,
|
||||
[failed.parent_job_id]
|
||||
);
|
||||
}
|
||||
// 'ignore' / 'continue' → parent stays in waiting-children waiting on siblings
|
||||
}
|
||||
|
||||
// remove_on_fail cleanup AFTER parent hook.
|
||||
if (terminal && failed.remove_on_fail) {
|
||||
await tx.executeRaw(
|
||||
`DELETE FROM minion_jobs WHERE id = $1`,
|
||||
[failed.id]
|
||||
);
|
||||
}
|
||||
|
||||
return failed;
|
||||
});
|
||||
}
|
||||
|
||||
/** Update job progress (token-fenced). */
|
||||
async updateProgress(id: number, lockToken: string, progress: unknown): Promise<boolean> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET progress = $1::jsonb, updated_at = now()
|
||||
WHERE id = $2 AND status = 'active' AND lock_token = $3
|
||||
RETURNING id`,
|
||||
[progress, id, lockToken]
|
||||
);
|
||||
return rows.length > 0;
|
||||
}
|
||||
|
||||
/** Renew lock (token-fenced). Returns false if token mismatch (job was reclaimed). */
|
||||
async renewLock(id: number, lockToken: string, lockDurationMs: number): Promise<boolean> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET lock_until = now() + ($1::double precision * interval '1 millisecond'), updated_at = now()
|
||||
WHERE id = $2 AND lock_token = $3 AND status = 'active'
|
||||
RETURNING id`,
|
||||
[lockDurationMs, id, lockToken]
|
||||
);
|
||||
return rows.length > 0;
|
||||
}
|
||||
|
||||
/** Promote delayed jobs whose delay_until has passed. Returns promoted jobs. */
|
||||
async promoteDelayed(): Promise<MinionJob[]> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET status = 'waiting', delay_until = NULL,
|
||||
lock_token = NULL, lock_until = NULL, updated_at = now()
|
||||
WHERE status = 'delayed' AND delay_until <= now()
|
||||
RETURNING *`
|
||||
);
|
||||
return rows.map(rowToMinionJob);
|
||||
}
|
||||
|
||||
/** Detect and handle stalled jobs. Single CTE, no off-by-one. Returns affected jobs. */
|
||||
async handleStalled(): Promise<{ requeued: MinionJob[]; dead: MinionJob[] }> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown> & { action: string }>(
|
||||
`WITH stalled AS (
|
||||
SELECT id, stalled_counter, max_stalled
|
||||
FROM minion_jobs
|
||||
WHERE status = 'active' AND lock_until < now()
|
||||
FOR UPDATE SKIP LOCKED
|
||||
),
|
||||
requeued AS (
|
||||
UPDATE minion_jobs SET
|
||||
status = 'waiting', stalled_counter = stalled_counter + 1,
|
||||
lock_token = NULL, lock_until = NULL, updated_at = now()
|
||||
WHERE id IN (SELECT id FROM stalled WHERE stalled_counter + 1 < max_stalled)
|
||||
RETURNING *, 'requeued' as action
|
||||
),
|
||||
dead_lettered AS (
|
||||
UPDATE minion_jobs SET
|
||||
status = 'dead', stalled_counter = stalled_counter + 1,
|
||||
error_text = 'max stalled count exceeded',
|
||||
lock_token = NULL, lock_until = NULL, finished_at = now(), updated_at = now()
|
||||
WHERE id IN (SELECT id FROM stalled WHERE stalled_counter + 1 >= max_stalled)
|
||||
RETURNING *, 'dead' as action
|
||||
)
|
||||
SELECT * FROM requeued UNION ALL SELECT * FROM dead_lettered`
|
||||
);
|
||||
|
||||
const requeued: MinionJob[] = [];
|
||||
const dead: MinionJob[] = [];
|
||||
for (const r of rows) {
|
||||
const job = rowToMinionJob(r);
|
||||
if (r.action === 'requeued') requeued.push(job);
|
||||
else dead.push(job);
|
||||
}
|
||||
return { requeued, dead };
|
||||
}
|
||||
|
||||
/** Check if all children of a parent are done. If so, unblock parent. */
|
||||
async resolveParent(parentId: number): Promise<MinionJob | null> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET status = 'waiting', updated_at = now()
|
||||
WHERE id = $1 AND status = 'waiting-children'
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM minion_jobs
|
||||
WHERE parent_job_id = $1
|
||||
AND status NOT IN ('completed', 'dead', 'cancelled')
|
||||
)
|
||||
RETURNING *`,
|
||||
[parentId]
|
||||
);
|
||||
return rows.length > 0 ? rowToMinionJob(rows[0]) : null;
|
||||
}
|
||||
|
||||
/** Fail the parent when a child fails with fail_parent policy. */
|
||||
async failParent(parentId: number, childId: number, errorText: string): Promise<MinionJob | null> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET status = 'failed',
|
||||
error_text = $1, finished_at = now(), updated_at = now()
|
||||
WHERE id = $2 AND status = 'waiting-children'
|
||||
RETURNING *`,
|
||||
[`child job ${childId} failed: ${errorText}`, parentId]
|
||||
);
|
||||
return rows.length > 0 ? rowToMinionJob(rows[0]) : null;
|
||||
}
|
||||
|
||||
/** Pause a waiting or active job. For active jobs, clears the lock so the worker's
|
||||
* AbortController fires and the handler stops gracefully. */
|
||||
async pauseJob(id: number): Promise<MinionJob | null> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET status = 'paused',
|
||||
lock_token = NULL, lock_until = NULL, updated_at = now()
|
||||
WHERE id = $1 AND status IN ('waiting', 'active', 'delayed')
|
||||
RETURNING *`,
|
||||
[id]
|
||||
);
|
||||
return rows.length > 0 ? rowToMinionJob(rows[0]) : null;
|
||||
}
|
||||
|
||||
/** Resume a paused job back to waiting. */
|
||||
async resumeJob(id: number): Promise<MinionJob | null> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET status = 'waiting',
|
||||
lock_token = NULL, lock_until = NULL, updated_at = now()
|
||||
WHERE id = $1 AND status = 'paused'
|
||||
RETURNING *`,
|
||||
[id]
|
||||
);
|
||||
return rows.length > 0 ? rowToMinionJob(rows[0]) : null;
|
||||
}
|
||||
|
||||
/** Send a message to a job's inbox. Sender must be the parent job or 'admin'. */
|
||||
async sendMessage(jobId: number, payload: unknown, sender: string): Promise<InboxMessage | null> {
|
||||
// Validate job exists and is in a messageable state
|
||||
const job = await this.getJob(jobId);
|
||||
if (!job) return null;
|
||||
if (['completed', 'dead', 'cancelled', 'failed'].includes(job.status)) return null;
|
||||
|
||||
// Sender validation: must be parent job ID or 'admin'
|
||||
if (sender !== 'admin' && sender !== String(job.parent_job_id)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`INSERT INTO minion_inbox (job_id, sender, payload)
|
||||
VALUES ($1, $2, $3)
|
||||
RETURNING *`,
|
||||
[jobId, sender, payload]
|
||||
);
|
||||
return rows.length > 0 ? rowToInboxMessage(rows[0]) : null;
|
||||
}
|
||||
|
||||
/** Read unread inbox messages for a job. Token-fenced. Marks messages as read. */
|
||||
async readInbox(jobId: number, lockToken: string): Promise<InboxMessage[]> {
|
||||
// Verify lock ownership
|
||||
const lockCheck = await this.engine.executeRaw<{ id: number }>(
|
||||
`SELECT id FROM minion_jobs WHERE id = $1 AND lock_token = $2 AND status = 'active'`,
|
||||
[jobId, lockToken]
|
||||
);
|
||||
if (lockCheck.length === 0) return [];
|
||||
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_inbox SET read_at = now()
|
||||
WHERE job_id = $1 AND read_at IS NULL
|
||||
RETURNING *`,
|
||||
[jobId]
|
||||
);
|
||||
return rows.map(rowToInboxMessage);
|
||||
}
|
||||
|
||||
/** Update token counts for a job. Accumulates (adds to existing). Token-fenced. */
|
||||
async updateTokens(id: number, lockToken: string, tokens: TokenUpdate): Promise<boolean> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`UPDATE minion_jobs SET
|
||||
tokens_input = tokens_input + $1,
|
||||
tokens_output = tokens_output + $2,
|
||||
tokens_cache_read = tokens_cache_read + $3,
|
||||
updated_at = now()
|
||||
WHERE id = $4 AND status = 'active' AND lock_token = $5
|
||||
RETURNING id`,
|
||||
[tokens.input ?? 0, tokens.output ?? 0, tokens.cache_read ?? 0, id, lockToken]
|
||||
);
|
||||
return rows.length > 0;
|
||||
}
|
||||
|
||||
/** Replay a completed/failed/dead job with optional data overrides. Creates a new job. */
|
||||
async replayJob(id: number, dataOverrides?: Record<string, unknown>): Promise<MinionJob | null> {
|
||||
const source = await this.getJob(id);
|
||||
if (!source) return null;
|
||||
if (!['completed', 'failed', 'dead'].includes(source.status)) return null;
|
||||
|
||||
const data = dataOverrides
|
||||
? { ...source.data, ...dataOverrides }
|
||||
: source.data;
|
||||
|
||||
return this.add(source.name, data, {
|
||||
queue: source.queue,
|
||||
priority: source.priority,
|
||||
max_attempts: source.max_attempts,
|
||||
backoff_type: source.backoff_type,
|
||||
backoff_delay: source.backoff_delay,
|
||||
backoff_jitter: source.backoff_jitter,
|
||||
});
|
||||
}
|
||||
|
||||
/** Remove a child's dependency on its parent. */
|
||||
async removeChildDependency(childId: number): Promise<void> {
|
||||
await this.engine.executeRaw(
|
||||
`UPDATE minion_jobs SET parent_job_id = NULL, updated_at = now() WHERE id = $1`,
|
||||
[childId]
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Read child_done messages from a parent's inbox. Token-fenced (the parent
|
||||
* job must currently hold lockToken — same fence as readInbox to prevent a
|
||||
* stale process polling completions for jobs it no longer owns).
|
||||
*
|
||||
* Does NOT mark messages read (parent may want to poll repeatedly with a
|
||||
* cursor). Use `since` to fetch only newer entries.
|
||||
*/
|
||||
async readChildCompletions(
|
||||
parentId: number,
|
||||
lockToken: string,
|
||||
opts?: { since?: Date }
|
||||
): Promise<ChildDoneMessage[]> {
|
||||
// Verify the caller holds the parent's lock.
|
||||
const lockCheck = await this.engine.executeRaw<{ id: number }>(
|
||||
`SELECT id FROM minion_jobs WHERE id = $1 AND lock_token = $2 AND status = 'active'`,
|
||||
[parentId, lockToken]
|
||||
);
|
||||
if (lockCheck.length === 0) return [];
|
||||
|
||||
const params: unknown[] = [parentId];
|
||||
let sinceClause = '';
|
||||
if (opts?.since) {
|
||||
sinceClause = ` AND sent_at > $2::timestamptz`;
|
||||
params.push(opts.since.toISOString());
|
||||
}
|
||||
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`SELECT payload FROM minion_inbox
|
||||
WHERE job_id = $1 AND (payload->>'type') = 'child_done'${sinceClause}
|
||||
ORDER BY sent_at ASC`,
|
||||
params
|
||||
);
|
||||
|
||||
return rows.map(r => {
|
||||
const p = typeof r.payload === 'string' ? JSON.parse(r.payload) : r.payload;
|
||||
return p as ChildDoneMessage;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Attach a file to a job. Validates size, base64, filename safety, and
|
||||
* duplicate filename. Returns the persisted attachment metadata (not the
|
||||
* bytes — use getAttachment to fetch).
|
||||
*
|
||||
* The DB UNIQUE (job_id, filename) constraint is the authoritative duplicate
|
||||
* fence; the in-memory check just gives a faster error.
|
||||
*/
|
||||
async addAttachment(jobId: number, input: AttachmentInput): Promise<Attachment> {
|
||||
await this.ensureSchema();
|
||||
|
||||
// Verify job exists (FK guarantees this on insert too, but explicit error is clearer)
|
||||
const exists = await this.engine.executeRaw<{ id: number }>(
|
||||
`SELECT id FROM minion_jobs WHERE id = $1`,
|
||||
[jobId]
|
||||
);
|
||||
if (exists.length === 0) {
|
||||
throw new Error(`job ${jobId} not found`);
|
||||
}
|
||||
|
||||
const existingRows = await this.engine.executeRaw<{ filename: string }>(
|
||||
`SELECT filename FROM minion_attachments WHERE job_id = $1`,
|
||||
[jobId]
|
||||
);
|
||||
const existingFilenames = new Set(existingRows.map(r => r.filename));
|
||||
|
||||
const result = validateAttachment(input, {
|
||||
maxBytes: this.maxAttachmentBytes,
|
||||
existingFilenames,
|
||||
});
|
||||
if (!result.ok) {
|
||||
throw new Error(`attachment validation failed: ${result.error}`);
|
||||
}
|
||||
const { filename, content_type, bytes, size_bytes, sha256 } = result.normalized;
|
||||
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`INSERT INTO minion_attachments (job_id, filename, content_type, content, size_bytes, sha256)
|
||||
VALUES ($1, $2, $3, $4, $5, $6)
|
||||
RETURNING id, job_id, filename, content_type, storage_uri, size_bytes, sha256, created_at`,
|
||||
[jobId, filename, content_type, bytes, size_bytes, sha256]
|
||||
);
|
||||
return rowToAttachment(rows[0]);
|
||||
}
|
||||
|
||||
/** List attachments for a job (metadata only, no bytes). */
|
||||
async listAttachments(jobId: number): Promise<Attachment[]> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`SELECT id, job_id, filename, content_type, storage_uri, size_bytes, sha256, created_at
|
||||
FROM minion_attachments
|
||||
WHERE job_id = $1
|
||||
ORDER BY created_at ASC, id ASC`,
|
||||
[jobId]
|
||||
);
|
||||
return rows.map(rowToAttachment);
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch a single attachment with bytes. Returns null if not found.
|
||||
* The bytes are returned as a Buffer (Uint8Array under the hood).
|
||||
*/
|
||||
async getAttachment(jobId: number, filename: string): Promise<{ meta: Attachment; bytes: Buffer } | null> {
|
||||
const rows = await this.engine.executeRaw<Record<string, unknown>>(
|
||||
`SELECT id, job_id, filename, content_type, storage_uri, size_bytes, sha256, created_at, content
|
||||
FROM minion_attachments
|
||||
WHERE job_id = $1 AND filename = $2`,
|
||||
[jobId, filename]
|
||||
);
|
||||
if (rows.length === 0) return null;
|
||||
const row = rows[0];
|
||||
const meta = rowToAttachment(row);
|
||||
const raw = row.content;
|
||||
let bytes: Buffer;
|
||||
if (raw == null) {
|
||||
bytes = Buffer.alloc(0);
|
||||
} else if (Buffer.isBuffer(raw)) {
|
||||
bytes = raw;
|
||||
} else if (raw instanceof Uint8Array) {
|
||||
bytes = Buffer.from(raw);
|
||||
} else {
|
||||
bytes = Buffer.from(raw as ArrayBuffer);
|
||||
}
|
||||
return { meta, bytes };
|
||||
}
|
||||
|
||||
/** Delete an attachment by job + filename. Returns true if a row was removed. */
|
||||
async deleteAttachment(jobId: number, filename: string): Promise<boolean> {
|
||||
const rows = await this.engine.executeRaw<{ id: number }>(
|
||||
`DELETE FROM minion_attachments WHERE job_id = $1 AND filename = $2 RETURNING id`,
|
||||
[jobId, filename]
|
||||
);
|
||||
return rows.length > 0;
|
||||
}
|
||||
}
|
||||
308
src/core/minions/types.ts
Normal file
308
src/core/minions/types.ts
Normal file
@@ -0,0 +1,308 @@
|
||||
/**
|
||||
* Minions — BullMQ-inspired Postgres-native job queue for GBrain.
|
||||
*
|
||||
* Usage:
|
||||
* const queue = new MinionQueue(engine);
|
||||
* const job = await queue.add('sync', { full: true });
|
||||
*
|
||||
* const worker = new MinionWorker(engine);
|
||||
* worker.register('sync', async (job) => {
|
||||
* await runSync(engine, job.data);
|
||||
* return { pages_synced: 42 };
|
||||
* });
|
||||
* await worker.start();
|
||||
*/
|
||||
|
||||
// --- Status & Type Unions ---
|
||||
|
||||
export type MinionJobStatus =
|
||||
| 'waiting'
|
||||
| 'active'
|
||||
| 'completed'
|
||||
| 'failed'
|
||||
| 'delayed'
|
||||
| 'dead'
|
||||
| 'cancelled'
|
||||
| 'waiting-children'
|
||||
| 'paused';
|
||||
|
||||
export type BackoffType = 'fixed' | 'exponential';
|
||||
|
||||
export type ChildFailPolicy = 'fail_parent' | 'remove_dep' | 'ignore' | 'continue';
|
||||
|
||||
// --- Job Record ---
|
||||
|
||||
export interface MinionJob {
|
||||
id: number;
|
||||
name: string;
|
||||
queue: string;
|
||||
status: MinionJobStatus;
|
||||
priority: number;
|
||||
data: Record<string, unknown>;
|
||||
|
||||
// Retry
|
||||
max_attempts: number;
|
||||
attempts_made: number;
|
||||
attempts_started: number;
|
||||
backoff_type: BackoffType;
|
||||
backoff_delay: number;
|
||||
backoff_jitter: number;
|
||||
|
||||
// Stall detection
|
||||
stalled_counter: number;
|
||||
max_stalled: number;
|
||||
lock_token: string | null;
|
||||
lock_until: Date | null;
|
||||
|
||||
// Scheduling
|
||||
delay_until: Date | null;
|
||||
|
||||
// Dependencies
|
||||
parent_job_id: number | null;
|
||||
on_child_fail: ChildFailPolicy;
|
||||
|
||||
// Token accounting
|
||||
tokens_input: number;
|
||||
tokens_output: number;
|
||||
tokens_cache_read: number;
|
||||
|
||||
// v7: subagent + parity
|
||||
depth: number;
|
||||
max_children: number | null;
|
||||
timeout_ms: number | null;
|
||||
timeout_at: Date | null;
|
||||
remove_on_complete: boolean;
|
||||
remove_on_fail: boolean;
|
||||
idempotency_key: string | null;
|
||||
|
||||
// Results
|
||||
result: Record<string, unknown> | null;
|
||||
progress: unknown | null;
|
||||
error_text: string | null;
|
||||
stacktrace: string[];
|
||||
|
||||
// Timestamps
|
||||
created_at: Date;
|
||||
started_at: Date | null;
|
||||
finished_at: Date | null;
|
||||
updated_at: Date;
|
||||
}
|
||||
|
||||
// --- Input Types ---
|
||||
|
||||
export interface MinionJobInput {
|
||||
name: string;
|
||||
data?: Record<string, unknown>;
|
||||
queue?: string;
|
||||
priority?: number;
|
||||
max_attempts?: number;
|
||||
backoff_type?: BackoffType;
|
||||
backoff_delay?: number;
|
||||
backoff_jitter?: number;
|
||||
delay?: number; // ms delay before eligible
|
||||
parent_job_id?: number;
|
||||
on_child_fail?: ChildFailPolicy;
|
||||
|
||||
// v7: subagent + parity
|
||||
/** Cap on live (non-terminal) children of THIS job. NULL/undefined = unlimited. */
|
||||
max_children?: number;
|
||||
/** Wall-clock per-job deadline in ms. Set on claim → timeout_at. Terminal on expire (no retry). */
|
||||
timeout_ms?: number;
|
||||
/** DELETE row on successful completion (after token rollup + child_done insert). */
|
||||
remove_on_complete?: boolean;
|
||||
/** DELETE row on terminal failure (after parent failure hook). */
|
||||
remove_on_fail?: boolean;
|
||||
/** Override the queue's maxSpawnDepth for THIS submission only. */
|
||||
max_spawn_depth?: number;
|
||||
/** Global dedup key. Same key returns the existing job, no second row created. */
|
||||
idempotency_key?: string;
|
||||
}
|
||||
|
||||
/** Constructor options for MinionQueue (v7). */
|
||||
export interface MinionQueueOpts {
|
||||
/** Max parent→child→grandchild depth. Default 5. Enforced on add() with parent_job_id. */
|
||||
maxSpawnDepth?: number;
|
||||
/** Max attachment size in bytes. Default 5 MiB. */
|
||||
maxAttachmentBytes?: number;
|
||||
}
|
||||
|
||||
export interface MinionWorkerOpts {
|
||||
queue?: string;
|
||||
concurrency?: number; // default 1
|
||||
lockDuration?: number; // ms, default 30000
|
||||
stalledInterval?: number; // ms, default 30000
|
||||
maxStalledCount?: number; // default 1
|
||||
pollInterval?: number; // ms, default 5000 (for PGLite fallback)
|
||||
}
|
||||
|
||||
// --- Job Context (passed to handlers) ---
|
||||
|
||||
export interface MinionJobContext {
|
||||
id: number;
|
||||
name: string;
|
||||
data: Record<string, unknown>;
|
||||
attempts_made: number;
|
||||
/** AbortSignal for cooperative cancellation (fires on pause or lock loss). */
|
||||
signal: AbortSignal;
|
||||
/** Update structured progress (not just 0-100). */
|
||||
updateProgress(progress: unknown): Promise<void>;
|
||||
/** Accumulate token usage for this job. */
|
||||
updateTokens(tokens: TokenUpdate): Promise<void>;
|
||||
/** Append a log message or transcript entry to the job's stacktrace array. */
|
||||
log(message: string | TranscriptEntry): Promise<void>;
|
||||
/** Check if the lock is still held (for long-running jobs). */
|
||||
isActive(): Promise<boolean>;
|
||||
/** Read unread inbox messages (marks as read). */
|
||||
readInbox(): Promise<InboxMessage[]>;
|
||||
}
|
||||
|
||||
export type MinionHandler = (job: MinionJobContext) => Promise<unknown>;
|
||||
|
||||
// --- Inbox Message ---
|
||||
|
||||
export interface InboxMessage {
|
||||
id: number;
|
||||
job_id: number;
|
||||
sender: string;
|
||||
payload: unknown;
|
||||
sent_at: Date;
|
||||
read_at: Date | null;
|
||||
}
|
||||
|
||||
export function rowToInboxMessage(row: Record<string, unknown>): InboxMessage {
|
||||
return {
|
||||
id: row.id as number,
|
||||
job_id: row.job_id as number,
|
||||
sender: row.sender as string,
|
||||
payload: typeof row.payload === 'string' ? JSON.parse(row.payload) : row.payload,
|
||||
sent_at: new Date(row.sent_at as string),
|
||||
read_at: row.read_at ? new Date(row.read_at as string) : null,
|
||||
};
|
||||
}
|
||||
|
||||
// --- Child-done inbox message (auto-posted on completeJob) ---
|
||||
|
||||
/** Posted into the parent's inbox when a child completes successfully. */
|
||||
export interface ChildDoneMessage {
|
||||
type: 'child_done';
|
||||
child_id: number;
|
||||
job_name: string;
|
||||
result: unknown;
|
||||
}
|
||||
|
||||
// --- Attachments (v7) ---
|
||||
|
||||
/** Caller-supplied attachment payload. content is base64-encoded bytes. */
|
||||
export interface AttachmentInput {
|
||||
filename: string;
|
||||
content_type: string;
|
||||
/** Base64-encoded file bytes. Validated server-side. */
|
||||
content_base64: string;
|
||||
}
|
||||
|
||||
/** Persisted attachment row (without inline bytes; use getAttachment to fetch). */
|
||||
export interface Attachment {
|
||||
id: number;
|
||||
job_id: number;
|
||||
filename: string;
|
||||
content_type: string;
|
||||
storage_uri: string | null;
|
||||
size_bytes: number;
|
||||
sha256: string;
|
||||
created_at: Date;
|
||||
}
|
||||
|
||||
export function rowToAttachment(row: Record<string, unknown>): Attachment {
|
||||
return {
|
||||
id: row.id as number,
|
||||
job_id: row.job_id as number,
|
||||
filename: row.filename as string,
|
||||
content_type: row.content_type as string,
|
||||
storage_uri: (row.storage_uri as string) || null,
|
||||
size_bytes: row.size_bytes as number,
|
||||
sha256: row.sha256 as string,
|
||||
created_at: new Date(row.created_at as string),
|
||||
};
|
||||
}
|
||||
|
||||
// --- Token Update ---
|
||||
|
||||
export interface TokenUpdate {
|
||||
input?: number;
|
||||
output?: number;
|
||||
cache_read?: number;
|
||||
}
|
||||
|
||||
// --- Structured Progress (convention, not enforced) ---
|
||||
|
||||
export interface AgentProgress {
|
||||
step: number;
|
||||
total: number;
|
||||
message: string;
|
||||
tokens_in: number;
|
||||
tokens_out: number;
|
||||
last_tool: string;
|
||||
started_at: string;
|
||||
}
|
||||
|
||||
// --- Transcript Entry ---
|
||||
|
||||
export type TranscriptEntry =
|
||||
| { type: 'log'; message: string; ts: string }
|
||||
| { type: 'tool_call'; tool: string; args_size: number; result_size: number; ts: string }
|
||||
| { type: 'llm_turn'; model: string; tokens_in: number; tokens_out: number; ts: string }
|
||||
| { type: 'error'; message: string; stack?: string; ts: string };
|
||||
|
||||
// --- Errors ---
|
||||
|
||||
/** Throw this from a handler to skip all retry logic and go straight to 'dead'. */
|
||||
export class UnrecoverableError extends Error {
|
||||
constructor(message: string) {
|
||||
super(message);
|
||||
this.name = 'UnrecoverableError';
|
||||
}
|
||||
}
|
||||
|
||||
// --- Row Mapping ---
|
||||
|
||||
export function rowToMinionJob(row: Record<string, unknown>): MinionJob {
|
||||
return {
|
||||
id: row.id as number,
|
||||
name: row.name as string,
|
||||
queue: row.queue as string,
|
||||
status: row.status as MinionJobStatus,
|
||||
priority: row.priority as number,
|
||||
data: (typeof row.data === 'string' ? JSON.parse(row.data) : row.data ?? {}) as Record<string, unknown>,
|
||||
max_attempts: row.max_attempts as number,
|
||||
attempts_made: row.attempts_made as number,
|
||||
attempts_started: row.attempts_started as number,
|
||||
backoff_type: row.backoff_type as BackoffType,
|
||||
backoff_delay: row.backoff_delay as number,
|
||||
backoff_jitter: row.backoff_jitter as number,
|
||||
stalled_counter: row.stalled_counter as number,
|
||||
max_stalled: row.max_stalled as number,
|
||||
lock_token: (row.lock_token as string) || null,
|
||||
lock_until: row.lock_until ? new Date(row.lock_until as string) : null,
|
||||
delay_until: row.delay_until ? new Date(row.delay_until as string) : null,
|
||||
parent_job_id: (row.parent_job_id as number | null) ?? null,
|
||||
on_child_fail: row.on_child_fail as ChildFailPolicy,
|
||||
tokens_input: (row.tokens_input as number) ?? 0,
|
||||
tokens_output: (row.tokens_output as number) ?? 0,
|
||||
tokens_cache_read: (row.tokens_cache_read as number) ?? 0,
|
||||
depth: (row.depth as number) ?? 0,
|
||||
max_children: (row.max_children as number) ?? null,
|
||||
timeout_ms: (row.timeout_ms as number) ?? null,
|
||||
timeout_at: row.timeout_at ? new Date(row.timeout_at as string) : null,
|
||||
remove_on_complete: row.remove_on_complete === true,
|
||||
remove_on_fail: row.remove_on_fail === true,
|
||||
idempotency_key: (row.idempotency_key as string) || null,
|
||||
result: row.result ? (typeof row.result === 'string' ? JSON.parse(row.result) : row.result) as Record<string, unknown> : null,
|
||||
progress: row.progress ? (typeof row.progress === 'string' ? JSON.parse(row.progress) : row.progress) : null,
|
||||
error_text: (row.error_text as string) || null,
|
||||
stacktrace: row.stacktrace ? (typeof row.stacktrace === 'string' ? JSON.parse(row.stacktrace) : row.stacktrace) as string[] : [],
|
||||
created_at: new Date(row.created_at as string),
|
||||
started_at: row.started_at ? new Date(row.started_at as string) : null,
|
||||
finished_at: row.finished_at ? new Date(row.finished_at as string) : null,
|
||||
updated_at: new Date(row.updated_at as string),
|
||||
};
|
||||
}
|
||||
311
src/core/minions/worker.ts
Normal file
311
src/core/minions/worker.ts
Normal file
@@ -0,0 +1,311 @@
|
||||
/**
|
||||
* MinionWorker — Concurrent in-process job worker with BullMQ-inspired patterns.
|
||||
*
|
||||
* Processes up to `concurrency` jobs simultaneously using a Promise pool.
|
||||
* Each job gets its own AbortController, lock renewal timer, and isolated state.
|
||||
*
|
||||
* Usage:
|
||||
* const worker = new MinionWorker(engine);
|
||||
* worker.register('sync', async (job) => { ... });
|
||||
* worker.register('embed', async (job) => { ... });
|
||||
* await worker.start(); // polls until SIGTERM
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import type {
|
||||
MinionJob, MinionJobContext, MinionHandler, MinionWorkerOpts,
|
||||
MinionQueueOpts, TokenUpdate,
|
||||
} from './types.ts';
|
||||
import { UnrecoverableError } from './types.ts';
|
||||
import { MinionQueue } from './queue.ts';
|
||||
import { calculateBackoff } from './backoff.ts';
|
||||
import { randomUUID } from 'crypto';
|
||||
|
||||
/** Per-job in-flight state (isolated per job, not shared on the worker). */
|
||||
interface InFlightJob {
|
||||
job: MinionJob;
|
||||
lockToken: string;
|
||||
lockTimer: ReturnType<typeof setInterval>;
|
||||
abort: AbortController;
|
||||
promise: Promise<void>;
|
||||
}
|
||||
|
||||
export class MinionWorker {
|
||||
private queue: MinionQueue;
|
||||
private handlers = new Map<string, MinionHandler>();
|
||||
private running = false;
|
||||
private inFlight = new Map<number, InFlightJob>();
|
||||
private workerId = randomUUID();
|
||||
|
||||
private opts: Required<MinionWorkerOpts>;
|
||||
|
||||
constructor(
|
||||
private engine: BrainEngine,
|
||||
opts?: MinionWorkerOpts & MinionQueueOpts,
|
||||
) {
|
||||
this.queue = new MinionQueue(engine, {
|
||||
maxSpawnDepth: opts?.maxSpawnDepth,
|
||||
maxAttachmentBytes: opts?.maxAttachmentBytes,
|
||||
});
|
||||
this.opts = {
|
||||
queue: opts?.queue ?? 'default',
|
||||
concurrency: opts?.concurrency ?? 1,
|
||||
lockDuration: opts?.lockDuration ?? 30000,
|
||||
stalledInterval: opts?.stalledInterval ?? 30000,
|
||||
maxStalledCount: opts?.maxStalledCount ?? 1,
|
||||
pollInterval: opts?.pollInterval ?? 5000,
|
||||
};
|
||||
}
|
||||
|
||||
/** Register a handler for a job type. */
|
||||
register(name: string, handler: MinionHandler): void {
|
||||
this.handlers.set(name, handler);
|
||||
}
|
||||
|
||||
/** Get registered handler names (used by claim query). */
|
||||
get registeredNames(): string[] {
|
||||
return Array.from(this.handlers.keys());
|
||||
}
|
||||
|
||||
/** Start the worker loop. Blocks until stopped. */
|
||||
async start(): Promise<void> {
|
||||
if (this.handlers.size === 0) {
|
||||
throw new Error('No handlers registered. Call worker.register(name, handler) before start().');
|
||||
}
|
||||
|
||||
await this.queue.ensureSchema();
|
||||
this.running = true;
|
||||
|
||||
// Graceful shutdown
|
||||
const shutdown = () => {
|
||||
console.log('Minion worker shutting down...');
|
||||
this.running = false;
|
||||
};
|
||||
process.on('SIGTERM', shutdown);
|
||||
process.on('SIGINT', shutdown);
|
||||
|
||||
// Stall + timeout detection on interval. Order matters: handleStalled FIRST
|
||||
// so a stalled job (lock_until expired) gets requeued before handleTimeouts'
|
||||
// `lock_until > now()` guard would skip it. Stall → retry, timeout → dead.
|
||||
const stalledTimer = setInterval(async () => {
|
||||
try {
|
||||
const { requeued, dead } = await this.queue.handleStalled();
|
||||
if (requeued.length > 0) console.log(`Stall detector: requeued ${requeued.length} jobs`);
|
||||
if (dead.length > 0) console.log(`Stall detector: dead-lettered ${dead.length} jobs`);
|
||||
} catch (e) {
|
||||
console.error('Stall detection error:', e instanceof Error ? e.message : String(e));
|
||||
}
|
||||
try {
|
||||
const timedOut = await this.queue.handleTimeouts();
|
||||
if (timedOut.length > 0) console.log(`Timeout detector: dead-lettered ${timedOut.length} jobs (timeout exceeded)`);
|
||||
} catch (e) {
|
||||
console.error('Timeout detection error:', e instanceof Error ? e.message : String(e));
|
||||
}
|
||||
}, this.opts.stalledInterval);
|
||||
|
||||
try {
|
||||
while (this.running) {
|
||||
// Promote delayed jobs
|
||||
try {
|
||||
await this.queue.promoteDelayed();
|
||||
} catch (e) {
|
||||
console.error('Promotion error:', e instanceof Error ? e.message : String(e));
|
||||
}
|
||||
|
||||
// Claim jobs up to concurrency limit
|
||||
if (this.inFlight.size < this.opts.concurrency) {
|
||||
const lockToken = `${this.workerId}:${Date.now()}`;
|
||||
const job = await this.queue.claim(
|
||||
lockToken,
|
||||
this.opts.lockDuration,
|
||||
this.opts.queue,
|
||||
this.registeredNames,
|
||||
);
|
||||
|
||||
if (job) {
|
||||
this.launchJob(job, lockToken);
|
||||
} else if (this.inFlight.size === 0) {
|
||||
// No jobs and nothing in flight, poll
|
||||
await new Promise(resolve => setTimeout(resolve, this.opts.pollInterval));
|
||||
} else {
|
||||
// Jobs are running but no new ones available, brief pause before re-checking
|
||||
await new Promise(resolve => setTimeout(resolve, 100));
|
||||
}
|
||||
} else {
|
||||
// At concurrency limit, wait briefly before re-checking for free slots
|
||||
await new Promise(resolve => setTimeout(resolve, 100));
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
clearInterval(stalledTimer);
|
||||
process.removeListener('SIGTERM', shutdown);
|
||||
process.removeListener('SIGINT', shutdown);
|
||||
|
||||
// Graceful shutdown: wait for all in-flight jobs with timeout
|
||||
if (this.inFlight.size > 0) {
|
||||
console.log(`Waiting for ${this.inFlight.size} in-flight job(s) to finish (30s timeout)...`);
|
||||
const pending = Array.from(this.inFlight.values()).map(f => f.promise);
|
||||
await Promise.race([
|
||||
Promise.allSettled(pending),
|
||||
new Promise(resolve => setTimeout(resolve, 30000)),
|
||||
]);
|
||||
}
|
||||
|
||||
console.log('Minion worker stopped.');
|
||||
}
|
||||
}
|
||||
|
||||
/** Stop the worker gracefully. */
|
||||
stop(): void {
|
||||
this.running = false;
|
||||
}
|
||||
|
||||
/** Launch a job as an independent in-flight promise. */
|
||||
private launchJob(job: MinionJob, lockToken: string): void {
|
||||
const abort = new AbortController();
|
||||
|
||||
// Start lock renewal (per-job timer, not shared)
|
||||
const lockTimer = setInterval(async () => {
|
||||
const renewed = await this.queue.renewLock(job.id, lockToken, this.opts.lockDuration);
|
||||
if (!renewed) {
|
||||
console.warn(`Lock lost for job ${job.id}, aborting execution`);
|
||||
clearInterval(lockTimer);
|
||||
abort.abort();
|
||||
}
|
||||
}, this.opts.lockDuration / 2);
|
||||
|
||||
// Per-job wall-clock timeout safety net. Cooperative: fires abort() so the
|
||||
// handler's signal flips. Handlers ignoring AbortSignal can't be force-killed
|
||||
// from JS; the DB-side handleTimeouts is the authoritative status flip.
|
||||
// The .finally clearTimeout below ensures process exit isn't delayed by a
|
||||
// dangling timer on normal completion.
|
||||
let timeoutTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
if (job.timeout_ms != null) {
|
||||
timeoutTimer = setTimeout(() => {
|
||||
if (!abort.signal.aborted) {
|
||||
console.warn(`Job ${job.id} (${job.name}) hit per-job timeout (${job.timeout_ms}ms), aborting`);
|
||||
abort.abort();
|
||||
}
|
||||
}, job.timeout_ms);
|
||||
}
|
||||
|
||||
const promise = this.executeJob(job, lockToken, abort, lockTimer)
|
||||
.finally(() => {
|
||||
clearInterval(lockTimer);
|
||||
if (timeoutTimer) clearTimeout(timeoutTimer);
|
||||
this.inFlight.delete(job.id);
|
||||
});
|
||||
|
||||
this.inFlight.set(job.id, { job, lockToken, lockTimer, abort, promise });
|
||||
}
|
||||
|
||||
private async executeJob(
|
||||
job: MinionJob,
|
||||
lockToken: string,
|
||||
abort: AbortController,
|
||||
lockTimer: ReturnType<typeof setInterval>,
|
||||
): Promise<void> {
|
||||
const handler = this.handlers.get(job.name);
|
||||
if (!handler) {
|
||||
await this.queue.failJob(job.id, lockToken, `No handler for job type '${job.name}'`, 'dead');
|
||||
return;
|
||||
}
|
||||
|
||||
// Build job context with per-job AbortSignal
|
||||
const context: MinionJobContext = {
|
||||
id: job.id,
|
||||
name: job.name,
|
||||
data: job.data,
|
||||
attempts_made: job.attempts_made,
|
||||
signal: abort.signal,
|
||||
updateProgress: async (progress: unknown) => {
|
||||
await this.queue.updateProgress(job.id, lockToken, progress);
|
||||
},
|
||||
updateTokens: async (tokens: TokenUpdate) => {
|
||||
await this.queue.updateTokens(job.id, lockToken, tokens);
|
||||
},
|
||||
log: async (message: string | Record<string, unknown>) => {
|
||||
const value = typeof message === 'string' ? message : JSON.stringify(message);
|
||||
await this.engine.executeRaw(
|
||||
`UPDATE minion_jobs SET stacktrace = COALESCE(stacktrace, '[]'::jsonb) || to_jsonb($1::text),
|
||||
updated_at = now()
|
||||
WHERE id = $2 AND status = 'active' AND lock_token = $3`,
|
||||
[value, job.id, lockToken]
|
||||
);
|
||||
},
|
||||
isActive: async () => {
|
||||
const rows = await this.engine.executeRaw<{ id: number }>(
|
||||
`SELECT id FROM minion_jobs WHERE id = $1 AND status = 'active' AND lock_token = $2`,
|
||||
[job.id, lockToken]
|
||||
);
|
||||
return rows.length > 0;
|
||||
},
|
||||
readInbox: async () => {
|
||||
return this.queue.readInbox(job.id, lockToken);
|
||||
},
|
||||
};
|
||||
|
||||
try {
|
||||
const result = await handler(context);
|
||||
|
||||
clearInterval(lockTimer);
|
||||
|
||||
// Complete the job (token-fenced)
|
||||
const completed = await this.queue.completeJob(
|
||||
job.id,
|
||||
lockToken,
|
||||
result != null ? (typeof result === 'object' ? result as Record<string, unknown> : { value: result }) : undefined,
|
||||
);
|
||||
|
||||
if (!completed) {
|
||||
console.warn(`Job ${job.id} completion dropped (lock token mismatch, job was reclaimed)`);
|
||||
return;
|
||||
}
|
||||
// resolveParent is folded into queue.completeJob() (same transaction as
|
||||
// status flip + token rollup + child_done), so a process crash here can't
|
||||
// strand the parent in waiting-children.
|
||||
} catch (err) {
|
||||
clearInterval(lockTimer);
|
||||
|
||||
// If aborted (paused or lock lost), don't try to fail the job
|
||||
if (abort.signal.aborted) {
|
||||
console.log(`Job ${job.id} (${job.name}) aborted (paused or lock lost)`);
|
||||
return;
|
||||
}
|
||||
|
||||
const errorText = err instanceof Error ? err.message : String(err);
|
||||
const isUnrecoverable = err instanceof UnrecoverableError;
|
||||
const attemptsExhausted = job.attempts_made + 1 >= job.max_attempts;
|
||||
|
||||
let newStatus: 'delayed' | 'failed' | 'dead';
|
||||
if (isUnrecoverable || attemptsExhausted) {
|
||||
newStatus = 'dead';
|
||||
} else {
|
||||
newStatus = 'delayed';
|
||||
}
|
||||
|
||||
const backoffMs = newStatus === 'delayed' ? calculateBackoff({
|
||||
backoff_type: job.backoff_type,
|
||||
backoff_delay: job.backoff_delay,
|
||||
backoff_jitter: job.backoff_jitter,
|
||||
attempts_made: job.attempts_made + 1,
|
||||
}) : 0;
|
||||
|
||||
const failed = await this.queue.failJob(job.id, lockToken, errorText, newStatus, backoffMs);
|
||||
if (!failed) {
|
||||
console.warn(`Job ${job.id} failure dropped (lock token mismatch)`);
|
||||
return;
|
||||
}
|
||||
// Parent-failure hook (fail_parent / remove_dep / ignore / continue) is
|
||||
// folded into queue.failJob() in the same transaction as the child status
|
||||
// flip + remove_on_fail delete. Worker stays out of multi-statement
|
||||
// crash-window territory.
|
||||
|
||||
if (newStatus === 'delayed') {
|
||||
console.log(`Job ${job.id} (${job.name}) failed, retrying in ${Math.round(backoffMs)}ms (attempt ${job.attempts_made + 1}/${job.max_attempts})`);
|
||||
} else {
|
||||
console.log(`Job ${job.id} (${job.name}) permanently failed: ${errorText}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -763,6 +763,183 @@ const file_url: Operation = {
|
||||
},
|
||||
};
|
||||
|
||||
// --- Jobs (Minions) ---
|
||||
|
||||
const submit_job: Operation = {
|
||||
name: 'submit_job',
|
||||
description: 'Submit a background job to the Minions queue',
|
||||
params: {
|
||||
name: { type: 'string', required: true, description: 'Job type (sync, embed, lint, import)' },
|
||||
data: { type: 'object', description: 'Job payload (JSON)' },
|
||||
queue: { type: 'string', description: 'Queue name (default: "default")' },
|
||||
priority: { type: 'number', description: 'Priority (0 = highest, default: 0)' },
|
||||
max_attempts: { type: 'number', description: 'Max retry attempts (default: 3)' },
|
||||
delay: { type: 'number', description: 'Delay in ms before eligible' },
|
||||
},
|
||||
mutating: true,
|
||||
handler: async (ctx, p) => {
|
||||
if (ctx.dryRun) return { dry_run: true, action: 'submit_job', name: p.name };
|
||||
const { MinionQueue } = await import('./minions/queue.ts');
|
||||
const queue = new MinionQueue(ctx.engine);
|
||||
return queue.add(p.name as string, (p.data as Record<string, unknown>) || {}, {
|
||||
queue: (p.queue as string) || 'default',
|
||||
priority: (p.priority as number) || 0,
|
||||
max_attempts: (p.max_attempts as number) || 3,
|
||||
delay: (p.delay as number) || undefined,
|
||||
});
|
||||
},
|
||||
};
|
||||
|
||||
const get_job: Operation = {
|
||||
name: 'get_job',
|
||||
description: 'Get job status and details by ID',
|
||||
params: {
|
||||
id: { type: 'number', required: true, description: 'Job ID' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const { MinionQueue } = await import('./minions/queue.ts');
|
||||
const queue = new MinionQueue(ctx.engine);
|
||||
const job = await queue.getJob(p.id as number);
|
||||
if (!job) throw new OperationError('invalid_params', `Job not found: ${p.id}`);
|
||||
return job;
|
||||
},
|
||||
};
|
||||
|
||||
const list_jobs: Operation = {
|
||||
name: 'list_jobs',
|
||||
description: 'List jobs with optional filters',
|
||||
params: {
|
||||
status: { type: 'string', description: 'Filter by status (waiting, active, completed, failed, delayed, dead, cancelled)' },
|
||||
queue: { type: 'string', description: 'Filter by queue name' },
|
||||
name: { type: 'string', description: 'Filter by job type' },
|
||||
limit: { type: 'number', description: 'Max results (default: 50)' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const { MinionQueue } = await import('./minions/queue.ts');
|
||||
const queue = new MinionQueue(ctx.engine);
|
||||
return queue.getJobs({
|
||||
status: p.status as string | undefined,
|
||||
queue: p.queue as string | undefined,
|
||||
name: p.name as string | undefined,
|
||||
limit: (p.limit as number) || 50,
|
||||
} as Parameters<typeof queue.getJobs>[0]);
|
||||
},
|
||||
};
|
||||
|
||||
const cancel_job: Operation = {
|
||||
name: 'cancel_job',
|
||||
description: 'Cancel a waiting, active, or delayed job',
|
||||
params: {
|
||||
id: { type: 'number', required: true, description: 'Job ID' },
|
||||
},
|
||||
mutating: true,
|
||||
handler: async (ctx, p) => {
|
||||
if (ctx.dryRun) return { dry_run: true, action: 'cancel_job', id: p.id };
|
||||
const { MinionQueue } = await import('./minions/queue.ts');
|
||||
const queue = new MinionQueue(ctx.engine);
|
||||
const cancelled = await queue.cancelJob(p.id as number);
|
||||
if (!cancelled) throw new OperationError('invalid_params', `Cannot cancel job ${p.id} (may already be in terminal status)`);
|
||||
return cancelled;
|
||||
},
|
||||
};
|
||||
|
||||
const retry_job: Operation = {
|
||||
name: 'retry_job',
|
||||
description: 'Re-queue a failed or dead job for retry',
|
||||
params: {
|
||||
id: { type: 'number', required: true, description: 'Job ID' },
|
||||
},
|
||||
mutating: true,
|
||||
handler: async (ctx, p) => {
|
||||
if (ctx.dryRun) return { dry_run: true, action: 'retry_job', id: p.id };
|
||||
const { MinionQueue } = await import('./minions/queue.ts');
|
||||
const queue = new MinionQueue(ctx.engine);
|
||||
const retried = await queue.retryJob(p.id as number);
|
||||
if (!retried) throw new OperationError('invalid_params', `Cannot retry job ${p.id} (must be failed or dead)`);
|
||||
return retried;
|
||||
},
|
||||
};
|
||||
|
||||
const get_job_progress: Operation = {
|
||||
name: 'get_job_progress',
|
||||
description: 'Get structured progress for a running job',
|
||||
params: {
|
||||
id: { type: 'number', required: true, description: 'Job ID' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const { MinionQueue } = await import('./minions/queue.ts');
|
||||
const queue = new MinionQueue(ctx.engine);
|
||||
const job = await queue.getJob(p.id as number);
|
||||
if (!job) throw new OperationError('invalid_params', `Job not found: ${p.id}`);
|
||||
return { id: job.id, name: job.name, status: job.status, progress: job.progress };
|
||||
},
|
||||
};
|
||||
|
||||
const pause_job: Operation = {
|
||||
name: 'pause_job',
|
||||
description: 'Pause a waiting, active, or delayed job',
|
||||
params: {
|
||||
id: { type: 'number', required: true, description: 'Job ID' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const { MinionQueue } = await import('./minions/queue.ts');
|
||||
const queue = new MinionQueue(ctx.engine);
|
||||
const job = await queue.pauseJob(p.id as number);
|
||||
if (!job) throw new OperationError('invalid_params', `Job not found or not pausable: ${p.id}`);
|
||||
return { id: job.id, status: job.status };
|
||||
},
|
||||
};
|
||||
|
||||
const resume_job: Operation = {
|
||||
name: 'resume_job',
|
||||
description: 'Resume a paused job back to waiting',
|
||||
params: {
|
||||
id: { type: 'number', required: true, description: 'Job ID' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const { MinionQueue } = await import('./minions/queue.ts');
|
||||
const queue = new MinionQueue(ctx.engine);
|
||||
const job = await queue.resumeJob(p.id as number);
|
||||
if (!job) throw new OperationError('invalid_params', `Job not found or not paused: ${p.id}`);
|
||||
return { id: job.id, status: job.status };
|
||||
},
|
||||
};
|
||||
|
||||
const replay_job: Operation = {
|
||||
name: 'replay_job',
|
||||
description: 'Replay a completed/failed/dead job, optionally with modified data',
|
||||
params: {
|
||||
id: { type: 'number', required: true, description: 'Source job ID to replay' },
|
||||
data_overrides: { type: 'object', required: false, description: 'Data fields to override (merged with original)' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
if (ctx.dryRun) return { dry_run: true, action: 'replay_job', id: p.id };
|
||||
const { MinionQueue } = await import('./minions/queue.ts');
|
||||
const queue = new MinionQueue(ctx.engine);
|
||||
const job = await queue.replayJob(p.id as number, p.data_overrides as Record<string, unknown> | undefined);
|
||||
if (!job) throw new OperationError('invalid_params', `Job not found or not in terminal state: ${p.id}`);
|
||||
return { id: job.id, name: job.name, status: job.status, source_id: p.id };
|
||||
},
|
||||
};
|
||||
|
||||
const send_job_message: Operation = {
|
||||
name: 'send_job_message',
|
||||
description: 'Send a sidechannel message to a running job\'s inbox',
|
||||
params: {
|
||||
id: { type: 'number', required: true, description: 'Job ID to message' },
|
||||
payload: { type: 'object', required: true, description: 'Message payload (arbitrary JSON)' },
|
||||
sender: { type: 'string', required: false, description: 'Sender identity (default: admin)' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
if (ctx.dryRun) return { dry_run: true, action: 'send_job_message', id: p.id };
|
||||
const { MinionQueue } = await import('./minions/queue.ts');
|
||||
const queue = new MinionQueue(ctx.engine);
|
||||
const msg = await queue.sendMessage(p.id as number, p.payload, (p.sender as string) ?? 'admin');
|
||||
if (!msg) throw new OperationError('invalid_params', `Job not found, not messageable, or sender unauthorized: ${p.id}`);
|
||||
return { sent: true, message_id: msg.id, job_id: p.id };
|
||||
},
|
||||
};
|
||||
|
||||
// --- Exports ---
|
||||
|
||||
export const operations: Operation[] = [
|
||||
@@ -788,6 +965,9 @@ export const operations: Operation[] = [
|
||||
log_ingest, get_ingest_log,
|
||||
// Files
|
||||
file_list, file_upload, file_url,
|
||||
// Jobs (Minions)
|
||||
submit_job, get_job, list_jobs, cancel_job, retry_job, get_job_progress,
|
||||
pause_job, resume_job, replay_job, send_job_message,
|
||||
];
|
||||
|
||||
export const operationsByName = Object.fromEntries(
|
||||
|
||||
@@ -683,4 +683,9 @@ export class PGLiteEngine implements BrainEngine {
|
||||
);
|
||||
return (rows as Record<string, unknown>[]).map(r => rowToChunk(r, true));
|
||||
}
|
||||
|
||||
async executeRaw<T = Record<string, unknown>>(sql: string, params?: unknown[]): Promise<T[]> {
|
||||
const { rows } = await this.db.query(sql, params);
|
||||
return rows as T[];
|
||||
}
|
||||
}
|
||||
|
||||
@@ -159,6 +159,102 @@ INSERT INTO config (key, value) VALUES
|
||||
('chunk_strategy', 'semantic')
|
||||
ON CONFLICT (key) DO NOTHING;
|
||||
|
||||
-- ============================================================
|
||||
-- Minion Jobs: BullMQ-inspired Postgres-native job queue
|
||||
-- ============================================================
|
||||
CREATE TABLE IF NOT EXISTS minion_jobs (
|
||||
id SERIAL PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
queue TEXT NOT NULL DEFAULT 'default',
|
||||
status TEXT NOT NULL DEFAULT 'waiting',
|
||||
priority INTEGER NOT NULL DEFAULT 0,
|
||||
data JSONB NOT NULL DEFAULT '{}',
|
||||
max_attempts INTEGER NOT NULL DEFAULT 3,
|
||||
attempts_made INTEGER NOT NULL DEFAULT 0,
|
||||
attempts_started INTEGER NOT NULL DEFAULT 0,
|
||||
backoff_type TEXT NOT NULL DEFAULT 'exponential',
|
||||
backoff_delay INTEGER NOT NULL DEFAULT 1000,
|
||||
backoff_jitter REAL NOT NULL DEFAULT 0.2,
|
||||
stalled_counter INTEGER NOT NULL DEFAULT 0,
|
||||
max_stalled INTEGER NOT NULL DEFAULT 1,
|
||||
lock_token TEXT,
|
||||
lock_until TIMESTAMPTZ,
|
||||
delay_until TIMESTAMPTZ,
|
||||
parent_job_id INTEGER REFERENCES minion_jobs(id) ON DELETE SET NULL,
|
||||
on_child_fail TEXT NOT NULL DEFAULT 'fail_parent',
|
||||
tokens_input INTEGER NOT NULL DEFAULT 0,
|
||||
tokens_output INTEGER NOT NULL DEFAULT 0,
|
||||
tokens_cache_read INTEGER NOT NULL DEFAULT 0,
|
||||
depth INTEGER NOT NULL DEFAULT 0,
|
||||
max_children INTEGER,
|
||||
timeout_ms INTEGER,
|
||||
timeout_at TIMESTAMPTZ,
|
||||
remove_on_complete BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
remove_on_fail BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
idempotency_key TEXT,
|
||||
result JSONB,
|
||||
progress JSONB,
|
||||
error_text TEXT,
|
||||
stacktrace JSONB DEFAULT '[]',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
started_at TIMESTAMPTZ,
|
||||
finished_at TIMESTAMPTZ,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT chk_status CHECK (status IN ('waiting','active','completed','failed','delayed','dead','cancelled','waiting-children','paused')),
|
||||
CONSTRAINT chk_backoff_type CHECK (backoff_type IN ('fixed','exponential')),
|
||||
CONSTRAINT chk_on_child_fail CHECK (on_child_fail IN ('fail_parent','remove_dep','ignore','continue')),
|
||||
CONSTRAINT chk_jitter_range CHECK (backoff_jitter >= 0.0 AND backoff_jitter <= 1.0),
|
||||
CONSTRAINT chk_attempts_order CHECK (attempts_made <= attempts_started),
|
||||
CONSTRAINT chk_nonnegative CHECK (attempts_made >= 0 AND attempts_started >= 0 AND stalled_counter >= 0 AND max_attempts >= 1 AND max_stalled >= 0),
|
||||
CONSTRAINT chk_depth_nonnegative CHECK (depth >= 0),
|
||||
CONSTRAINT chk_max_children_positive CHECK (max_children IS NULL OR max_children > 0),
|
||||
CONSTRAINT chk_timeout_positive CHECK (timeout_ms IS NULL OR timeout_ms > 0)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_claim ON minion_jobs (queue, priority ASC, created_at ASC) WHERE status = 'waiting';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_status ON minion_jobs(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_stalled ON minion_jobs (lock_until) WHERE status = 'active';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_delayed ON minion_jobs (delay_until) WHERE status = 'delayed';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_parent ON minion_jobs(parent_job_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_timeout ON minion_jobs (timeout_at)
|
||||
WHERE status = 'active' AND timeout_at IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_parent_status ON minion_jobs (parent_job_id, status)
|
||||
WHERE parent_job_id IS NOT NULL;
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS uniq_minion_jobs_idempotency ON minion_jobs (idempotency_key)
|
||||
WHERE idempotency_key IS NOT NULL;
|
||||
|
||||
-- Inbox table for sidechannel messaging
|
||||
CREATE TABLE IF NOT EXISTS minion_inbox (
|
||||
id SERIAL PRIMARY KEY,
|
||||
job_id INTEGER NOT NULL REFERENCES minion_jobs(id) ON DELETE CASCADE,
|
||||
sender TEXT NOT NULL,
|
||||
payload JSONB NOT NULL,
|
||||
sent_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
read_at TIMESTAMPTZ
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_inbox_unread ON minion_inbox (job_id) WHERE read_at IS NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_inbox_child_done ON minion_inbox (job_id, sent_at)
|
||||
WHERE (payload->>'type') = 'child_done';
|
||||
|
||||
-- Attachment manifest (BYTEA inline + forward-compat storage_uri)
|
||||
CREATE TABLE IF NOT EXISTS minion_attachments (
|
||||
id SERIAL PRIMARY KEY,
|
||||
job_id INTEGER NOT NULL REFERENCES minion_jobs(id) ON DELETE CASCADE,
|
||||
filename TEXT NOT NULL,
|
||||
content_type TEXT NOT NULL,
|
||||
content BYTEA,
|
||||
storage_uri TEXT,
|
||||
size_bytes INTEGER NOT NULL,
|
||||
sha256 TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT uniq_minion_attachments_job_filename UNIQUE (job_id, filename),
|
||||
CONSTRAINT chk_attachment_storage CHECK (content IS NOT NULL OR storage_uri IS NOT NULL),
|
||||
CONSTRAINT chk_attachment_size CHECK (size_bytes >= 0)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_attachments_job ON minion_attachments (job_id);
|
||||
-- NOTE: SET STORAGE EXTERNAL is omitted on PGLite; it's a Postgres TOAST optimization
|
||||
-- and PGLite may not support it. Postgres path applies it via migration v7.
|
||||
|
||||
-- ============================================================
|
||||
-- Trigger-based search_vector (spans pages + timeline_entries)
|
||||
-- ============================================================
|
||||
|
||||
@@ -729,4 +729,9 @@ export class PostgresEngine implements BrainEngine {
|
||||
`;
|
||||
return rows.map((r: Record<string, unknown>) => rowToChunk(r, true));
|
||||
}
|
||||
|
||||
async executeRaw<T = Record<string, unknown>>(sql: string, params?: unknown[]): Promise<T[]> {
|
||||
const conn = this.sql;
|
||||
return conn.unsafe(sql, params) as unknown as T[];
|
||||
}
|
||||
}
|
||||
|
||||
142
src/core/preferences.ts
Normal file
142
src/core/preferences.ts
Normal file
@@ -0,0 +1,142 @@
|
||||
/**
|
||||
* ~/.gbrain/preferences.json — user-facing agent-behavior flags (minion_mode, etc.).
|
||||
*
|
||||
* Separate from src/core/config.ts (engine config), written to its own file so
|
||||
* engine config and agent preferences can evolve independently. Atomic writes
|
||||
* via mktemp + rename; 0o600 perms; forward-compatible (preserves unknown keys).
|
||||
*
|
||||
* Also houses ~/.gbrain/migrations/completed.jsonl append helper.
|
||||
*/
|
||||
|
||||
import { readFileSync, writeFileSync, renameSync, chmodSync, mkdtempSync, rmSync, existsSync, mkdirSync, appendFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { homedir } from 'os';
|
||||
|
||||
function home(): string {
|
||||
// `os.homedir()` in Bun caches its initial value and ignores later
|
||||
// `process.env.HOME` mutations, which breaks test isolation and any
|
||||
// workflow that needs to run against a specific $HOME (CI, scripted installs).
|
||||
// Prefer the env var; fall back to the cached OS value. Matches the existing
|
||||
// `src/commands/upgrade.ts` pattern.
|
||||
return process.env.HOME || homedir();
|
||||
}
|
||||
|
||||
export type MinionMode = 'always' | 'pain_triggered' | 'off';
|
||||
|
||||
export interface Preferences {
|
||||
minion_mode?: MinionMode;
|
||||
set_at?: string;
|
||||
set_in_version?: string;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
export interface CompletedMigrationEntry {
|
||||
version: string;
|
||||
ts?: string;
|
||||
status: 'complete' | 'partial';
|
||||
mode?: MinionMode;
|
||||
files_rewritten?: number;
|
||||
autopilot_installed?: boolean;
|
||||
install_target?: string;
|
||||
apply_migrations_pending?: boolean;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
const VALID_MODES: ReadonlyArray<MinionMode> = ['always', 'pain_triggered', 'off'];
|
||||
|
||||
function prefsDir(): string { return join(home(), '.gbrain'); }
|
||||
function prefsPath(): string { return join(prefsDir(), 'preferences.json'); }
|
||||
function migrationsDir(): string { return join(home(), '.gbrain', 'migrations'); }
|
||||
function completedJsonlPath(): string { return join(migrationsDir(), 'completed.jsonl'); }
|
||||
|
||||
/** Validate that a value is a recognized minion mode. Throws with the allowed list. */
|
||||
export function validateMinionMode(value: unknown): asserts value is MinionMode {
|
||||
if (typeof value !== 'string' || !VALID_MODES.includes(value as MinionMode)) {
|
||||
throw new Error(`Invalid minion_mode "${String(value)}". Allowed: ${VALID_MODES.join(', ')}.`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Load preferences. Returns {} when the file is missing (not null — callers
|
||||
* can always treat the result as a Preferences object).
|
||||
*
|
||||
* Malformed JSON throws; caller can catch if they want graceful fallback.
|
||||
*/
|
||||
export function loadPreferences(): Preferences {
|
||||
const path = prefsPath();
|
||||
if (!existsSync(path)) return {};
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const parsed = JSON.parse(raw) as Preferences;
|
||||
return parsed;
|
||||
}
|
||||
|
||||
/**
|
||||
* Save preferences atomically (mktemp on same filesystem + rename). Preserves
|
||||
* any unknown keys passed in. Chmods 0o600 after write.
|
||||
*/
|
||||
export function savePreferences(prefs: Preferences): void {
|
||||
if (prefs.minion_mode !== undefined) validateMinionMode(prefs.minion_mode);
|
||||
|
||||
const dir = prefsDir();
|
||||
mkdirSync(dir, { recursive: true });
|
||||
|
||||
// Write via a tempfile on the same filesystem, then rename. Avoids the
|
||||
// "reader sees a half-written file" window that write-in-place has.
|
||||
const tmpDirForWrite = mkdtempSync(join(dir, '.prefs-tmp-'));
|
||||
const tmpPath = join(tmpDirForWrite, 'preferences.json');
|
||||
try {
|
||||
writeFileSync(tmpPath, JSON.stringify(prefs, null, 2) + '\n', { mode: 0o600 });
|
||||
try { chmodSync(tmpPath, 0o600); } catch { /* chmod may fail on some platforms */ }
|
||||
renameSync(tmpPath, prefsPath());
|
||||
} finally {
|
||||
try { rmSync(tmpDirForWrite, { recursive: true, force: true }); } catch { /* best-effort */ }
|
||||
}
|
||||
try { chmodSync(prefsPath(), 0o600); } catch { /* best-effort */ }
|
||||
}
|
||||
|
||||
/**
|
||||
* Append one line to ~/.gbrain/migrations/completed.jsonl. Creates the
|
||||
* directory if missing. Does not read existing lines (append is cheap and
|
||||
* the reader tolerates malformed lines by skipping them).
|
||||
*
|
||||
* Writes `ts` as the current ISO timestamp if not provided.
|
||||
*/
|
||||
export function appendCompletedMigration(entry: CompletedMigrationEntry): void {
|
||||
if (!entry.version) throw new Error('appendCompletedMigration: version required');
|
||||
if (entry.status !== 'complete' && entry.status !== 'partial') {
|
||||
throw new Error(`appendCompletedMigration: status must be 'complete' or 'partial', got "${entry.status}"`);
|
||||
}
|
||||
const full: CompletedMigrationEntry = {
|
||||
ts: new Date().toISOString(),
|
||||
...entry,
|
||||
};
|
||||
const dir = migrationsDir();
|
||||
mkdirSync(dir, { recursive: true });
|
||||
appendFileSync(completedJsonlPath(), JSON.stringify(full) + '\n');
|
||||
}
|
||||
|
||||
/** Read the completed.jsonl file, skipping malformed lines with a warning to stderr. */
|
||||
export function loadCompletedMigrations(): CompletedMigrationEntry[] {
|
||||
const path = completedJsonlPath();
|
||||
if (!existsSync(path)) return [];
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const out: CompletedMigrationEntry[] = [];
|
||||
for (const line of raw.split('\n')) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) continue;
|
||||
try {
|
||||
out.push(JSON.parse(trimmed) as CompletedMigrationEntry);
|
||||
} catch (err) {
|
||||
console.warn(`[preferences] skipping malformed completed.jsonl line: ${trimmed.slice(0, 120)}`);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Paths — exported for tests and rare consumers. */
|
||||
export const preferencesPaths = {
|
||||
dir: prefsDir,
|
||||
file: prefsPath,
|
||||
migrationsDir,
|
||||
completedJsonl: completedJsonlPath,
|
||||
};
|
||||
@@ -6,6 +6,8 @@ export const SCHEMA_SQL = `
|
||||
|
||||
CREATE EXTENSION IF NOT EXISTS vector;
|
||||
CREATE EXTENSION IF NOT EXISTS pg_trgm;
|
||||
-- gen_random_uuid() is core in Postgres 13+; enable pgcrypto as fallback for older versions
|
||||
CREATE EXTENSION IF NOT EXISTS pgcrypto;
|
||||
|
||||
-- ============================================================
|
||||
-- pages: the core content table
|
||||
@@ -248,6 +250,112 @@ CREATE TRIGGER trg_timeline_search_vector
|
||||
FOR EACH ROW
|
||||
EXECUTE FUNCTION update_page_search_vector_from_timeline();
|
||||
|
||||
-- ============================================================
|
||||
-- Minion Jobs: BullMQ-inspired Postgres-native job queue
|
||||
-- ============================================================
|
||||
CREATE TABLE IF NOT EXISTS minion_jobs (
|
||||
id SERIAL PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
queue TEXT NOT NULL DEFAULT 'default',
|
||||
status TEXT NOT NULL DEFAULT 'waiting',
|
||||
priority INTEGER NOT NULL DEFAULT 0,
|
||||
data JSONB NOT NULL DEFAULT '{}',
|
||||
max_attempts INTEGER NOT NULL DEFAULT 3,
|
||||
attempts_made INTEGER NOT NULL DEFAULT 0,
|
||||
attempts_started INTEGER NOT NULL DEFAULT 0,
|
||||
backoff_type TEXT NOT NULL DEFAULT 'exponential',
|
||||
backoff_delay INTEGER NOT NULL DEFAULT 1000,
|
||||
backoff_jitter REAL NOT NULL DEFAULT 0.2,
|
||||
stalled_counter INTEGER NOT NULL DEFAULT 0,
|
||||
max_stalled INTEGER NOT NULL DEFAULT 1,
|
||||
lock_token TEXT,
|
||||
lock_until TIMESTAMPTZ,
|
||||
delay_until TIMESTAMPTZ,
|
||||
parent_job_id INTEGER REFERENCES minion_jobs(id) ON DELETE SET NULL,
|
||||
on_child_fail TEXT NOT NULL DEFAULT 'fail_parent',
|
||||
tokens_input INTEGER NOT NULL DEFAULT 0,
|
||||
tokens_output INTEGER NOT NULL DEFAULT 0,
|
||||
tokens_cache_read INTEGER NOT NULL DEFAULT 0,
|
||||
result JSONB,
|
||||
progress JSONB,
|
||||
error_text TEXT,
|
||||
stacktrace JSONB DEFAULT '[]',
|
||||
depth INTEGER NOT NULL DEFAULT 0,
|
||||
max_children INTEGER,
|
||||
timeout_ms INTEGER,
|
||||
timeout_at TIMESTAMPTZ,
|
||||
remove_on_complete BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
remove_on_fail BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
idempotency_key TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
started_at TIMESTAMPTZ,
|
||||
finished_at TIMESTAMPTZ,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT chk_status CHECK (status IN ('waiting','active','completed','failed','delayed','dead','cancelled','waiting-children','paused')),
|
||||
CONSTRAINT chk_backoff_type CHECK (backoff_type IN ('fixed','exponential')),
|
||||
CONSTRAINT chk_on_child_fail CHECK (on_child_fail IN ('fail_parent','remove_dep','ignore','continue')),
|
||||
CONSTRAINT chk_jitter_range CHECK (backoff_jitter >= 0.0 AND backoff_jitter <= 1.0),
|
||||
CONSTRAINT chk_attempts_order CHECK (attempts_made <= attempts_started),
|
||||
CONSTRAINT chk_nonnegative CHECK (attempts_made >= 0 AND attempts_started >= 0 AND stalled_counter >= 0 AND max_attempts >= 1 AND max_stalled >= 0),
|
||||
CONSTRAINT chk_depth_nonnegative CHECK (depth >= 0),
|
||||
CONSTRAINT chk_max_children_positive CHECK (max_children IS NULL OR max_children > 0),
|
||||
CONSTRAINT chk_timeout_positive CHECK (timeout_ms IS NULL OR timeout_ms > 0)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_claim ON minion_jobs (queue, priority ASC, created_at ASC) WHERE status = 'waiting';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_status ON minion_jobs(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_stalled ON minion_jobs (lock_until) WHERE status = 'active';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_delayed ON minion_jobs (delay_until) WHERE status = 'delayed';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_parent ON minion_jobs(parent_job_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_timeout ON minion_jobs (timeout_at) WHERE status = 'active' AND timeout_at IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_parent_status ON minion_jobs (parent_job_id, status) WHERE parent_job_id IS NOT NULL;
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS uniq_minion_jobs_idempotency ON minion_jobs (idempotency_key) WHERE idempotency_key IS NOT NULL;
|
||||
|
||||
-- Inbox table for sidechannel messaging
|
||||
CREATE TABLE IF NOT EXISTS minion_inbox (
|
||||
id SERIAL PRIMARY KEY,
|
||||
job_id INTEGER NOT NULL REFERENCES minion_jobs(id) ON DELETE CASCADE,
|
||||
sender TEXT NOT NULL,
|
||||
payload JSONB NOT NULL,
|
||||
sent_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
read_at TIMESTAMPTZ
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_inbox_unread ON minion_inbox (job_id) WHERE read_at IS NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_inbox_child_done ON minion_inbox (job_id, sent_at) WHERE payload->>'type' = 'child_done';
|
||||
|
||||
-- Attachments table: per-job binary blobs (manifests, agent outputs, files)
|
||||
CREATE TABLE IF NOT EXISTS minion_attachments (
|
||||
id SERIAL PRIMARY KEY,
|
||||
job_id INTEGER NOT NULL REFERENCES minion_jobs(id) ON DELETE CASCADE,
|
||||
filename TEXT NOT NULL,
|
||||
content_type TEXT NOT NULL,
|
||||
content BYTEA,
|
||||
storage_uri TEXT,
|
||||
size_bytes INTEGER NOT NULL,
|
||||
sha256 TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT uniq_minion_attachments_job_filename UNIQUE (job_id, filename),
|
||||
CONSTRAINT chk_attachment_storage CHECK (content IS NOT NULL OR storage_uri IS NOT NULL),
|
||||
CONSTRAINT chk_attachment_size CHECK (size_bytes >= 0)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_attachments_job ON minion_attachments (job_id);
|
||||
ALTER TABLE minion_attachments ALTER COLUMN content SET STORAGE EXTERNAL;
|
||||
|
||||
-- NOTIFY trigger for real-time job events (Postgres only, not PGLite)
|
||||
CREATE OR REPLACE FUNCTION notify_minion_job_change() RETURNS trigger AS \$\$
|
||||
BEGIN
|
||||
PERFORM pg_notify('minion_jobs', json_build_object(
|
||||
'id', NEW.id, 'status', NEW.status, 'name', NEW.name,
|
||||
'queue', NEW.queue, 'prev_status', COALESCE(OLD.status, 'new')
|
||||
)::text);
|
||||
RETURN NEW;
|
||||
END;
|
||||
\$\$ LANGUAGE plpgsql;
|
||||
|
||||
DROP TRIGGER IF EXISTS minion_job_notify ON minion_jobs;
|
||||
CREATE TRIGGER minion_job_notify AFTER INSERT OR UPDATE OF status ON minion_jobs
|
||||
FOR EACH ROW EXECUTE FUNCTION notify_minion_job_change();
|
||||
|
||||
-- ============================================================
|
||||
-- Row Level Security: block anon access, postgres role bypasses
|
||||
-- ============================================================
|
||||
@@ -271,6 +379,7 @@ BEGIN
|
||||
ALTER TABLE ingest_log ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE config ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE files ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE minion_jobs ENABLE ROW LEVEL SECURITY;
|
||||
RAISE NOTICE 'RLS enabled on all tables (role % has BYPASSRLS)', current_user;
|
||||
ELSE
|
||||
RAISE WARNING 'Skipping RLS: role % does not have BYPASSRLS privilege. Run as postgres role to enable.', current_user;
|
||||
|
||||
107
src/schema.sql
107
src/schema.sql
@@ -246,6 +246,112 @@ CREATE TRIGGER trg_timeline_search_vector
|
||||
FOR EACH ROW
|
||||
EXECUTE FUNCTION update_page_search_vector_from_timeline();
|
||||
|
||||
-- ============================================================
|
||||
-- Minion Jobs: BullMQ-inspired Postgres-native job queue
|
||||
-- ============================================================
|
||||
CREATE TABLE IF NOT EXISTS minion_jobs (
|
||||
id SERIAL PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
queue TEXT NOT NULL DEFAULT 'default',
|
||||
status TEXT NOT NULL DEFAULT 'waiting',
|
||||
priority INTEGER NOT NULL DEFAULT 0,
|
||||
data JSONB NOT NULL DEFAULT '{}',
|
||||
max_attempts INTEGER NOT NULL DEFAULT 3,
|
||||
attempts_made INTEGER NOT NULL DEFAULT 0,
|
||||
attempts_started INTEGER NOT NULL DEFAULT 0,
|
||||
backoff_type TEXT NOT NULL DEFAULT 'exponential',
|
||||
backoff_delay INTEGER NOT NULL DEFAULT 1000,
|
||||
backoff_jitter REAL NOT NULL DEFAULT 0.2,
|
||||
stalled_counter INTEGER NOT NULL DEFAULT 0,
|
||||
max_stalled INTEGER NOT NULL DEFAULT 1,
|
||||
lock_token TEXT,
|
||||
lock_until TIMESTAMPTZ,
|
||||
delay_until TIMESTAMPTZ,
|
||||
parent_job_id INTEGER REFERENCES minion_jobs(id) ON DELETE SET NULL,
|
||||
on_child_fail TEXT NOT NULL DEFAULT 'fail_parent',
|
||||
tokens_input INTEGER NOT NULL DEFAULT 0,
|
||||
tokens_output INTEGER NOT NULL DEFAULT 0,
|
||||
tokens_cache_read INTEGER NOT NULL DEFAULT 0,
|
||||
result JSONB,
|
||||
progress JSONB,
|
||||
error_text TEXT,
|
||||
stacktrace JSONB DEFAULT '[]',
|
||||
depth INTEGER NOT NULL DEFAULT 0,
|
||||
max_children INTEGER,
|
||||
timeout_ms INTEGER,
|
||||
timeout_at TIMESTAMPTZ,
|
||||
remove_on_complete BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
remove_on_fail BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
idempotency_key TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
started_at TIMESTAMPTZ,
|
||||
finished_at TIMESTAMPTZ,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT chk_status CHECK (status IN ('waiting','active','completed','failed','delayed','dead','cancelled','waiting-children','paused')),
|
||||
CONSTRAINT chk_backoff_type CHECK (backoff_type IN ('fixed','exponential')),
|
||||
CONSTRAINT chk_on_child_fail CHECK (on_child_fail IN ('fail_parent','remove_dep','ignore','continue')),
|
||||
CONSTRAINT chk_jitter_range CHECK (backoff_jitter >= 0.0 AND backoff_jitter <= 1.0),
|
||||
CONSTRAINT chk_attempts_order CHECK (attempts_made <= attempts_started),
|
||||
CONSTRAINT chk_nonnegative CHECK (attempts_made >= 0 AND attempts_started >= 0 AND stalled_counter >= 0 AND max_attempts >= 1 AND max_stalled >= 0),
|
||||
CONSTRAINT chk_depth_nonnegative CHECK (depth >= 0),
|
||||
CONSTRAINT chk_max_children_positive CHECK (max_children IS NULL OR max_children > 0),
|
||||
CONSTRAINT chk_timeout_positive CHECK (timeout_ms IS NULL OR timeout_ms > 0)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_claim ON minion_jobs (queue, priority ASC, created_at ASC) WHERE status = 'waiting';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_status ON minion_jobs(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_stalled ON minion_jobs (lock_until) WHERE status = 'active';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_delayed ON minion_jobs (delay_until) WHERE status = 'delayed';
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_parent ON minion_jobs(parent_job_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_timeout ON minion_jobs (timeout_at) WHERE status = 'active' AND timeout_at IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_jobs_parent_status ON minion_jobs (parent_job_id, status) WHERE parent_job_id IS NOT NULL;
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS uniq_minion_jobs_idempotency ON minion_jobs (idempotency_key) WHERE idempotency_key IS NOT NULL;
|
||||
|
||||
-- Inbox table for sidechannel messaging
|
||||
CREATE TABLE IF NOT EXISTS minion_inbox (
|
||||
id SERIAL PRIMARY KEY,
|
||||
job_id INTEGER NOT NULL REFERENCES minion_jobs(id) ON DELETE CASCADE,
|
||||
sender TEXT NOT NULL,
|
||||
payload JSONB NOT NULL,
|
||||
sent_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
read_at TIMESTAMPTZ
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_inbox_unread ON minion_inbox (job_id) WHERE read_at IS NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_inbox_child_done ON minion_inbox (job_id, sent_at) WHERE payload->>'type' = 'child_done';
|
||||
|
||||
-- Attachments table: per-job binary blobs (manifests, agent outputs, files)
|
||||
CREATE TABLE IF NOT EXISTS minion_attachments (
|
||||
id SERIAL PRIMARY KEY,
|
||||
job_id INTEGER NOT NULL REFERENCES minion_jobs(id) ON DELETE CASCADE,
|
||||
filename TEXT NOT NULL,
|
||||
content_type TEXT NOT NULL,
|
||||
content BYTEA,
|
||||
storage_uri TEXT,
|
||||
size_bytes INTEGER NOT NULL,
|
||||
sha256 TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT uniq_minion_attachments_job_filename UNIQUE (job_id, filename),
|
||||
CONSTRAINT chk_attachment_storage CHECK (content IS NOT NULL OR storage_uri IS NOT NULL),
|
||||
CONSTRAINT chk_attachment_size CHECK (size_bytes >= 0)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_minion_attachments_job ON minion_attachments (job_id);
|
||||
ALTER TABLE minion_attachments ALTER COLUMN content SET STORAGE EXTERNAL;
|
||||
|
||||
-- NOTIFY trigger for real-time job events (Postgres only, not PGLite)
|
||||
CREATE OR REPLACE FUNCTION notify_minion_job_change() RETURNS trigger AS $$
|
||||
BEGIN
|
||||
PERFORM pg_notify('minion_jobs', json_build_object(
|
||||
'id', NEW.id, 'status', NEW.status, 'name', NEW.name,
|
||||
'queue', NEW.queue, 'prev_status', COALESCE(OLD.status, 'new')
|
||||
)::text);
|
||||
RETURN NEW;
|
||||
END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
|
||||
DROP TRIGGER IF EXISTS minion_job_notify ON minion_jobs;
|
||||
CREATE TRIGGER minion_job_notify AFTER INSERT OR UPDATE OF status ON minion_jobs
|
||||
FOR EACH ROW EXECUTE FUNCTION notify_minion_job_change();
|
||||
|
||||
-- ============================================================
|
||||
-- Row Level Security: block anon access, postgres role bypasses
|
||||
-- ============================================================
|
||||
@@ -269,6 +375,7 @@ BEGIN
|
||||
ALTER TABLE ingest_log ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE config ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE files ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE minion_jobs ENABLE ROW LEVEL SECURITY;
|
||||
RAISE NOTICE 'RLS enabled on all tables (role % has BYPASSRLS)', current_user;
|
||||
ELSE
|
||||
RAISE WARNING 'Skipping RLS: role % does not have BYPASSRLS privilege. Run as postgres role to enable.', current_user;
|
||||
|
||||
Reference in New Issue
Block a user