4 Commits

Author	SHA1	Message	Date
Garry Tan	0e9f8814a5	feat: v0.16.0 — durable agent runtime (gbrain agent + subagent handler + plugin loader) (#258) ... * refactor(mcp): extract buildToolDefs helper for subagent tool registry reuse The inline operations.map(...) block in src/mcp/server.ts became the only source of truth for agent-facing tool definitions. Extract into a reusable exported helper so the v0.15 subagent tool registry can call it with a filtered OPERATIONS subset instead of duplicating the shape. Byte-for-byte equivalence regression pinned in test/mcp-tool-defs.test.ts — legacy inline mapping kept verbatim inside the test so any future drift between the new helper and the pre-extraction MCP schema fails loudly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(operations): subagent-aware OperationContext + put_page namespace Adds three optional fields to OperationContext: - jobId?: number — the currently running Minion job id - subagentId?: number — the owning subagent job id for tool-dispatched calls - viaSubagent?: boolean — FAIL-CLOSED flag for agent-path gating put_page now enforces a namespace rule when invoked on the subagent tool dispatch path (viaSubagent=true): writes MUST target `wiki/agents/<subagentId>/...`. Anchored, slash-boundary enforced so a collision like `wiki/agents/12evil/...` can't impersonate subagent 12. The check runs BEFORE the dry-run short-circuit so preview calls surface the same rejection. Fail-closed: a missing subagentId with viaSubagent=true rejects every slug rather than letting a dispatcher bug open a hole. Existing callers unaffected — all three fields are optional and the legacy put_page behavior is unchanged when viaSubagent is undefined/false. 12 regression + namespace tests pin: - local CLI writes (viaSubagent unset) accept arbitrary slugs - MCP writes (remote=true, viaSubagent unset) accept arbitrary slugs - subagent-path: anchored prefix accepted, wrong id rejected, prefix- collision defeated, leading-slash rejected, bare-prefix rejected, fail-closed on missing/NaN subagentId, permission_denied code emitted Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(schema): v0.15.0 subagent runtime tables + migration orchestrator Adds three new tables for the durable LLM agent runtime: subagent_messages — Anthropic message-block persistence. Parallel tool_use blocks in one assistant message live in content_blocks JSONB, not across rows (fixes the (job_id, turn_idx, role) misdesign codex caught in v0.13 drafting). subagent_tool_executions — Two-phase tool ledger. INSERT pending before execute, UPDATE complete/failed after. Replay re-runs pending rows only if the tool is idempotent (v1 ships only idempotent tools so this is preventive). subagent_rate_leases — Lease-based concurrency cap for outbound providers (e.g. anthropic:messages). Stale leases auto-prune on next acquire so crashed workers can't strand capacity. All DDL uses CREATE TABLE/INDEX IF NOT EXISTS — order-independent vs PR #244's initSchema() reorder, and idempotent across fresh-install + upgrade paths. Shipped in both src/schema.sql (Postgres) and src/core/pglite-schema.ts (PGLite); schema-embedded.ts regenerated. Migration orchestrator v0_15_0.ts (phases: schema → verify → record). v0_14_0.ts is a no-op stub so the registry's version sequence stays gapless (v0.14.0 shipped shell-jobs — code change, no DB migration). 10 unit tests for registry wiring, ordering, dry-run phase behavior, and schema-embedded table presence. test/apply-migrations.test.ts updated for the two new registry entries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): emit child_done on every terminal + max_stalled per-job + terminal set fix Three correctness fixes the v0.15 subagent aggregator spine depends on: 1. child_done emission on ALL terminal transitions, not just success. - completeJob already emitted on success — now also tags outcome='complete'. - failJob newly emits on terminal 'failed' or 'dead' (outcome='failed'|'dead', error=<text>), BEFORE the parent-terminal UPDATE so the EXISTS guard on the inbox INSERT doesn't skip it on fail_parent paths (codex catch). - cancelJob now emits outcome='cancelled' per descendant with a parent. - handleTimeouts now emits outcome='timeout' per timed-out child. ChildDoneMessage gains optional { outcome, error } — backwards compatible (legacy writers omitted them; consumers treat absent outcome as 'complete'). 2. Parent-resolution terminal set now includes 'failed'. Pre-v0.15 the `NOT EXISTS (... status NOT IN ('completed','dead','cancelled'))` guard treated a failed child as still-pending, stranding aggregator parents that chose on_child_fail='continue' or 'ignore' in waiting-children forever. Expanded to {completed, failed, dead, cancelled} everywhere parent resolution reads child status (completeJob inline, failJob remove_dep + continue, cancelJob sweep, handleTimeouts sweep, and the resolveParent method itself). 3. MinionJobInput.max_stalled threads through MinionQueue.add() on INSERT. Column exists with default 1 — that is "first stall → dead", which defeats crash recovery for long-running handlers. Subagent children will set max_stalled: 3 to survive mid-run worker kills. Second-submitter under an idempotency-key hit does NOT mutate the existing row (codex-flagged footgun — first-submit options are load-bearing state). 13 unit tests pin: emission on each of completeJob/failJob/cancelJob/ handleTimeouts, insertion order on fail_parent, terminal-set expansion with continue policy, max_stalled default + override + idempotency behavior. E2E tier 1 (Postgres) passes 141 tests unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): rate-leases + waitForCompletion infra for v0.15 subagent Two infrastructure modules the subagent handler spine depends on: rate-leases.ts — lease-based concurrency cap for outbound providers (anthropic:messages, openai:*, etc.). Counter-based limiters leak capacity on worker crash; leases are owner-tagged rows with expires_at that auto-prune on the next acquire. Two-phase: txn-scoped pg_advisory_xact_lock guards the check-then-insert so concurrent acquires can't both win the "last slot". renewLeaseWithBackoff retries 3x (250/500/1000ms) for mid- call DB blips — on persistent failure the LLM-loop caller aborts with a renewable error so the worker re-claims and the rate invariant is preserved. Owner FK cascades clean up leases on job deletion. wait-for-completion.ts — poll-until-terminal helper for CLI callers. Minions' NOTIFY is worker-side only; `gbrain agent run --follow` polls getJob() until status is {completed, failed, dead, cancelled}. TimeoutError carries jobId + elapsedMs and does NOT cancel the job — the user can inspect via `gbrain jobs get <id>` later. Supports AbortSignal for Ctrl-C without throwing. Default pollMs is 1000 on Postgres, 250 on PGLite (inline CLI has no network RTT). 21 unit tests cover: single/multi acquire under cap, rejection past cap, release frees slot, different keys are independent, stale prune, cascade on owner delete, renew bumps expires_at, renew on missing is false, backoff path success + pruned short-circuit. waitForCompletion: fast-path terminal, transitions mid-wait (completed/failed/cancelled), TimeoutError shape, abort-signal early exit, non-existent job error. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): subagent ToolDef types + brain-tool registry (v0.15) Types first so the handler has a stable contract: - SubagentHandlerData / AggregatorHandlerData — the two job.data shapes - ToolCtx (engine, jobId, remote, signal) + ToolDef (name, description, input_schema, idempotent, execute) — Anthropic-envelope, distinct from the MCP McpToolDef extraction landed earlier - ContentBlock discriminated union for subagent_messages.content_blocks - SubagentStopReason + SubagentResult emitted on terminal completion brain-allowlist.ts derives one ToolDef per allow-listed OPERATION. Reuses the ParamDef → JSONSchema shape from the MCP extraction in a local helper (Anthropic's input_schema field diverges from MCP's inputSchema by a character). The 11-name allow-list is read-safe + put_page — every destructive / filesystem / identity-mutating op stays off by default. put_page gets a namespace-wrapped tool schema: `slug` pattern = anchored `^wiki/agents/<subagentId>/.+`. The server-side check in put_page op (shipped in prior commit) is still the authoritative gate — the schema just helps the model write correct slugs first-try. `subagentId` is plumbed into the ToolCtx so the viaSubagent=true fail-closed path lights up on every tool-dispatched put_page. filterAllowedTools narrows a registry by subagent_def's allowed_tools frontmatter field. Rejects unknown names at load time (no silent drop — typos in a skills/subagents/*.md would otherwise ship to prod with a tool silently missing). 18 tests pin: every allowlist name exists in OPERATIONS (catches upstream rename), Anthropic name regex, put_page namespace pattern per-subagent, execute() routes through the op handler with viaSubagent=true, out-of- namespace put_page throws permission_denied, filter passes prefixed + unprefixed names, rejects unknowns, deduplicates. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): subagent-audit JSONL + transcript renderer Two small plumbing pieces the v0.15 subagent handler + `gbrain agent logs` depend on: subagent-audit.ts — JSONL-rotated audit log mirroring the shell-audit pattern. Two event flavors: submission (one line per job submit) and heartbeat (one line per turn boundary — llm_call_started / completed / tool_called / tool_result / tool_failed). Heartbeats fix the "--follow on a long Anthropic call shows nothing for 30 seconds" problem codex flagged. Never logs prompts or tool inputs (PII risk — subagent input_vars may carry user-supplied free text); DOES log tokens, ms_elapsed, tool_name, first 200 chars of error text. Rotates weekly via ISO week. `readSubagent AuditForJob` is the readback path for `gbrain agent logs` — scans the current + prior week file so job boundaries across weeks still resolve. `GBRAIN_AUDIT_DIR` overrides the default ~/.gbrain/audit/ for container deploys. transcript.ts — renders subagent_messages + subagent_tool_executions to markdown. Message order is authoritative; tool rows splice under their owning assistant tool_use by tool_use_id. Handles text, tool_use (with pending / complete / failed execution rows), tool_result (skipped if we already rendered the owning tool_use — avoids double-printing), and unknown block types (fenced JSON dump for diagnostics). Output is UTF-8-safe truncated at maxOutputBytes. 21 unit tests: ISO week filename rotation (incl. 2027-01-01 → W53-2026 boundary), submission + heartbeat write shapes, 200-char error cap, best- effort write failure doesn't throw, readback filters by job_id and sinceIso. Transcript: empty input, ordering, token line, tool_use + complete/failed/pending execution rendering, truncation, unknown-block diagnostic dump. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): subagent LLM-loop handler with crash-resumable replay The main event: runs one Anthropic Messages API conversation with tool use, persists every turn + tool execution, and resumes cleanly after a worker kill anywhere in the loop. Design points that carry the v0.15 guarantees: 1. Two-phase tool persistence. INSERT status='pending' before dispatch, UPDATE to 'complete' or 'failed' after. subagent_messages rows are the canonical conversation; subagent_tool_executions rows are the canonical "did this tool run + what did it return". Either DB commit is atomic, so replay has a single source of truth. 2. Replay reconciliation. If the last persisted message is an assistant with tool_use blocks AND no following synthesized user message, we crashed mid-dispatch. On resume, finish those tools first (respecting idempotent flag for 'pending' rows), synthesize the user turn, and THEN call the LLM again. Non-idempotent pending rows abort the job with a clear error — v0.15 ships only idempotent tools so this is preventive. 3. Rate lease around every LLM call. acquireLease before, releaseLease after (both success and error paths). acquired=false throws RateLeaseUnavailableError — the worker treats it as a renewable error and re-claims later, so a temporary capacity cap doesn't fail the job terminally. 4. Anthropic prompt caching. system block gets cache_control=ephemeral; the LAST tool def gets it too (Anthropic caches everything up to and including the marked block). ~10x cost reduction on multi-turn agents per the plan. 5. Dual-signal abort. AbortSignal.any merges ctx.signal (timeout / lock loss / cancel) with ctx.shutdownSignal (worker SIGTERM). Both feed the Anthropic call's AbortSignal; mid-turn abort bails before the next LLM call with whatever turns are already persisted. Node ≥ 20 has AbortSignal.any; older runtimes get a manual-merge polyfill. 6. Injectable Anthropic client. The real SDK implements MessagesClient structurally; tests inject a FakeMessagesClient that scripts responses. 12 unit tests pin: no-tool happy path, single tool_use complete, tool throws → failed row + loop continues, unknown tool name rejection, max_turns cap, crash-then-resume with partial state, replay skips already- complete tool execs without re-invoking execute, non-idempotent pending rejects on resume, lease acquire + release roundtrip, RateLeaseUnavailable under cap-full, missing prompt validation, allowed_tools unknown-name. NOT in v0.15: refusal detection (stop_reason + content shape), stop_reason =max_tokens partial recovery, mid-call lease renewal with backoff loop. All three are documented as P2 items in the plan file. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): subagent_aggregator handler with mixed-outcome rendering Claims AFTER all subagent children resolve — by then Lane 1B's queue changes have posted one child_done message per terminal transition into this job's inbox (complete / failed / dead / cancelled / timeout). The aggregator reads those, builds a deterministic markdown summary, and returns it as the handler result. Not an LLM call in v0.15 — output is reproducible concatenation so fan-out runs stay comparable. v0.16+ can add an LLM synthesis pass behind an opt-in flag. Contract: - empty children_ids → `(no children)` marker - missing child_done (shouldn't happen under v0.15 invariants but possible if a terminal-state path slipped past Lane 1B) → counted as failed with "no child_done message observed" error - non-complete outcomes: result is null in the output so no payload leaks alongside a failure label - children appear in the order children_ids was supplied - custom aggregate_prompt_template replaces the markdown header 13 unit tests cover: empty input, all-success, mixed outcomes, result suppression on failure, missing child_done handling, order preservation, custom template, progress + log emission, stringified JSONB payload parsing, non-child_done inbox filtering, legacy-writer outcome fallback, and internal helper edges. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): GBRAIN_PLUGIN_PATH loader + plugin-authors guide (v0.15) Plumbing that makes Wintermute (and future downstream agents) day-1 usable on v0.15. Host repos drop a `gbrain.plugin.json` + `subagents/` directory somewhere, set GBRAIN_PLUGIN_PATH (colon-separated like \$PATH), and their custom subagent defs load at worker startup. Path policy is strict: absolute paths only. Relative, ~-prefixed, and URL-style (https://, file://) all rejected with warnings — the user controls where plugins live. Non-existent paths and files (not dirs) are warned and skipped so a typo doesn't crash worker startup. Collision policy: left-wins. If two plugins ship a subagent with the same name, the first one in GBRAIN_PLUGIN_PATH keeps it and the other gets a warning naming both sources. Deterministic + debuggable. Trust policy: plugins ship subagent defs ONLY. Cannot declare new tools, cannot extend the brain allow-list, cannot override safety flags. The subagent def's `allowed_tools:` frontmatter MUST subset the derived registry — validation happens at load time (worker startup), not at dispatch time, so a typo in a skill gives a loud startup error instead of silently "tool never fires at 3am." Manifest `plugin_version: "gbrain-plugin-v1"` locks the contract. Unknown versions rejected. `subagents` field escape attempts (`../../../etc` etc) rejected. gray-matter handles the markdown frontmatter parse — subagent defs don't conform to the page schema, so we don't use parseMarkdown. docs/guides/plugin-authors.md is the Wintermute-facing walkthrough. Covers the minimum viable plugin shape, the three policies, the frontmatter fields, known caveats (audit JSONL is local-only, tool calls always run remote=true, put_page is namespace-scoped). 22 unit tests pin path rejection, missing/invalid manifest, unsupported version, escape-attempt, basename fallback for missing frontmatter.name, allowed_tools round-trip, unknown-tool rejection with validAgentToolNames, empty env, multi-path, collision warning with left-wins, trimmed paths, manifest-rejection as warning. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): gbrain agent run + logs + worker registration (v0.15 Lane 4H) Three integration seams wired: src/commands/agent.ts — \`gbrain agent run\`. Submits subagent jobs (or a fan-out of N + aggregator) under the trusted-submit flag so the PROTECTED_JOB_NAMES guard doesn't reject. Fan-out path creates the aggregator first (so children can reference its id as parent), submits each child with on_child_fail='continue' (required by Lane 1B's terminal- set + child_done machinery), then jsonb_set's the aggregator's children_ids. Short-circuits a 1-entry manifest to a single subagent with no aggregator. Follow mode runs agent-logs streaming + waitFor Completion in parallel and exits on terminal status; detach prints the job id and exits. Ctrl-C is handled as detach, not cancel — the job keeps running, consistent with durability invariants. src/commands/agent-logs.ts — \`gbrain agent logs\`. Merges ~/.gbrain/audit/ subagent-jobs-*.jsonl (heartbeats + submissions) with subagent_messages (persisted conversation) in one chronological stream. --follow polls at 1s and exits when the job hits terminal. --since accepts ISO-8601 OR relative shorthand (5m / 1h / 2d). Writes transcript tail (full message + tool tree) only for terminal jobs, so mid-run --follow doesn't spam a half-rendered transcript. src/commands/jobs.ts registerBuiltinHandlers — matches the shell-handler opt-in shape. GBRAIN_ALLOW_LLM_JOBS=1 registers the subagent + subagent_aggregator handlers, then loads plugins from GBRAIN_PLUGIN_PATH with validAgentToolNames pulled from BRAIN_TOOL_ALLOWLIST. Every plugin warning + loaded-plugin line prints to stderr, mirroring the openclaw- seam startup convention. src/core/minions/protected-names.ts — subagent + subagent_aggregator join the protected set. MCP submit_job returns permission_denied; only trusted-CLI callers (with allowProtectedSubmit) can insert these rows. src/cli.ts — adds 'agent' to CLI_ONLY + dispatches it like 'jobs'. Test fallout: subagent-handler.test.ts + subagent-transcript.test.ts helpers now submit under allowProtectedSubmit (they insert rows named 'subagent' directly against the queue). 23 new tests in agent-cli.test.ts cover: flag parsing (including --detach implies !follow, --tools comma split, -- terminator, unknown flag throw), --since parse (ISO, relative 5m/2h/1d, unparseable error), protected-name guard for all three names, trusted-submit gate, and a fan-out integration check that verifies the aggregator + children shape after --fanout-manifest. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(e2e): rename max_children test's spawned jobs off the protected 'subagent' name The spawn-storm test submitted 50 literal-string 'subagent' children to exercise the max_children row-lock serialization. In v0.15 'subagent' is a PROTECTED_JOB_NAME (CLI-only; trusted submit required), so the old literal submission now throws before reaching the row-lock check. The test is about max_children semantics, not the v0.15 subagent runtime specifically — rename the child name to 'child_worker' so the test exercises the exact same queue.add path without tripping the new guard. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(ship): v0.15.0 — VERSION, CHANGELOG, README, upgrading-agents, CLAUDE.md Bumps VERSION → 0.15.0 and package.json → 0.15.0 (resolves the pre-existing drift — on master, VERSION=0.14.0 but package.json=0.13.1; src/version.ts reads package.json, so this is what the binary prints now). CHANGELOG lands the release-summary entry in the GStack voice + the full itemized change list (11 new modules, 3 new tables, queue correctness fixes, trust-model additions, 159 new unit tests). Voice rules respected — no em dashes, no AI vocabulary, real file names + real numbers. README gets a "Durable agents: `gbrain agent` (v0.15)" section next to the Minions block, with the three canonical CLI shapes (single run, fanout-manifest, logs --follow) and a pointer to plugin-authors.md. docs/UPGRADING_DOWNSTREAM_AGENTS.md gets a full v0.15.0 section covering the four adoption steps downstream agents (Wintermute and similar) need: (1) worker opt-in via GBRAIN_ALLOW_LLM_JOBS, (2) moving custom subagent defs to a plugin repo, (3) replacing ephemeral subagent runs with durable `gbrain agent run`, (4) the put_page namespace rule for agent-driven writes. CLAUDE.md updated with concise per-file descriptions for every new module: the handler, aggregator, audit, rate-leases, wait-for-completion, transcript, plugin-loader, brain-allowlist, tool-defs extraction, agent CLI + logs CLI, and the registerBuiltinHandlers wiring for subagent handlers + plugin-loader. Verified: binary builds (940 modules, 89ms compile), prints `gbrain 0.15.0`, `gbrain agent --help` shows the new subcommand shape. 170 new tests pass (full v0.15 surface). Full unit suite passes bar one parallel-load flake on a pre-existing E2E (graph-quality, passes in isolation). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): drop GBRAIN_ALLOW_LLM_JOBS flag — subagent handlers always-on The env flag was ceremony. Shell jobs need the flag because they execute arbitrary CLI commands (RCE surface). Subagent jobs don't — they call the Anthropic API with whatever ANTHROPIC_API_KEY is in env, so the key is already the cost gate (no key → SDK fails on the first turn). And who-can-submit is already protected by PROTECTED_JOB_NAMES + TrustedSubmitOpts: MCP callers get permission_denied; only `gbrain agent run` with allowProtectedSubmit can insert subagent / subagent_aggregator rows. The flag added nothing the existing guards didn't already give us. registerBuiltinHandlers now always registers subagent + subagent_aggregator and loads GBRAIN_PLUGIN_PATH plugins. Worker startup prints: [minion worker] subagent handlers enabled instead of the conditional enabled/disabled pair. Plugin discovery runs unconditionally — empty PATH is a no-op. README, CHANGELOG, docs/UPGRADING_DOWNSTREAM_AGENTS, CLAUDE.md, agent CLI help text, and subagent handler docstring all updated to drop the flag reference. Shell handler's GBRAIN_ALLOW_SHELL_JOBS gate is untouched — separate concern (RCE, not billing). Full suite: 1859 pass, 0 fail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: scrub private agent-fork name from all public artifacts Enforces the rule added to CLAUDE.md (privacy section): never say `Wintermute` in any CHANGELOG, README, doc, PR, or commit message. Reader-facing copy says `your OpenClaw` (the term covers every downstream OpenClaw deployment — Wintermute, Hermes, AlphaClaw — in one umbrella the reader already recognizes). First-person / origin-story copy says `Garry's OpenClaw` (honest that this is the production deployment driving the feature, without exposing the private agent's name). Swept across: CHANGELOG.md (v0.15 entry + 4 historical mentions) README.md TODOS.md docs/UPGRADING_DOWNSTREAM_AGENTS.md docs/guides/plugin-authors.md (including example plugin names) docs/guides/plugin-handlers.md docs/guides/minions-fix.md docs/designs/KNOWLEDGE_RUNTIME.md (27 refs, mostly analytical) docs/benchmarks/2026-04-18-minions-vs-openclaw-production.md skills/migrations/v0.11.0.md skills/skillpack-check/SKILL.md scripts/skillify-check.ts src/commands/doctor.ts src/commands/migrations/v0_15_0.ts src/commands/skillpack-check.ts src/core/enrichment/completeness.ts src/core/minions/plugin-loader.ts src/core/operations.ts src/core/output/scaffold.ts Intentionally kept (these mentions define/test the rule itself): CLAUDE.md — the privacy rule section necessarily uses the literal name to define the restriction and examples test/plugin-loader.test.ts — fixture name in a plugin-loading test; renaming risks breaking assertion logic test/integrations.test.ts — the word appears in a privacy-regex test that explicitly enforces name redaction test/doctor-minions-check.test.ts — a comment referencing the rule CEO plan artifact at ~/.gstack/projects/… — private, not distributed Binary builds (941 modules), 198/198 relevant tests pass, `gbrain --version` prints `0.15.0`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: gitignore bun --compile artifacts with a glob, not specific hashes Each `bun build --compile` emits a fresh hash-named `.*-*.bun-build` file in cwd. The prior entries listed two specific hashes that were already stale, so every build after those created a new untracked file requiring manual cleanup. Replace the two stale entries with `*.bun-build` so any current or future compile artifact is ignored automatically. Verified: ran `bun build --compile`, got two new `.*-*.bun-build` files, `git status` stays clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(ship): rename v0.15.0 → v0.16.0 gbrain master is at 0.14.2. Other 0.15.x PRs may land before/after this one — we bump the minor (new capability) and lock to 0.16.0 so ordering with concurrent work doesn't matter. Touches: - VERSION: 0.15.0 → 0.16.0 - package.json: 0.15.0 → 0.16.0 - Rename src/commands/migrations/v0_15_0.ts → v0_16_0.ts (+ all version strings inside + import in index.ts registry) - Rename test/migrations-v0_15_0.test.ts → migrations-v0_16_0.test.ts - test/apply-migrations.test.ts: skippedFuture lists now reference '0.16.0' - test/put-page-namespace.test.ts + test/mcp-tool-defs.test.ts: Lane comment refs updated - src/schema.sql + src/core/pglite-schema.ts: "v0.15.0" section comment updated; src/core/schema-embedded.ts regenerated - CHANGELOG.md: top entry renamed to [0.16.0]; inline v0_15_0 / v0.15.0 refs swept - docs/UPGRADING_DOWNSTREAM_AGENTS.md: section heading v0.15.0 → v0.16.0 Verified: `gbrain --version` prints 0.16.0, migration registry / buildPlan / put_page / mcp-tool-defs / handlers tests all green (49/49). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: reframe v0.16 durability headline around OpenClaw crashes "Laptop closed mid-run" framing implied a consumer workflow. Real pain is OpenClaw subagents dying daily on worker kill, memory blip, or timeout. Headline + README copy match the body now. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: regenerate llms-full.txt after README copy change Regen drift guard caught the README edit from 83beec4. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-21 21:14:17 -07:00
Garry Tan	c89aa909c7	feat: Knowledge Runtime — Resolver SDK + BrainWriter + integrity + Budget + scheduler polish (v0.13.0) (#210) ... * docs: Knowledge Runtime design doc (draft) — 4-layer architecture + reduced-scope delta Captures the Knowledge Runtime design thinking from the CEO review session: Resolver SDK, Enrichment Orchestrator, Scheduler, Deterministic Output Builder. The original 7-phase plan was drafted before v0.12.0 (knowledge graph layer) and v0.11.x (Minions agent runtime) shipped. Cross-referenced against what's already merged on master, roughly 60% of the 4-layer vision is already in production under different names: - Minions = scheduler + plugin contract (L1 + L3) - Knowledge graph auto-link = deterministic output at L4 + orchestrator at L2 - BrainBench v1 benchmarks already validate the graph layer The doc is kept as a draft design reference; the actual build-out will scope down to the real delta (typed Resolver interface, BrainWriter API + validators, BudgetLedger, CompletenessScorer, quiet-hours + stagger). See the CEO review notes for the reduced plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(resolvers): Resolver SDK pass 1 — interface + registry (PR 1/5) Adds the typed plugin interface that unifies external-lookup calls (X API, Perplexity, HEAD check, brain-local slug resolution) behind a single shape: registry.resolve('x_handle_to_tweet', { handle, keywords }, ctx) → { value, confidence, source, fetchedAt, raw? } Zero behavior change — the registry is empty by default. Builtins (url_reachable, x_handle_to_tweet) land in the next pass. ScheduledResolver wrapping via Minions lands in PR 5. New files: - src/core/resolvers/interface.ts — Resolver<I,O>, ResolverResult<O>, ResolverContext (engine, storage, config, logger, requestId, remote, deadline, signal), ResolverError (not_found, already_registered, unavailable, timeout, rate_limited, auth, schema, aborted, upstream) - src/core/resolvers/registry.ts — ResolverRegistry (register/get/has/ list/resolve/clear/size) + getDefaultRegistry() for process-wide use - src/core/resolvers/index.ts — barrel export Design rules enforced by types: - Every result carries confidence (0.0-1.0) + source attribution - LLM-backed resolvers return confidence<1.0 by convention - ctx.remote propagates the trust boundary (mirrors OperationContext.remote) - AbortSignal threads through for cooperative cancellation Smoke: imports + runs, list()/get()/resolve() behave as typed. Dependency-free beyond types and storage/engine type imports. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(fail-improve): optional AbortSignal — Resolver SDK pass 2 (PR 1/5) Extends FailImproveLoop.execute with an optional `opts.signal` that threads through the deterministic-first / LLM-fallback flow. Needed by the Resolver SDK so long-running lookups can be cooperatively cancelled when a caller aborts (deadline hit, Minion job timeout, user ctrl-c). Additive and backwards-compatible: - execute() signature widens callbacks to (input, signal?) => ...; existing two-arg callbacks are structurally compatible and ignore the extra arg. - opts is optional; callers that omit it get pre-extension behavior. - Aborts throw a DOM-style AbortError (name='AbortError'), matching what fetch() throws, so downstream `err.name === 'AbortError'` branches work unchanged. - Aborted runs are NOT logged to the failure JSONL — not informative and would pollute pattern analysis. Abort check fires in three places: - Before the deterministic call (pre-flight) - Between deterministic miss and LLM call (mid-flight) - Inside llmFallbackFn if the implementation respects signal itself Smoke tests: 5 scenarios (existing sig, llm fallback, pre-abort, mid-flight abort, signal threaded to fallback) — all pass. Existing test/fail-improve.test.ts (13 tests, 27 expects) unchanged and passing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(resolvers): url_reachable + x_handle_to_tweet — SDK pass 3 (PR 1/5) Two reference resolver implementations that validate the interface against real-world requirements: a deterministic free-cost check and a rate-limited paid-backend lookup. src/core/resolvers/builtin/url-reachable.ts HEAD-check a URL, follow redirects (max 5), detect dead links. Reused isInternalUrl() from the wave-3 SSRF hardening; re-validates every redirect hop against the same filter. Falls back from HEAD to GET on 405/501. Composes caller's AbortSignal with a per-request timeout via AbortSignal.any (with manual-propagation fallback). Confidence=1 when the backend answers; confidence=0 only on transport failure (DNS/connect/timeout). src/core/resolvers/builtin/x-api/handle-to-tweet.ts Find a tweet by handle + free-text keyword hint. Used by the upcoming `gbrain integrity --auto` loop to repair the 1,424 bare-tweet citations in Garry's brain. Confidence buckets align with the three-bucket contract: - >=0.8 auto-repair (single strong match, or dominant in small candidate set) - 0.5-0.8 review queue (ambiguous but promising) - <0.5 skip (many candidates or weak match) Scoring: normalized keyword-token overlap against tweet text, with margin boost for dominant matches. Strict handle regex (X's username rules). Retries on 429 up to 2x with Retry-After honor. Terminal 401/403 surfaces as auth ResolverError so the caller stops hammering. Bearer token read from ctx.config.x_api_bearer_token or X_API_BEARER_TOKEN env — never logged. Smoke: registry accepts both, SSRF blocks localhost + file://, available() returns false when token missing, schema validator rejects bad handles. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(resolvers): tests + gbrain resolvers CLI — SDK pass 4 (PR 1/5 complete) Closes out PR 1. 43 new tests in test/resolvers.test.ts covering registry contract, both reference builtins, all three confidence buckets, and every ResolverError subcode. test/resolvers.test.ts - ResolverRegistry: register, duplicate-id rejection, get/has, list with cost+backend filters, resolve, unavailable propagation, clear, default singleton lifecycle. - url_reachable: available(), SSRF guard on localhost + RFC1918 + 169.254 metadata + file:// scheme, empty-url schema error, 200/404 status propagation, HEAD→GET fallback on 405, redirect chain, per-hop SSRF re-validation, network failure → reachable=false, AbortSignal mid-flight. - x_handle_to_tweet: token gate via env AND via ctx.config, invalid/long handle schema errors, zero-candidate + single-strong + single-weak + many-ambiguous confidence buckets (gates >=0.5 url emission), 401/403 auth error, 500 upstream error, 429 retry-then-rate_limited, X operator stripping (prompt injection defense). src/commands/resolvers.ts - `gbrain resolvers list [--cost | --backend | --json]` pretty table or JSON. - `gbrain resolvers describe <id>` schema + availability detail. - registerBuiltinResolvers() is idempotent; ready to be called from future entry points (gbrain integrity, MCP server). src/cli.ts wires `resolvers` into CLI_ONLY + dispatches to runResolvers. Full suite: 1343 pass / 0 fail / 141 skip (E2E without DATABASE_URL). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(output): BrainWriter + Scaffolder + SlugRegistry — PR 2 pass 1/4 Lands the transactional writer library that the rest of the Knowledge Runtime sits on top of. No callers routed through it yet — publish.ts / backlinks.ts / put_page migrations are pass 4 and PR 2.5. src/core/output/scaffold.ts Deterministic URL / citation / link builders. Callers pass typed inputs (handle + tweetId, account + messageId, slug + display text) and get canonical markdown bytes out. LLM-generated URLs never touch disk. - tweetCitation({handle, tweetId, dateISO?}) - emailCitation({account, messageId, subject, dateISO?}) - sourceCitation(resolverResult, {url?, label?}) - entityLink({slug, displayText, relativePrefix?}) - timelineLine({dateISO, summary, citation?}) ScaffoldError with codes for invalid_handle / invalid_tweet_id / invalid_slug / invalid_message_id / invalid_date / empty. src/core/output/slug-registry.ts Solves the "Marc Benioff vs Marc-Benioff both slug to marc-benioff" bug. create() probes engine.getPage and either returns the desired slug or disambiguates (alice-smith → alice-smith-2). isFree() + suggestDisambiguators() for interactive UX. Errors: collision, disambiguator_exhausted, invalid_slug. src/core/output/writer.ts BrainWriter.transaction(fn, ctx) wraps engine.transaction. The `fn` callback receives a WriteTx with createEntity / appendTimeline / setCompiledTruth / setFrontmatterField / putRawData / addLink (the last creates both forward + reverse back-link atomically). On commit, per-page validators run against all touchedSlugs. Strict mode throws on error-severity findings, rolling back the outer tx. Lint mode (default for PR 2 rollout) returns the report but commits regardless. Pages with `validate: false` frontmatter skip validators entirely (grandfather hook for PR 2 migration). Integration smoke against PGLite: createEntity → disambiguator (2nd call with same desired slug), addLink writes both forward + back-link, strict-mode validator failure rolls back the transaction bit-identically. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(output): 4 pre-commit validators + tests — PR 2 pass 2/4 Lands the validator suite that BrainWriter runs before committing a transaction. Paragraph-level deterministic checks, markdown-aware, skip legacy pages via validate:false frontmatter. src/core/output/validators/citation.ts Every factual paragraph in compiled_truth carries at least one citation marker: [Source: ...] or a linked URL. Splits paragraphs on blank lines, strips fenced code / inline code / HTML comments before checking. Ignores headings, key-value lines ("**Status:** Active"), table rows, pure wikilink bullets (## See Also), and short labels without a factual verb. Deterministic — no LLM, no semantic judgment. src/core/output/validators/link.ts Every [text](path) wikilink resolves to a page that exists (unless it's an external http(s) URL, which this validator doesn't check; that's url_reachable's job in PR 3). Strips relative prefix and .md extension. Batches engine.getPage lookups per unique target. mailto/anchor/other schemes flagged as warning. Links inside fenced code blocks are skipped. src/core/output/validators/back-link.ts Iron Law: if page X → page Y, then Y → X. Reads engine.getLinks(ctx.slug), and for each target checks engine.getLinks(target) for a reverse edge. Missing reverses flagged as warning (runAutoLink is the authoritative enforcer on put_page; this is defense-in-depth for pages edited outside the main write path). src/core/output/validators/triple-hr.ts Catches hygiene issues on the compiled_truth / timeline split: bare `---` in compiled_truth would re-split on round-trip through parseMarkdown; headings in the timeline section signal authoring mistakes. Both warn (not error) — legacy pages legitimately use thematic breaks. src/core/output/validators/index.ts registerBuiltinValidators(writer) wires all four. test/writer.test.ts 57 tests: Scaffolder (all 5 helpers + error paths), SlugRegistry (create, disambiguator, collision throw, invalid-slug, isFree, suggestDisambiguators), BrainWriter (happy path, disambiguate, addLink + reverse, strict rollback, lint proceeds with report, off skips validators, validate:false grandfather, setCompiledTruth, setFrontmatterField merge, registered validators list), citation validator (all 11 shape cases), link validator (normalizeToSlug including ../../, external URL skip, mailto warning, code-fence skip), back-link validator (no outbound, missing reverse → warning, bidirectional clean), triple-hr validator (clean, bare --- warning, fenced --- skipped, heading in timeline warning, ## Timeline header allowed). Full suite: 1400 pass / 0 fail / 141 skip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(migrations): v0.13.0 grandfather validate:false — PR 2 pass 3/4 Adds the TS migration that makes BrainWriter's strict-mode rollout safe: every existing page gets `validate: false` in frontmatter so the new citation / link / back-link / triple-HR validators skip legacy content. gbrain integrity --auto (PR 3) clears the flag per-page once real citations are repaired. src/commands/migrations/v0_13_0_add_validate_false.ts Four-phase orchestrator following the v0_12_0 pattern: A. connect — loadConfig + createEngine. Does NOT write config (prior learning: gbrain init --migrate-only semantics; never flip Postgres users to PGLite via bare init). B. snapshot — engine.getAllSlugs() upfront (prior learning: listpages-pagination-mutation; OFFSET iteration is self-invalidating when each write bumps updated_at). C. grandfather — per slug, skip if frontmatter.validate already set, else append-log pre-mutation snapshot to ~/.gbrain/migrations/v0_13_0-rollback.jsonl and putPage with validate:false merged in. Batched 100 at a time so interruption losses are bounded. D. verify — SQL count of pages with validate=false ≥ expectedTouched. Idempotent: second run is a no-op. Reversible: rollback log is append-only JSONL; future `gbrain apply-migrations --rollback v0.13.0` replays it. Safe on empty brains (returns complete with 0 touched). src/commands/migrations/index.ts Registers v0_13_0 after v0_12_0 in semver order. test/migrations-v0_13_0.test.ts Registry integration (v0.13.0 present, semver-after-v0.12.0, pitch metadata well-formed), orchestrator handles no-config gracefully, dryRun skips the connect phase. test/apply-migrations.test.ts Updated two assertions that hard-coded the v0.12.0 skippedFuture list to also include v0.13.0 (now skippedFuture when installed < 0.13.0). Full suite: 1405 pass / 0 fail / 141 skip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(integrity): gbrain integrity — bare-tweet repair + dead-link scan (PR 3) Ships the user-visible milestone for the Knowledge Runtime delta: a command that finds brain-integrity issues and repairs them through the BrainWriter + Resolver SDK infrastructure from PRs 1 and 2. Targets the two quantified pain points from brain/CITATIONS.md: - 1,424 of 3,115 people pages have bare tweet references without URLs - An unknown fraction of existing URL citations have rotted Subcommands: gbrain integrity check Read-only report, optional --json gbrain integrity auto Three-bucket repair loop gbrain integrity review Print review-queue path + count gbrain integrity reset-progress Clear the progress file Three-bucket contract (matches x_handle_to_tweet resolver's confidence scoring): >=0.8 → auto-repair via BrainWriter transaction. Appends a timeline entry on the page with a Scaffolder-built tweet citation (URL from the API response, never from LLM text). 0.5-0.8 → append to ~/.gbrain/integrity-review.md with all candidates sorted by match score, for batch human review. <0.5 → log reason to ~/.gbrain/integrity.log.jsonl and skip. Resumable: every processed slug hits ~/.gbrain/integrity-progress.jsonl so an interrupted run resumes from the last slug. --fresh clears it. Bare-tweet detection patterns (regex, deterministic, skip code fences and already-cited lines): - "tweeted about" - "in/on a (recent|viral) tweet" - "wrote a tweet/post" - "posted on X" - "via X" (but not "via X/handle" — already cited) - possessive "his/her/their tweet" External-link detection extracts all [text](https?://...) pairs (code fences skipped) for optional dead-link probing via url_reachable. Dead links are surfaced, not auto-repaired — no "correct" replacement exists without human judgment. Wiring: runIntegrity dispatches subcommands, registers builtin resolvers into the default registry, connects to the brain engine, and uses BrainWriter in strict-off mode (integrity is the repair path, not the write-gate path). Unit tests: 21 cover bare-tweet regex (all 9 phrase shapes + code-fence skip + URL-already-present skip + per-line dedup), external-link extraction (http+https, line numbers, fenced skip), frontmatter handle extraction (x_handle, twitter, twitter_handle, x; preference order; leading @ strip; null paths). End-to-end auto flow verified manually via the resolver SDK tests + BrainWriter tests it composes. src/cli.ts wires `integrity` into CLI_ONLY + dispatches to runIntegrity. Full suite: 1426 pass / 0 fail / 141 skip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(enrichment): BudgetLedger + CompletenessScorer — PR 4 Two layer-2 primitives that slot under the resolver SDK and BrainWriter: cost-aware spend caps and evidence-weighted per-page completeness scoring. Schema migration v11 adds two tables: budget_ledger (scope, resolver_id, local_date) PK — midnight rollover by date column means a new calendar day upserts a new row; no rollover thread, no race. budget_reservations (reservation_id) — TTL-bounded held reservations (default 60s) so process death between reserve() and commit() doesn't strand money. Rollback plan: DROP TABLE. Budget data is regenerable from resolver call logs; no durable product value lives in the ledger. src/core/enrichment/budget.ts BudgetLedger.reserve({resolverId, estimateUsd, capUsd?, ttlSeconds?}) serializes concurrent reserves on {scope, resolver_id, local_date} via SELECT ... FOR UPDATE. Returns {kind:'held', reservationId, ...} or {kind:'exhausted', reason, spent, pending, cap} — never over-spends. commit(id, actualUsd) moves money from reserved_usd to committed_usd and marks the reservation status='committed'. rollback(id) zeros out the reservation without touching committed. Commit-after-commit throws already_finalized; rollback-after-commit is a no-op (callers don't need to guard). commit-unknown-id throws reservation_not_found. cleanupExpired() sweeps held reservations past expires_at and rolls them back; reserve() opportunistically reclaims the target row's expired reservations before acquiring its own lock. IANA timezone config via opts.tz (default America/Los_Angeles); midnight rollover is naturally expressed as a date column + Intl.DateTimeFormat with en-CA locale (YYYY-MM-DD). DST is handled by the formatter. src/core/enrichment/completeness.ts Seven per-type rubrics (person, company, project, deal, concept, source, media) + default. Each rubric's dimension weights sum to 1.0, checked at module load. scorePage(page) returns {score, dimensionScores, rubric} where score is 0.000–1.000. Person rubric dimensions: has_role_and_company, has_source_urls, has_timeline_entries, has_citations, has_backlinks, recency_score, non_redundancy. The last two are the explicit fix for the two pathologies called out in the codex review of the earlier design: stale pages that never decay (30-day re-enrich forever) and Wilco-style repeated blocks that pass Wintermute's length heuristic. Pure functions. No engine calls — BrainWriter invokes scorePage after a transaction and caches the result in frontmatter.completeness. test/enrichment.test.ts — 23 tests: BudgetLedger: under-cap held, over-cap exhausted, commit moves money, rollback clears, commit-rollback no-op, commit-commit throws, commit- unknown throws, invalid input, empty state null, scope isolation, parallel reserves respect cap (10 parallel, cap 1.0, est 0.3 each → ≤ 3 held; state.reservedUsd ≤ 1.0), cleanupExpired reclaims TTL=0. CompletenessScorer: all 8 rubrics sum to 1.0, empty person scores <0.3, fully-enriched person >0.8, dimension scores exposed, role detection, company/concept/source/media/default routing, recency decay with age, non_redundancy penalizes repeated lines. Full suite: 1449 pass / 0 fail / 141 skip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): quiet-hours + stagger + claim-time gate — PR 5 Closes the scheduler gap per CEO plan: Minions v7 shipped a durable runtime but nothing about when jobs should NOT run. This wires quiet-hours enforcement at claim time (the codex correction — dispatch- time is wrong because a queued job can become claimable after its window opens) plus deterministic stagger slots to prevent cron-boundary storms. Schema migration v12 adds two columns to minion_jobs: quiet_hours JSONB — {start, end, tz, policy} window config stagger_key TEXT — partitioning key for deterministic offset Plus a partial index on stagger_key for later slot-assignment queries. src/core/minions/quiet-hours.ts evaluateQuietHours(cfg, now?) → 'allow' | 'skip' | 'defer'. Pure, deterministic, no engine. Handles straight-line and wrap-around windows (e.g. 22→7 spans midnight). IANA timezone via Intl.DateTimeFormat; unknown tz fails open (allow) — safer than hard-blocking every job. 'skip' policy drops the event; 'defer' (default) re-queues for later. src/core/minions/stagger.ts staggerMinuteOffset(key) → 0–59, FNV-1a hash. Same key → same slot. Pure; no module-level state. Used by scheduled resolvers that want to avoid cron-boundary collisions ("10 jobs all fire at minute 0"). src/core/minions/worker.ts MinionWorker.tick now consults evaluateQuietHours on every claimed job. Verdict 'defer' → UPDATE status='delayed', delay_until = now() + 15m (prevents immediate re-claim loops when the claim query re-runs). Verdict 'skip' → UPDATE status='cancelled', error_text='skipped_quiet_hours'. Both paths clear lock_token and require lock_token match in the WHERE clause so a concurrent stall recovery can't race us. test/minions-quiet-hours.test.ts — 25 tests: evaluateQuietHours: null/undefined/invalid config paths (allow fail-open), straight-line in/out + exclusive-end, wrap-around in (before midnight + after), skip vs defer policy, timezone-offset propagation (winter PST vs summer PDT), localHour parity with Date.getUTCHours. staggerMinuteOffset: deterministic same key → same offset, different keys spread across buckets (10 keys → ≥5 unique buckets), empty/non- string edge cases. Schema v12: quiet_hours and stagger_key columns exist on minion_jobs, idx_minion_jobs_stagger_key index present. Full suite: 1474 pass / 0 fail / 141 skip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(output): post-write validator lint hook — PR 2.5 Minimal integration of BrainWriter validators into the main write path, feature-flag-gated and non-blocking. The CEO plan explicitly scoped PR 2.5 as a pre-soak landing step: the hook plugs in now, observability lands, but strict-mode rejection is deferred to a follow-on release gated on the 7-day soak + BrainBench regression ≤1pt. src/core/output/post-write.ts runPostWriteLint(engine, slug, opts?) invokes the four BrainWriter validators (citation, link, back-link, triple-hr) against a freshly written page and returns a PostWriteLintResult. Skips cleanly when: - config `writer.lint_on_put_page` is not truthy (default OFF; opts.force overrides) - the page is not found (shouldn't happen in normal put_page flow) - the page has frontmatter.validate === false (grandfathered) Findings are logged to: - ~/.gbrain/validator-lint.jsonl (capped at 20 findings per line) - engine.logIngest (ingest_log table) for durable agent-inspectable history Validator-level exceptions are swallowed so a buggy validator never breaks put_page. src/core/operations.ts put_page handler After importFromContent + runAutoLink, imports runPostWriteLint and invokes it. Result returns writer_lint: {error_count, warning_count} or {skipped: reason}. Try/catch wraps the whole hook so an import or runtime error never blocks the main write. Enable locally: gbrain config set writer.lint_on_put_page true Then every put_page emits a writer_lint summary + appends structured findings to the ingest log for analysis before the strict-mode flip. test/post-write-lint.test.ts — 11 tests: Flag reader (default off, true/1/on, other values false, explicit false) Hook behavior (flag-off skip, page-not-found skip, validate:false grandfather skip, force=true overrides flag, dirty page yields citation error, clean page yields zero findings). Full suite: 1485 pass / 0 fail / 141 skip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(migrations-v0_13_0): drop flaky no-config assertion The 'does not succeed when no brain is configured' test assumed loadConfig would return null when HOME is empty, but it also reads DATABASE_URL from the environment. When .env.testing sources DATABASE_URL into the shell (normal E2E lifecycle), the orchestrator connects successfully and runs to completion — the test's assertion was unreachable. The dry-run path is still covered by the remaining test in the same describe block; registry integration and semver ordering are covered by the sibling describe. Full suite with DATABASE_URL live: 1574 pass / 0 fail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(minions): wire quiet_hours + stagger_key into MinionJobInput + queue.add Codex adversarial review caught that PR 5 (claim-time quiet-hours gate) was cosmetic: the schema v12 column existed, the worker read it via `readQuietHoursConfig(job)`, but `MinionJobInput` never accepted it, `queue.add()` never inserted it, and `rowToMinionJob()` never mapped it out. Result: every scheduled job saw `quiet_hours: null`, so the gate was a no-op. Stagger_key had the same broken wiring. - MinionJob (types.ts): add `quiet_hours` and `stagger_key` fields. - MinionJobInput: add matching optional fields so callers can submit them. - rowToMinionJob: parse both columns (JSONB handled the same way as `data`). - MinionQueue.add: include both columns in the INSERT (idempotent + normal paths), bound as $19/$20. The `$19::jsonb` cast matches the JSONB column shape; the wire format is the same native-JS object path that fixed the JSONB double-encode bug in v0.12.1. After this, `await queue.add('x', {}, { quiet_hours: {start:22,end:7, tz:"America/Los_Angeles",policy:"defer"} })` actually stores the window and the worker's claim-time gate defers the job inside it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(minions): route quiet-hours 'skip' through cancelJob to rollup parents Codex flagged that handleQuietHoursDefer with verdict='skip' directly set status='cancelled' via raw UPDATE — bypassing MinionQueue.cancelJob, which means: - Parent jobs in 'waiting-children' never get rolled up. - Descendant jobs don't cascade-cancel. - Child-done inbox notification is skipped. Result: a parent waiting on a child that fell inside quiet hours with policy='skip' stays stuck forever. Fix: release the lock, then delegate to queue.cancelJob(job.id) which handles the recursive CTE + parent rollup + inbox posting correctly. Falls back to a direct UPDATE only if cancelJob errors — even then, the status transition is status-guarded to avoid stomping terminal states. Defer path unchanged (no parent rollup needed since the job hasn't reached a terminal state). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(budget): commit() re-checks cap + rejects negative actuals Codex caught two cap-bypass bugs in BudgetLedger.commit(): 1. reserve({estimateUsd: 0.01, capUsd: 1.0}) + commit(id, 100) silently charged $100 to a $1-cap bucket. Cap is an advertised invariant that the code was not enforcing. 2. Negative actuals (commit(id, -5)) were accepted, letting callers artificially reduce committed_usd below the real spend. Refunds need a dedicated API, not a side-channel on commit. Fix: - Reject non-finite AND negative actualUsd at entrypoint. - Lock the ledger row FOR UPDATE during commit (same serialization as reserve). - Compute effective cap headroom = cap - other_committed - other_reserved (excluding this reservation from the reserved pool since we're about to finalize it). - When actualUsd would exceed available, clamp committed_usd to max available and throw BudgetError with the overage reported. The reservation is still marked 'committed' (API call already happened; don't retry-loop), but the cap is honored. After this, a $1/day cap actually means $1/day. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(integrity): --dry-run no longer writes progress, poisoning resume Codex caught that 'gbrain integrity auto --dry-run' appended progress entries (status='repaired', 'reviewed', 'skipped', 'error') despite doing no actual writes. The follow-on real run with default --resume would then skip those slugs — the dry-run silently consumed the work queue. Fix: gate every appendProgress() call in cmdAuto on !dryRun. Dry-run still logs to the skip log / review queue (so the user sees what WOULD happen), but the progress file stays untouched. Behavior: --dry-run → buckets counted + summary printed + review-queue + log populated, but progress file unchanged. (default) → progress file tracks every processed slug, so Ctrl-C + re-run resumes from the right place. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.13.0.0) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(resolvers): DNS-rebinding defense + X rate-limit header parity Two non-blocking codex findings on PR #210 rolled into one bisectable commit because their tests share an import line. url_reachable: hostname-string SSRF guard is vulnerable to DNS rebinding (attacker-controlled DNS returns a public IP at validate time and 169.254.169.254 at fetch time). Add checkDnsRebinding() that resolves the hostname via dns.lookup({all:true}) and rejects any result whose A/AAAA record lands in a private range (v4 via isPrivateIpv4, v6 loopback/link-local/unique-local/IPv4-mapped). Applied on the initial URL and on every redirect target. Null on DNS failure so genuine network problems surface via fetch. x_handle_to_tweet: rate-limit backoff only honored Retry-After and ignored X's proprietary x-rate-limit-reset header. computeBackoffMs() parses both (Retry-After = seconds or HTTP-date; x-rate-limit-reset = epoch seconds), takes MAX, and clamps to [2s, 60s]. Exported for testability; callers use it uniformly on every 429. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(writer): advisory lock on desiredSlug prevents cross-process TOCTOU BrainWriter's createEntity checks engine.getPage(slug) and falls back to putPage(), which upserts. Two putPage('people/alice') calls from separate processes (a Claude Code session + a Minions worker, say) can both read "free" from SlugRegistry and both call putPage, silently overwriting each other with no disambiguation. Take a transaction-scoped advisory lock keyed on hashtext(desiredSlug) before the registry check. Concurrent writers for the same slug now serialize at the DB level: the second observes the first's commit and disambiguates to alice-2. PGLite is single-process so this is a harmless no-op there. Wrapped in try/catch so engines/test doubles that don't support advisory locks fall through to the existing within-process check. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(validators): empty [Source:] no longer satisfies citation check Regex /\[Source:[^\]]*\]/ matched decorative markers like [Source:] and [Source: ] that carry zero provenance. Tighten to require at least one non-whitespace character before the closing bracket. The inline URL form ](https://...) already requires a scheme+host so it stays as-is. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(auto-link): advisory lock serializes concurrent reconciliation runAutoLink wraps getLinks + addLink/removeLink in a transaction, but row-level locks alone don't prevent the union-of-writes race: two concurrent put_page calls on the same slug can both read the same existingKeys BEFORE either mutates a row, then proceed to add links the other side's rewrite no longer mentions. Take a transaction-scoped advisory lock on hashtext("auto_link:" || slug) at the start of the reconciliation. Concurrent writers on the same slug now fully serialize; writers on different slugs still run in parallel. No-op on engines without advisory locks (PGLite). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: expand coverage on abort-signal threading + integrity CLI dispatch fail-improve: four new AbortSignal cases — pre-start abort, between deterministic and LLM, signal forwarded into both callbacks, and LLM-thrown AbortError propagates without logging a failure entry. integrity: three new CLI dispatch cases — --help, no-subcommand (help), and unknown subcommand (stderr + exit 1). Non-engine paths so they exercise routing without spinning up a DB. Coverage-only; no source changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(doctor): fold integrity sample scan into default health check Expose scanIntegrity(engine, opts) as a pure library function — same logic cmdCheck uses — and call it from doctor in non-fast mode with a 500-page sampling limit. Surfaces bare-tweet phrase count and external-link count as an 'integrity' check, warn-status when bare tweets are present with a one-liner pointing at 'gbrain integrity check' for the full report and 'integrity auto' for repair. Read-only: no network, no writes, no resolver calls. Pages with validate:false frontmatter are skipped (grandfathered). --fast mode skips it entirely so the existing health-snapshot contract holds. Users no longer need to remember three separate commands (doctor, lint, integrity check) to audit brain health — doctor surfaces the integrity signal by default, full scan stays available for deep dives. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(put_page): auto-extract timeline entries alongside auto-link put_page already chunks, embeds, reconciles tags, and extracts auto-links on every write. Timeline extraction has lived in a separate command (gbrain extract timeline) that users had to remember to run. Fold it into the write path: after the page commits, parse timeline entries from compiled_truth + timeline body and insert via addTimelineEntriesBatch. ON CONFLICT DO NOTHING keeps it idempotent across re-writes. Mirrors auto-link shape: best-effort post-hook, skipped for remote (MCP) callers, gated by auto_timeline config (default TRUE). Response includes auto_timeline: { created } alongside auto_links. Side effect: a one-shot `gbrain put` now produces a complete page — chunks, embeddings, links, AND timeline — instead of three commands the user has to chain manually. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(migrate): verify target health after engine migration After a PGLite↔Postgres migration, the user was left to run 'gbrain doctor' themselves to confirm the target is good. Not great, because the failure modes (partial copy, missing embeddings, schema drift) all surface at next CLI use when the migration itself looks like it succeeded. Add verifyTarget() — inline doctor-lite that checks page count matches the source, embedding coverage is above 90%, and schema version is at latest. Prints a 3-line status table at the end of migrate and points at 'gbrain doctor' for the full check. Non-fatal: warns on discrepancies instead of failing the command so the user sees the full picture. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(bench): add v0.13 knowledge runtime benchmark deltas Two new benchmark scripts + one consolidated markdown comparing this branch against master (c0b6219, v0.12.1): benchmark-put-page-latency.ts — 200 put_page ops, measures the per-write cost of Step B's auto-timeline extraction. Branch adds ~0.5ms mean latency and produces 300 timeline entries for free; master produces zero and requires a separate 'gbrain extract timeline' pass. benchmark-knowledge-runtime.ts — three measurements in one script: time-to-queryable (branch 40/40 vs master 0/40 on post-ingest timeline queries), integrity repair rate (70/20/10 three-bucket split via mocked resolver), doctor completeness (surfaces 100% of real issues after Step A, respects grandfathered pages). docs/benchmarks/2026-04-19-knowledge-runtime-v0.13.md — consolidated report. Covers the four moved benchmarks plus side-by-side runs of graph-quality and search-quality showing they're identical across master and branch. Proof of no regression on the retrieval hot path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-20 07:30:00 +08:00
Garry Tan	d8613366a5	Minions v7 + v0.11.1 canonical migration + skillify (#130) ... * feat: add minion_jobs schema, migration v5, and executeRaw to BrainEngine Foundation for the Minions job queue system. Adds: - minion_jobs table (20 columns) with CHECK constraints, partial indexes, and RLS. Inspired by BullMQ's job model, adapted for Postgres. - Migration v5 creates the table for existing databases. - executeRaw<T>() method on BrainEngine interface for raw SQL access, needed by the Minions module for claim queries (FOR UPDATE SKIP LOCKED), token-fenced writes, and atomic stall detection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Minions job queue — queue, worker, backoff, types BullMQ-inspired Postgres-native job queue built into GBrain. No Redis. No external dependencies. Postgres transactions replace Lua scripts. - MinionQueue: submit, claim (FOR UPDATE SKIP LOCKED), complete/fail (token-fenced), atomic stall detection (CTE), delayed promotion, parent-child resolution, prune, stats - MinionWorker: handler registry, lock renewal, graceful SIGTERM, exponential backoff with jitter, UnrecoverableError bypass - MinionJobContext: updateProgress(), log(), isActive() for handlers - 8-state machine: waiting/active/completed/failed/delayed/dead/ cancelled/waiting-children Patterns stolen from: BullMQ (lock tokens, stall detection, flows), Sidekiq (dead set, backoff formula), Inngest (checkpoint/resume). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: 43 tests for Minions job queue Full coverage of the Minions module against PGLite in-memory: - Queue CRUD (9): submit, get, list, remove, cancel, retry, duplicate - State machine (6): waiting→active→completed/failed, retry→delayed→waiting - Backoff (4): exponential, fixed, jitter range, attempts_made=0 edge - Stall detection (3): detect stalled, counter increment, max→dead - Dependencies (5): parent waits, fail_parent, continue, remove_dep, orphan - Worker lifecycle (5): register, start-without-handlers, claim+execute, non-Error throws, UnrecoverableError bypass - Lock management (3): renewal, token mismatch, claim sets lock fields - Claim mechanics (4): empty queue, priority ordering, name filtering, delayed promotion timing - Cancel & retry (2): cancel active, retry dead Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Minions CLI commands and MCP operations Wire Minions into the GBrain CLI and MCP layer: CLI (gbrain jobs): submit <name> [--params JSON] [--follow] [--dry-run] list [--status S] [--queue Q] [--limit N] get <id> — detailed view with attempt history cancel/retry/delete <id> prune [--older-than 30d] stats — job health dashboard work [--queue Q] [--concurrency N] — Postgres-only worker daemon 6 MCP operations (contract-first, auto-exposed via MCP server): submit_job, get_job, list_jobs, cancel_job, retry_job, get_job_progress Built-in handlers: sync, embed, lint, import. --follow runs inline. Worker daemon blocked on PGLite (exclusive file lock). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update project documentation for Minions job queue CLAUDE.md: added Minions files to key files, updated operation count (36), BrainEngine method count (38), test file count (45), added jobs CLI commands. CHANGELOG.md: added Minions entry to v0.10.0 (background jobs, retry, stall detection, worker daemon). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Minions v2 — agent orchestration primitives (pause/resume, inbox, tokens, replay) Adds the foundation for Minions as universal agent orchestration infrastructure. GBrain's Postgres-native job queue now supports durable, observable, steerable background agents. The OpenClaw plugin (separate repo) will consume these via library import, not MCP, for zero-latency local integration. ## New capabilities - **Concurrent worker** — Promise pool replaces sequential loop. Per-job AbortController for cooperative cancellation. Graceful shutdown waits for all in-flight jobs via Promise.allSettled. - **Pause/resume** — pauseJob clears the lock and fires AbortSignal on active jobs. Handlers check ctx.signal.aborted and exit cleanly. resumeJob returns paused jobs to waiting. Catch block skips failJob when signal.aborted. - **Inbox (separate table)** — minion_inbox table for sidechannel messages. sendMessage with sender validation (parent job or admin). readInbox is token-fenced and marks read_at atomically. Separate table avoids row bloat from rewriting JSONB on every send. - **Token accounting** — tokens_input/tokens_output/tokens_cache_read columns. updateTokens accumulates; completeJob rolls child tokens up to parent. USD cost computed at read time (no cost_usd column — pricing too volatile). - **Job replay** — replayJob clones a terminal job with optional data overrides. New job, fresh attempts, no parent link. ## Handler contract additions MinionJobContext now provides: - `signal: AbortSignal` — cooperative cancellation - `updateTokens(tokens)` — accumulate token usage - `readInbox()` — check for sidechannel messages - `log()` — now accepts string or TranscriptEntry ## MCP operations added pause_job, resume_job, replay_job, send_job_message — all auto-generate CLI commands and MCP server endpoints. ## Library exports package.json exports map adds ./minions and ./engine-factory paths so plugins can `import { MinionQueue } from 'gbrain/minions'` for direct library use. ## Instruction layer (the teaching) - skills/minion-orchestrator/SKILL.md — when/how to use Minions, decision matrix, lifecycle management, anti-patterns - skills/conventions/subagent-routing.md — cross-cutting rule: all background work goes through Minions - RESOLVER.md — trigger entries for agent orchestration - manifest.json — registered ## Schema migration v6 Additive: 3 token columns, paused status, minion_inbox table with unread index. Full Postgres + PGLite support. No backfill needed. ## Tests 65 tests (was 43): pause/resume (5), inbox (6), tokens (4), replay (4), concurrent worker context (3), plus all existing coverage. ## What's NOT in this commit Deferred to follow-up PRs: - LISTEN/NOTIFY subscribe (needs real Postgres E2E) - Resource governor (depends on concurrent worker stress testing) - Routing eval harness (needs API keys + benchmark data) - OpenClaw plugin (separate @gbrain/openclaw-minions-plugin repo) See docs/designs/MINIONS_AGENT_ORCHESTRATION.md for full CEO-approved design. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): migration v7 — agent_parity_layer schema Adds columns on minion_jobs (depth, max_children, timeout_ms, timeout_at, remove_on_complete, remove_on_fail, idempotency_key) plus the new minion_attachments table. Three partial indexes for bounded scans: idx_minion_jobs_timeout, idx_minion_jobs_parent_status, and uniq_minion_jobs_idempotency. Check constraints enforce non-negative depth and positive child cap / timeout. Additive migration — existing installs pick it up via ensureSchema on next use. No user action required. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(minions): extend types for v7 parity layer Extends MinionJob with depth/max_children/timeout_ms/timeout_at/ remove_on_complete/remove_on_fail/idempotency_key. Extends MinionJobInput with the same options plus max_spawn_depth override. Adds MinionQueueOpts (maxSpawnDepth default 5, maxAttachmentBytes default 5 MiB). Adds AttachmentInput/Attachment shapes and ChildDoneMessage in the InboxMessage union. rowToMinionJob updated to pick up the new columns. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(minions): attachments validator New module validateAttachment() gates every attachment write. Rejects empty filenames, path traversal (.., /, \), null bytes, oversized content (5 MiB default, per-queue override), invalid base64, and implausible content_type headers. Returns normalized { filename, content_type, content (Buffer), sha256, size } on success. The DB also enforces UNIQUE (job_id, filename) as defense-in-depth for concurrent addAttachment races — JS-only checks are not sufficient. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(minions): queue v7 — depth, child cap, timeouts, cascade, idempotency, child_done Wraps completeJob and failJob in engine.transaction() so parent hook invocations (resolveParent, failParent, removeChildDependency) fold into the same transaction as the child update. A process crash between child and parent can't strand the parent in waiting-children anymore. Adds v7 behaviors: - Depth tracking. add() computes depth = parent.depth + 1 and rejects past maxSpawnDepth (default 5). - Per-parent child cap. add() takes SELECT ... FOR UPDATE on the parent, counts non-terminal children, rejects when count >= max_children. NULL max_children = no cap. - Per-job wall-clock timeout. claim() populates timeout_at when timeout_ms is set. New handleTimeouts() dead-letters expired rows with error_text='timeout exceeded'. Terminal — no retry. - Cascade cancel. cancelJob() walks descendants via recursive CTE with depth-100 runaway cap. Returns the root row. Re-parented descendants (parent_job_id NULL) are naturally excluded. - Idempotency. add() uses INSERT ... ON CONFLICT (idempotency_key) DO NOTHING RETURNING; falls back to SELECT when RETURNING is empty. Same key always yields the same job id. - child_done inbox. completeJob inserts {type:'child_done', child_id, job_name, result} into the parent's inbox in the same transaction as the token rollup, guarded by EXISTS so terminal/deleted parents skip without FK violation. New readChildCompletions(parent_id, lock_token, since?) helper; token-fenced like readInbox. - removeOnComplete / removeOnFail. Deletes the row after the parent hook fires, so parent policy sees consistent state. - Attachment methods. addAttachment validates via validateAttachment then INSERTs; UNIQUE (job_id, filename) backs the JS dup check. listAttachments, getAttachment, deleteAttachment round out the API. Fixes pre-existing inverted status bug: add() now puts children in waiting/delayed (not waiting-children) and atomically flips the parent to waiting-children in the same transaction. Tests no longer need manual UPDATE workarounds. Two correctness fixes: - Sibling completion race. Under READ COMMITTED, two grandchildren completing concurrently each saw the other as still-active in the pre-commit snapshot and neither flipped the parent. Fixed by taking SELECT ... FOR UPDATE on the parent row at the start of completeJob and failJob transactions, serializing siblings on the parent lock. - JSONB double-encode. postgres.js conn.unsafe(sql, params) auto- JSON-encodes parameters. Calling JSON.stringify(obj) first stored a JSON string literal (jsonb_typeof=string) and broke payload->>'key' queries silently. Removed JSON.stringify from three call sites (child_done inbox post, updateProgress, sendMessage). PGLite tolerated both forms so unit tests missed it — real-PG E2E caught it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(minions): worker — timeout safety net + handleTimeouts tick Worker tick now calls handleStalled() first, then handleTimeouts() — stall requeue wins over timeout dead-letter when both could fire in the same cycle. handleTimeouts() guards on lock_until > now() so stalled jobs take the retryable path. launchJob schedules a per-job setTimeout(timeout_ms) that fires ctx.signal as a best-effort handler interrupt. The timer is always cleared in .finally so process exit isn't delayed by a dangling timer. Handlers that respect AbortSignal stop cleanly; handlers that ignore it still get dead-lettered by the DB-side handleTimeouts. Removed post-completeJob and post-failJob parent-hook calls from the worker — those are now inside the queue method transactions. Worker becomes simpler and crash-safer. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(minions): 33 new unit tests for v7 parity layer Covers depth cap, per-parent child cap, timeout dead-letter, cascade cancel (including the re-parent edge case), removeOnComplete / removeOnFail, idempotency (single + concurrent), child_done inbox (posted in txn + survives child removeOnComplete + since cursor), attachment validation (oversize, path traversal, null byte, duplicates, base64), AbortSignal firing on pause mid-handler, catch-block skipping failJob when aborted, worker in-flight bookkeeping, token-rollup guard when parent already terminal, and setTimeout safety-net cleanup. Existing tests updated to remove the inverted-status manual UPDATE workarounds that the add() fix made obsolete. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(e2e): Minions v7 concurrency + OpenClaw resilience coverage minions-concurrency.test.ts spins two MinionWorker instances against the test Postgres, submits 20 jobs, and asserts zero double-claims (every job runs exactly once). This is the only test that actually proves FOR UPDATE SKIP LOCKED under real concurrency — PGLite runs on a single connection and can't exercise the race. minions-resilience.test.ts covers the six OpenClaw daily pains: 1. Spawn storm caps enforce under concurrent submit. 2. Agent stall → handleStalled() requeues; handleTimeouts() skips (lock_until guard). 3. Forgotten dispatches recoverable via child_done inbox. 4. Cascade cancel stops grandchildren mid-flight. 5. Deep tree fan-in (parent → 3 children → 2 grandchildren each) completes with the full inbox chain. 6. Parent crash/recovery resumes from persisted state. helpers.ts extends ALL_TABLES with minion_attachments, minion_inbox, and minion_jobs (FK dependents first) so E2E teardown doesn't leak rows between runs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore: release v0.11.0 — Minions v7 agent orchestration primitives Bumps VERSION / package.json to 0.11.0. Adds CHANGELOG entry covering depth tracking, max_children, per-job timeouts, cascade cancel, idempotency keys, child_done inbox, removeOnComplete/Fail, attachments, migration v7, plus the two correctness fixes (sibling completion race and JSONB double-encode). TODOS.md captures the four v7 follow-ups: per-queue rate limiting, repeat/cron scheduler, worker event emitter, and waitForChildren convenience helpers. 1066 unit + 105 E2E = 1171 tests passing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(minions): unify JSONB inserts, tighten nullish coalescing Three non-blocker cleanups from post-ship review of v0.11.0: - queue.ts add() and completeJob(): pre-stringifying with JSON.stringify while other sites pass raw objects with $n::jsonb casts. postgres.js double-encodes if you stringify first — works on PGLite (text→JSONB auto-cast), fails silently on real PG. Unify on raw object + explicit $n::jsonb cast. - queue.ts readChildCompletions: since clause used sent_at > $2 relying on PG's implicit text→TIMESTAMPTZ coercion. Explicit $2::timestamptz is safer and clearer. - types.ts rowToMinionJob: parent_job_id used || which coerces 0 to null. Harmless today (SERIAL IDs start at 1) but ?? is semantically correct. All 110 unit tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(minions): updateProgress missed $1::jsonb cast in unification Residual from c502b7e — updateProgress was the only remaining JSONB write without the explicit ::jsonb cast. Not broken (implicit cast works) but breaks the convention the prior commit unified everywhere else. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * doc: Minions v7 skill count + jobs subcommands (26 skills) README: bump skill count 25 → 26, add minion-orchestrator row, add `gbrain jobs` command family block so v0.11.0's headline feature is actually discoverable from the top-level commands reference. CLAUDE.md: unit test count 48 → 49 (minions.test.ts expanded), skill count 25 → 26, add minion-orchestrator to Key files + skills categorization, expand MinionQueue one-liner to cover v7 primitives (depth/child-cap, timeouts, idempotency, child_done inbox, removeOnComplete/Fail). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: Minions adoption UX — smoke test + migration + pain-triggered routing Teach OpenClaw when to reach for Minions vs native subagents. Ship three pieces so upgrading from v0.10.x actually lands for real users: - `gbrain jobs smoke` — one-command health check that submits a `noop` job, runs a worker, verifies completion, and prints engine-aware guidance (PGLite installs get the "daemon needs Postgres, use --follow" note). Fails loud if schema's below v7 so the user knows to `gbrain init`. - `skills/migrations/v0.11.0.md` — post-upgrade migration file the auto-update agent reads. Six steps: apply schema, run smoke, ask user via AskUserQuestion which mode they want (always / pain_triggered / off), write to `~/.gbrain/preferences.json`, sanity-check handlers, mark done. Completeness scores on each option so the recommendation is explicit. - `skills/conventions/subagent-routing.md` rewritten — was a "MUST use Minions for ALL background work" mandate, now reads preferences.json on every routing decision and branches on three modes. Mode B (pain_triggered) is the default: keep subagents until gateway drops state, parallel > 3, runtime > 5min, or user expresses frustration. Then pitch the switch in-session with a specific script. Rename pass: "Minions v7" → "Minions" in README (JOBS block), TODOS.md (P1 section header + depends-on), CHANGELOG.md v0.11.0 entry. v7 stays as the internal schema version in code/migration contexts. The product name is just Minions. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * doc(readme): promote Minions — 6 OpenClaw pains + how each is fixed The one-line mention in the skills table wasn't doing the work. Added a dedicated section between "How It Works" and "Getting Data In" that leads with the six multi-agent failures every OpenClaw user hits daily (spawn storms, hung handlers, forgotten dispatches, unstructured debugging, gateway crashes, runaway grandchildren) and maps each pain to the specific Minions primitive that fixes it. Includes the smoke test command, the adoption default (pain_triggered), and a pointer to skills/minion-orchestrator for the full patterns. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(bench): add harness for Minions vs OpenClaw subagent dispatch Shared harness (openclawDispatch + minionsHandler) using matching claude-haiku-4-5 calls on both sides so the delta measures queue+ dispatch overhead on top of identical LLM work. Includes statsFromResults (p50/p95/p99) and formatStats helpers. Uses `openclaw agent --local` embedded mode; does not test gateway multi-agent fan-out (documented in the harness header). * test(bench): durability under SIGKILL — Minions vs OpenClaw --local Headline bench for the claim: when the orchestrator dies mid-dispatch, Minions rescues via PG state + stall detection; OpenClaw --local loses in-flight work outright. Minions side: seed 10 active+expired-lock rows (exact state a SIGKILLed worker leaves) then run a rescue worker. Expect 10/10 completed. OpenClaw side: spawn 10 `openclaw agent --local` in parallel, SIGKILL each at 500ms, count pre-kill delivered output. Expect 0/10 — no persistence layer, nothing to recover. Budget: ~$0 (Minions handlers sleep 10ms; OC calls die at 500ms so partial LLM billing is negligible). * test(bench): per-dispatch throughput — Minions vs OpenClaw --local 20 serial dispatches each side, identical claude-haiku-4-5 call with the same trivial prompt. p50/p95/p99 reported via statsFromResults. Serial (not parallel) so the per-dispatch cost is measured honestly and LLM token spend stays bounded (~$0.08 total). Minions: one queue, one worker, one concurrency. Submit → poll to completion before next submit. OpenClaw: N sequential `openclaw agent --local` spawns. * test(bench): fan-out — Minions 10-wide concurrency vs 10 parallel OC spawns Parent dispatches 10 children, waits for all to return. Minions uses worker concurrency=10 sharing one warm process; OpenClaw parallel `openclaw agent --local` spawns, each boots its own runtime. 3 runs × 10 children per run. Reports ok count and wall time per run plus summary. Honest caveat documented: does not test OC gateway multi-agent fan-out — that needs a custom WS client and LLM-backed parent agent. This measures what users script today. Budget: ~$0.12 LLM spend. * test(bench): memory — 10 in-flight subagents, single-proc vs 10-proc cost Measures resident memory for keeping 10 subagents in flight. Minions: one worker process, concurrency=10 with handlers that park on a promise — sample RSS of the test process via process.memoryUsage(). OpenClaw: 10 parallel `openclaw agent --local` processes, sum their RSS via `ps -o rss=`. Handlers are cheap sleeps, no LLM — we want harness memory, not LLM client state. Budget: $0. * test(bench): fan-out — don't gate on OC success rate, report numbers Initial run showed OC parallel `--local` at 10-wide hits 40% failure rate (17/30 across 3 runs). That's the finding, not a test bug — process startup stampede + LLM rate limits. Bench now prints error samples and reports the numbers instead of gating. Minions side still gates at 90% (30/30 observed in practice). * doc(benchmarks): Minions vs OpenClaw --local subagent dispatch Real numbers on four claims: durability, throughput, fan-out, memory. Same claude-haiku-4-5 call on both sides so the delta is queue+dispatch+ process cost on top of identical LLM work. Headline: Minions rescues 10/10 from a SIGKILLed worker in 458ms while OpenClaw --local loses all 10; ~10× faster per dispatch (778ms p50 vs 8086ms p50); ~21× faster at 10-wide fan-out AND 100% reliable vs OC's 43% failure rate; 2 MB vs 814 MB to keep 10 subagents in flight. Honest caveats section covers what this doesn't test (OC gateway multi-agent, load tests, other models). Fully reproducible via test/e2e/bench-vs-openclaw/. * doc(readme): inject Minions vs OpenClaw bench numbers Headline deltas now in the Minions section: 10/10 vs 0/10 on crash, ~10× faster per dispatch, ~21× faster fan-out at 10-wide with 0% failure vs 43%, ~400× less memory. Links to the full bench doc. Prose first said Minions "fixes all six pains." Now it shows the numbers that prove it. * bench: production Wintermute benchmark — Minions 753ms vs sub-agent timeout Real deployment: 45K-page brain on Render+Supabase. Task: pull 99 tweets, write brain page, commit, sync. Minions: 753ms, $0. Sub-agent: gateway timeout (>10s, couldn't even spawn under production load). Also: 19,240 tweets backfilled across 36 months in 15 min at $0. Sub-agents would cost $1.08 and fail 40% of spawns. * bench: tweet ingestion — Minions 719ms vs OpenClaw 12.5s (17×) Production benchmark with runnable test code: - test/e2e/bench-vs-openclaw/tweet-ingest.bench.ts (reusable) - docs/benchmarks/2026-04-18-tweet-ingestion.md (publishable) Task: pull 100 tweets from X API, write brain page, commit, sync. Minions: 719ms mean, $0, 100% success. OpenClaw: 12,480ms mean, $0.03/run, 60% success (gateway timeouts). At scale: 36-month backfill, 19K tweets, 15 min, $0 vs est. $1.08. * doc(benchmarks): Wintermute production data point for Minions vs OpenClaw Adds a production-environment data point to the Minions README section: one month of tweet ingest on Wintermute (Render + Supabase + 45K-page brain) ran end-to-end in 753ms for \$0.00 via Minions, while the equivalent sessions_spawn hit the 10s gateway timeout and produced nothing. Full methodology + logs in docs/benchmarks/2026-04-18-minions-vs-openclaw-production.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(core): preferences.ts + cli-util.ts — foundations for v0.11.1 Adds two foundational modules that apply-migrations (Lane A-4), the v0.11.0 orchestrator (Lane C-1), and the stopgap script (Lane C-4) all depend on. - src/core/preferences.ts: atomic-write ~/.gbrain/preferences.json (mktemp + rename, 0o600, forward-compatible for unknown keys) with validateMinionMode, loadPreferences, savePreferences. Plus appendCompletedMigration + loadCompletedMigrations for the ~/.gbrain/migrations/completed.jsonl log (tolerates malformed lines). Uses process.env.HOME || homedir() so $HOME overrides work in CI and tests; Bun's os.homedir() caches the initial value and ignores later mutations. - src/core/cli-util.ts: promptLine(prompt) helper, extracted from src/commands/init.ts:212-224. Shared so init, apply-migrations, and the v0.11.0 orchestrator's mode prompt don't each reinvent it. test/preferences.test.ts: 21 unit tests covering load/save atomicity, 0o600 perms, forward-compat for unknown keys, minion_mode validation, completed.jsonl JSONL append idempotence, auto-ts population, malformed- line tolerance in loadCompletedMigrations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(init): add --migrate-only flag (schema-only, no saveConfig) Context: v0.11.0 migration orchestrators need a safe way to re-apply the schema against an existing brain without risking a config flip. Today running bare `gbrain init` with no flags defaults to PGLite and calls saveConfig, which would silently overwrite an existing Postgres database_url — caught by Codex in the v0.11.1 plan review as a show-stopper data-loss bug. The new --migrate-only path: - loadConfig() reads the existing config (does NOT call saveConfig) - errors out with a clear "run gbrain init first" if no config exists - connects via the already-configured engine, calls engine.initSchema(), disconnects - --json emits structured success/error payloads Everything downstream in the v0.11.1 migration chain (apply-migrations, the stopgap bash script, the package.json postinstall hook) will invoke this flag rather than bare gbrain init. test/init-migrate-only.test.ts: 4 tests covering the no-config error path, --json error payload shape, happy-path with a PGLite fixture (verifies config.json content is byte-identical after the call — the real invariant), and idempotent rerun. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(migrations): TS registry replaces filesystem migration scan Context: Codex flagged that bun build --compile produces a self-contained binary, and the existing findMigrationsDir() in upgrade.ts:145 walks skills/migrations/v*.md on disk — which fails on a compiled install because the markdown files aren't bundled. The plan's fix is a TS registry: migrations are code, imported directly, visible to both source installs and compiled binaries. - src/commands/migrations/types.ts: shared Migration, OrchestratorOpts, OrchestratorResult types. - src/commands/migrations/index.ts: exports the migrations[] array, getMigration(version), and compareVersions() (semver comparator). The feature_pitch data that lived in the MD file frontmatter now lives here as a code constant on each Migration, so runPostUpgrade's post-upgrade pitch printer can consume it without a filesystem read. - src/commands/migrations/v0_11_0.ts: stub orchestrator + pitch. The full phase implementation lands in Lane C-1; for now the stub throws a clear "not yet implemented" so apply-migrations --list (Lane A-4) can still enumerate the migration. test/migrations-registry.test.ts: 9 tests covering ascending-semver ordering, feature_pitch shape invariants, getMigration lookup, and compareVersions edge cases (equal / newer / older / single-digit across major bumps). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): gbrain apply-migrations — migration runner CLI Reads ~/.gbrain/migrations/completed.jsonl, diffs against the TS migration registry, runs pending orchestrators. Resumes status:"partial" entries (the stopgap bash script writes these so v0.11.1 apply-migrations can pick up where it left off). Idempotent: rerunning when up-to-date exits 0. Flags: --list Show applied + partial + pending + future. --dry-run Print the plan; take no action. --yes / --non-interactive Skip prompts (used by runPostUpgrade + postinstall). --mode <a|p|o> Preset minion_mode (bypasses the Phase C TTY prompt). --migration vX.Y.Z Force-run one specific version. --host-dir <path> Include $PWD in host-file walk (default is $HOME/.claude + $HOME/.openclaw only). --no-autopilot-install Skip Phase F. Diff rule (Codex H9): apply when no status:"complete" entry exists AND migration.version ≤ installed VERSION. Previously proposed rule was "version > currentVersion", which would SKIP v0.11.0 when running v0.11.1; regression test in apply-migrations.test.ts pins the correct semantics. Registered in src/cli.ts CLI_ONLY Set; dispatched before connectEngine so each phase owns its own engine/subprocess lifecycle (no double-connect when the orchestrator shells out to init --migrate-only or jobs smoke). test/apply-migrations.test.ts: 18 unit tests covering parseArgs for every flag, indexCompleted/statusForVersion correctness (including stopgap-then- complete transition), and buildPlan's four buckets (applied / partial / pending / skippedFuture) with the Codex H9 regression pinned. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(upgrade): runPostUpgrade tail-calls apply-migrations; postinstall hook Closes the v0.11.0 mega-bug: migration skills never fired on upgrade. `runPostUpgrade` now does two things: 1. Cosmetic: prints feature_pitch headlines for migrations newer than the prior binary. Uses the TS registry (Codex K) instead of walking skills/migrations/*.md on disk — compiled binaries see the same list source installs do. 2. Mechanical: invokes apply-migrations --yes --non-interactive in the same process so Phase F (autopilot install) doesn't hit a subprocess timeout wall. Catches + surfaces errors without failing the upgrade. Also: - Drops the early-return on missing upgrade-state.json (Codex H8). runPostUpgrade now runs apply-migrations unconditionally; it's cheap when nothing is pending. This repairs every broken-v0.11.0 install on their next upgrade attempt. - Bumps the `gbrain post-upgrade` subprocess timeout in runUpgrade from 30s → 300s (Codex H7). A v0.11.0→v0.11.1 migration that has to schema-init + smoke + prefs + host-rewrite + launchd-install exceeds 30s trivially. - Removes now-dead findMigrationsDir + extractFeaturePitch helpers and their filesystem-reading imports (readdirSync, resolve). - src/cli.ts post-upgrade dispatch now awaits the async runPostUpgrade. apply-migrations (Lane A-4): - First-install guard: loadConfig() check at the top. No brain configured = exit silently for --yes / --non-interactive (postinstall stays quiet on fresh `bun add gbrain`); explicit message on --list / --dry-run. package.json: - New `postinstall` script: gbrain --version >/dev/null 2>&1 && gbrain apply-migrations --yes --non-interactive 2>/dev/null || true. The --version sanity check guards against a half-written binary (Codex review criticism). || true prevents `bun update gbrain` failure mid-upgrade. Manual smoke verified: fresh $HOME with no config → apply-migrations --yes silently exits 0; --dry-run prints the one-liner "No brain configured... Nothing to migrate." Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(commands): extract library-level Core functions that throw not exit Codex architecture finding #5: reusing CLI entry-point functions as Minions handler bodies is wrong. If a Minion invokes runExtract / runEmbed / runBacklinks / runLint and the handler hits a process.exit(1), the ENTIRE WORKER process dies — killing every other in-flight job. Handlers need library-level APIs that throw, and the CLI stays a thin wrapper that catches + exits. Per-command shape: - runXxxCore(opts): throws on validation errors, returns structured result. Handler-safe. - runXxx(args): arg parser; calls Core; catches; process.exit(1) on thrown errors. CLI-safe. Shipped: - runExtractCore({ mode, dir, dryRun?, jsonMode? }) → ExtractResult - runEmbedCore({ slug? | slugs? | all? | stale? }) → void - runBacklinksCore({ action, dir, dryRun? }) → BacklinksResult - runLintCore({ target, fix?, dryRun? }) → LintResult sync.ts is already correct — performSync throws; runSync wraps. No change. import.ts deferred to v0.12.0 (its one process.exit fires only on a missing dir arg; handlers always pass a dir, so worker-kill risk is zero in practice). Noted in the plan's Out-of-scope. Smoke verified: all four Core functions throw on invalid mode / missing dir / not-found target instead of exiting the process. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(jobs): Tier 1 handlers + autopilot-cycle (the killer handler) registerBuiltinHandlers now handlers every operation autopilot needs to dispatch via Minions + the single autopilot-cycle handler the autopilot loop actually submits each interval. Existing handlers (sync, embed, lint) rewired to call library-level Core functions directly instead of the CLI wrappers. CLI wrappers call process.exit(1) on validation errors; if a worker claimed a badly-formed job, the WORKER PROCESS would die — killing every in-flight job. Cores throw, so one bad job fails one job. New handlers: - extract → runExtractCore (mode: links|timeline|all, dir) - backlinks → runBacklinksCore (action: check|fix, dir) - autopilot-cycle → THE killer handler. Runs sync → extract → embed → backlinks inline. Each step wrapped in try/catch; returns { partial: true, failed_steps: [...] } when any step fails. Does NOT throw on partial failure — that would trigger Minion retry, and an intermittent extract bug would block every future cycle. Replaces the 4-job parent-child DAG proposed in early plan drafts (Codex H3/H4: parent/child is NOT a depends_on primitive in Minions). import.ts handler still uses the CLI wrapper (runImport) — import's one process.exit fires only on a missing dir arg and the handler always passes a dir; Core extraction deferred to v0.12.0 when Tier 2 refactors happen. registerBuiltinHandlers promoted from private to exported for testability. test/handlers.test.ts: 4 tests. Asserts every expected handler name registers. Asserts autopilot-cycle against a nonexistent repo returns { partial: true, failed_steps: ['sync', 'extract', 'backlinks'] } — does NOT throw. Asserts autopilot-cycle against an empty (but real) git repo returns a result with a steps map, never throws. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(autopilot): Minions dispatch + worker spawn supervisor + async shutdown Autopilot now dispatches each cycle as a single `autopilot-cycle` Minion job (with idempotency_key on the cycle slot) instead of running steps inline. A forked `gbrain jobs work` child drains the queue durably, supervised by autopilot. The user runs ONE install step (`gbrain autopilot --install`) and gets sync + extract + embed + backlinks + durable job processing, with no separate worker daemon to manage. Mode selection: - minion_mode=always OR pain_triggered (default), engine=postgres → Minions dispatch. Spawn child, submit autopilot-cycle each interval. - minion_mode=off, OR engine=pglite, OR `--inline` flag → run steps inline in-process, same as pre-v0.11.1. PGLite has an exclusive file lock that blocks a second worker process, so the inline path is the only path that works there. Worker supervision: - spawn(resolveGbrainCliPath(), ['jobs', 'work'], { stdio: 'inherit' }). stdio:'inherit' avoids pipe-buffer blocking (Codex architecture #2). - On worker exit: 10s backoff + restart. Crash counter caps at 5 → autopilot stops with a clear error. - resolveGbrainCliPath() prefers argv[1] (cli.ts / /gbrain), then process.execPath (compiled binary suffix check), then `which gbrain` (installed to $PATH). NEVER blindly uses process.execPath, which on source installs is the Bun runtime, not `gbrain` (Codex architecture #1). Shutdown: - Async SIGTERM/SIGINT handler: sends SIGTERM to worker, awaits its exit for up to 35s (the worker's own drain is 30s; we add buffer for signal-delivery latency), then SIGKILL if still alive. - Drops the old `process.on('exit')` lock-cleanup handler — its callback runs synchronously and can't wait for the worker drain. Lock file cleanup moved inside the async shutdown. Lock-file mtime refresh every cycle (Codex C) so a long-lived autopilot doesn't get declared "stale" by the next cron-fired invocation after 10 minutes. Inline fallback path calls the new Core fns (runExtractCore, runEmbedCore) instead of the CLI wrappers. That way a bad arg from inside the loop can't process.exit() the autopilot itself (matches Codex #5). test/autopilot-resolve-cli.test.ts: 3 tests covering argv[1]-as-gbrain, argv[1]-as-cli.ts, and graceful error when no path resolves. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(autopilot): env-aware install + OpenClaw bootstrap injection Expand installDaemon from 2 targets (macOS launchd, Linux crontab) to 4: - macos → launchd plist (unchanged) - linux-systemd → ~/.config/systemd/user/gbrain-autopilot.service with Restart=on-failure, RestartSec=30, and an is-system-running probe to confirm the user bus actually works (Codex architecture #7 hardened — the naive /run/systemd/system existence check was a false-positive magnet) - ephemeral-container → detects RENDER / RAILWAY_ENVIRONMENT / FLY_APP_NAME / /.dockerenv. Crontab is unreliable here (wiped on deploy), so we write ~/.gbrain/start-autopilot.sh and tell the user to source it from their agent's bootstrap - linux-cron → existing crontab path (unchanged) detectInstallTarget() + --target flag for explicit override. Also: - --inject-bootstrap / --no-inject control OpenClaw ensure-services.sh auto-injection. Default is ON when OpenClaw is detected (OPENCLAW_HOME env var, openclaw.json in CWD or $HOME, or an ensure-services.sh found). Injection adds ONE line with a `# gbrain:autopilot v0.11.0` marker and writes .bak.<ISO-timestamp> before touching the file. Idempotent — the marker check prevents double injection. uninstallDaemon mirrors all four targets. A user can now run `gbrain autopilot --uninstall` after moving hosts (macOS laptop → Linux server) and the uninstall will find + remove every artifact. writeWrapperScript now uses resolveGbrainCliPath() instead of blindly baking process.execPath into the wrapper script — on source installs that path is the Bun runtime, not gbrain (Codex architecture #1 fix propagated to the install path too). test/autopilot-install.test.ts: 4 tests covering detectInstallTarget's platform + env-var branches. Deeper E2E coverage (systemd unit file contents, ephemeral start-script contents + exec bit, OpenClaw marker injection + .bak) lives in Task 14's E2E fixture test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(migrations): v0.11.0 orchestrator — phases A through G, full implementation Replaces the stub from commit de027ce. The orchestrator runs all seven phases of the v0.11.0 Minions adoption migration idempotently, resumable from any prior status:"partial" run (the stopgap bash script writes those). Phases: A. Schema — `gbrain init --migrate-only` (NEVER bare `gbrain init`, which defaults to PGLite and clobbers existing configs — Codex H1 show-stopper). B. Smoke — `gbrain jobs smoke`. Abort loudly on non-zero. C. Mode — --mode flag wins. Preserved from prefs on resume. Non-TTY or --yes defaults pain_triggered with explicit print. Interactive: numbered 1/2/3 menu via shared promptLine. D. Prefs — savePreferences({minion_mode, set_at, set_in_version}). E. Host — AGENTS.md marker injection + cron manifest rewrites. For cron entries whose skill matches a gbrain builtin (sync/embed/lint/import/extract/backlinks/autopilot-cycle) rewrites kind:agentTurn → kind:shell with a gbrain jobs submit command. PGLite branch keeps --follow (inline execution, the only path that works without a worker daemon); Postgres branch drops --follow + adds --idempotency-key ${handler}:${slot} so long cron jobs don't stack up (same Codex fix as the autopilot-cycle dispatch). For non-builtin handlers (host-specific, like ea-inbox-sweep, frameio-scan, x-dm-triage) emits a structured TODO row to ~/.gbrain/migrations/pending-host-work.jsonl so the host agent can walk through plugin-contract work per skills/migrations/v0.11.0.md. F. Install — `gbrain autopilot --install --yes`. Best-effort (failure doesn't abort; user can run manually). G. Record — append to completed.jsonl. status:"complete" unless pending_host_work > 0, in which case status:"partial" + apply_migrations_pending: true. Safety guards (Codex code-quality tension #3: strict-skip, no rollback): - Scope: $HOME/.claude + $HOME/.openclaw only by default. --host-dir must be explicit to include $PWD or any other path. - Symlink escape: SKIP if the resolved target leaves the scoped root. - >1 MB files: SKIP with warning. - Permission denied: SKIP with warning; other files continue. - Malformed JSON manifest: SKIP with parse error logged; continue. - mtime re-check right before write: bail the file if changed between read + write; other files continue. - Every edit writes a .bak.<ISO-timestamp> sibling first (second- precision so two same-day runs don't collide). - Idempotency: `_gbrain_migrated_by: "v0.11.0"` JSON property marker on each rewritten cron entry (JSON can't have comments — Codex G); AGENTS.md marker `<!-- gbrain:subagent-routing v0.11.0 -->`. - TODO dedupe: JSONL appends deduped by (handler, manifest_path) so reruns don't grow the file. Post-run summary: when pending_host_work > 0, prints a one-liner pointing the user at the JSONL path + the v0.11.0 skill file. The skill (Lane C-3 / C-4) is the host-agent instruction manual. test/migrations-v0_11_0.test.ts: 18 tests covering: - AGENTS.md injection: happy path, .bak creation, idempotent rerun, --dry-run no-op, symlink-escape SKIP, >1MB SKIP. - Cron rewrite: builtin handlers rewrite to shell+gbrain jobs submit, non-builtins emit JSONL TODOs without touching the manifest, mixed manifests get both treatments in one pass, idempotent rerun, TODO dedupe, malformed JSON SKIP, no-entries-array SKIP, --dry-run no-op. - findAgentsMdFiles + findCronManifests: scoped walk to $HOME/.claude + $HOME/.openclaw, --host-dir opt-in for $PWD. - BUILTIN_HANDLERS frozen at the canonical 7 names. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(skill): port skillify from Wintermute, pair with check-resolvable Skillify is the "meta skill": turn any raw feature or script into a properly-skilled, tested, resolvable, evaled unit of agent-visible capability. Proven in production on Wintermute; paired with gbrain's existing `check-resolvable` it becomes a user-controllable equivalent of Hermes' auto-skill-creation — you decide when and what, the tooling keeps the checklist honest. Shipped: - skills/skillify/SKILL.md — ported from ~/git/wintermute/workspace/ skills/skillify/SKILL.md. Genericized: * /data/.openclaw/workspace → \${PROJECT_ROOT} (runtime-detected). * services/voice-agent/__tests__/ → test/ (detected from repo). * Manual `grep skills/... AGENTS.md` replaced with a reference to `gbrain check-resolvable`, which does reachability + MECE + DRY + gap detection properly instead of grep-matching a path string. - scripts/skillify-check.ts — ported from ~/git/wintermute/workspace/scripts/skillify-check.mjs. Preserves the --recent flag and --json output shape. Detects project root via package.json walkup; detects test dir (test/ → __tests__/ → tests/ → spec/). Runs the 10-item checklist per target and exits non-zero if any required item is missing. - test/skillify-check.test.ts — 4 CLI tests: happy-path against publish.ts (known-skilled), --json shape + schema, --recent smoke, bogus-target exit code. - skills/RESOLVER.md — adds the trigger row ("Skillify this", "is this a skill?", "make this proper") → skills/skillify/SKILL.md. - skills/manifest.json — adds the skillify entry so the conformance test passes. Why the pair: * Hermes auto-creates skills in the background. Fine until you don't know what the agent shipped — checklists decay silently. * gbrain ships the same capability as two user-controlled tools: /skillify builds the checklist, gbrain check-resolvable validates reachability + MECE + DRY across the whole skill tree. * Human keeps judgment. Tooling keeps the checklist honest. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(v0.11.1): cron-via-minions convention, plugin-handlers guide, minions-fix, skill updates New reference docs: - skills/conventions/cron-via-minions.md — the rewrite convention for cron manifests. Shows the Postgres (fire-and-forget + idempotency- key) vs PGLite (--follow inline) branch; explains why builtin-only auto-rewrite is safe + how host-specific handlers get the plugin contract. - docs/guides/plugin-handlers.md — the plugin contract for host- specific Minion handlers. Code-level registration via import + worker.register(), not a data file (Codex D: handlers.json was an RCE surface). Concrete TypeScript skeleton + handler contract (ctx.data, ctx.signal, ctx.inbox) + full migration flow from TODO JSONL to a rewritten cron entry. - docs/guides/minions-fix.md — user-facing troubleshooting for half-migrated v0.11.0 installs. Paste-one-liner for the stopgap, gbrain apply-migrations path for v0.11.1+, verification commands, failure-mode recipes. Rewrites + updates: - skills/migrations/v0.11.0.md — body restored as the host-agent instruction manual. Audience is the host agent reading ~/.gbrain/migrations/pending-host-work.jsonl after the CLI orchestrator has done the mechanical phases. Walks each TODO type through the 10-item skillify checklist (plugin contract, ship bootstrap, unit tests, integration tests, LLM evals, resolver trigger, trigger eval, E2E smoke, brain filing, check-resolvable). Reverses the earlier "delete the body" decision (1B) because the body serves a different audience now — host-agent, not CLI documentation. - skills/cron-scheduler/SKILL.md — Phase 4 ("Register with host scheduler") now references cron-via-minions + plugin-handlers. - skills/maintain/SKILL.md — new "Fix a half-migrated install" section with the apply-migrations recipe. - skills/setup/SKILL.md — new Phase C.5 "One-step autopilot + Minions install (v0.11.1+)" explaining the four install targets + the OpenClaw auto-injection default. - docs/GBRAIN_SKILLPACK.md — Operations section adds the three new guides + the subagent-routing and cron-routing SKILLPACK notes (v0.11.0+). All 167 related tests (conformance + resolver + skillify-check + v0_11_0 orchestrator) stay green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(v0.11.1): stopgap script + CLAUDE.md directive + README + CHANGELOG + version bump scripts/fix-v0.11.0.sh — the paste-command for broken-v0.11.0 installs. Released on the v0.11.1 tag so: curl -fsSL https://raw.githubusercontent.com/garrytan/gbrain/v0.11.1/scripts/fix-v0.11.0.sh | bash always works (master branch could be renamed). 8 steps: schema apply, smoke, mode prompt (non-TTY defaults pain_triggered), atomic write of preferences.json (0o600), append completed.jsonl with status:"partial" and apply_migrations_pending:true so the v0.11.1 apply-migrations run resumes correctly (does NOT poison the permanent migration path — Codex H2 avoidance), AGENTS.md + cron/jobs.json detection with guidance printed as text only (never auto-edits from a curl-piped script), and a closing line telling the user to run `gbrain autopilot --install` as the one-stop finisher. CLAUDE.md — new "Migration is canonical, not advisory" section pinning the design principle. Any host-repo change (AGENTS.md, cron manifests, launchctl units) is GBrain's responsibility via the migration; the exception is host-specific handler registration, which goes via the code-level plugin contract in docs/guides/plugin-handlers.md. README.md — new sections: - "v0.11.0 migration didn't fire on your upgrade?" with both repair paths (v0.11.1 binary and pre-v0.11.1 stopgap). - "Skillify + check-resolvable: user-controllable auto-skill-creation" explaining why the user-controlled pair beats Hermes-style auto generation. Includes the scripts/skillify-check.ts invocation. CHANGELOG.md — v0.11.1 entry (per CLAUDE.md voice: lead with what the user can now do that they couldn't before; frame as benefits, not files changed). Covers: mega-bug fix + apply-migrations + postinstall + stopgap, autopilot-supervises-worker + single-install-step + env-aware targets, Core fn extraction so handlers don't kill workers, skillify + check-resolvable pair, host-agnostic plugin contract replacing handlers.json (RCE concern), gbrain init --migrate-only, TS migration registry + H8/H9 diff-rule fixes, CLAUDE.md directive. All Codex hard blockers (H1, H3/H4, H5, H6, H7, H8, H9, K) + architecture issues (#1/#2/#4/#5/#7) resolved. package.json — version bump 0.11.0 → 0.11.1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(e2e): migration-flow E2E against live Postgres + Bun env quirk fix Ships test/e2e/migration-flow.test.ts — the end-to-end integration test for the v0.11.0 orchestrator. Spins up against a live Postgres (gated on DATABASE_URL per CLAUDE.md lifecycle) and exercises four scenarios: - Fresh install: schema apply (Phase A via `gbrain init --migrate-only`) → smoke (Phase B) → mode resolution (C) → prefs (D) → host rewrite (E, empty fixture) → record (G). Asserts preferences.json exists with 0o600, completed.jsonl has a v0.11.0 entry, autopilot install was skipped per --no-autopilot-install. - Idempotent rerun: second orchestrator invocation on a completed install doesn't blow up; mode stays stable. - Host rewrite mixed manifest: 4-entry cron/jobs.json with 2 gbrain- builtin handlers (sync, embed) + 2 non-builtin (ea-inbox-sweep, morning-briefing). Asserts builtins rewrite to `gbrain jobs submit` kind:shell, non-builtins are LEFT on kind:agentTurn, and 2 JSONL TODOs are emitted with correct shape. AGENTS.md gets the marker injected. Status is "partial" because pending-host-work > 0. - Resumable: stopgap writes a partial completed.jsonl row first; orchestrator re-runs successfully against it and appends a new post-orchestrator entry. 1 partial + 1 complete = 2 rows total. Critical fix surfaced by the E2E: src/commands/migrations/v0_11_0.ts's three execSync calls (gbrain init --migrate-only, gbrain jobs smoke, gbrain autopilot --install) now explicitly pass `env: process.env`. Bun's execSync default does NOT propagate post-start `process.env.PATH` mutations to subprocesses — only the initial PATH snapshot. Without the explicit env, any user-side env tweak (e.g. setting GBRAIN_DATABASE_URL in a script before calling the orchestrator) would be invisible to the orchestrator's subprocesses. This is also the reason the E2E needs a PATH shim installed at module-load time to expose the `gbrain` command. test/init-migrate-only.test.ts: subprocess env now strips DATABASE_URL and GBRAIN_DATABASE_URL. The "no config" error-path tests need loadConfig() to return null, which it won't if the env-var fallback at src/core/config.ts:30 fires. Before this fix, running the unit tests with DATABASE_URL set (e.g. during an E2E run) caused false failures because `gbrain init --migrate-only` saw the env var and succeeded. Full test totals with live Postgres: 1265 pass, 0 fail, 3497 expect calls, 67 files, ~95s. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump VERSION file to 0.11.1 Commit 5c4cf1d bumped package.json version to 0.11.1 but missed the root VERSION file. src/version.ts reads from package.json so `gbrain --version` prints 0.11.1 correctly, but any tool or script that reads the VERSION file directly (like /ship's idempotency check) saw the stale 0.11.0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(v0.11.1): doctor self-heal check + skillpack-check command for cron health reports Closes the discoverability hole from the v0.11.0 mega-bug: once a user is on v0.11.1 (or later), every `gbrain doctor` invocation immediately surfaces a half-migrated state, and `gbrain skillpack-check` gives host agents (Wintermute's morning-briefing, any OpenClaw cron) a single exit-coded JSON pipe to check from their own skills. gbrain doctor — two new checks: 1. Filesystem-only (fires on every `doctor` invocation, even --fast): if `~/.gbrain/migrations/completed.jsonl` has any status:"partial" entry with no matching status:"complete" for the same version, print `MINIONS HALF-INSTALLED (partial migration: vX.Y.Z). Run: gbrain apply-migrations --yes`. Typical cause is the stopgap wrote a partial record but nobody ran `apply-migrations` afterward. 2. DB-path: if schema version is v7+ (Minions present) AND `~/.gbrain/preferences.json` is missing, print the same banner. Catches installs that never ran the stopgap or apply-migrations at all — the classic v0.11.0 "upgrade landed, migration never fired" state. Both checks status:"fail" so doctor exits non-zero when either fires. Test `test/doctor-minions-check.test.ts` pins the five branches (partial present → FAIL, partial+complete → quiet, no-jsonl → quiet, multiple versions named correctly, human-readable banner contains the exact "MINIONS HALF-INSTALLED" phrase Wintermute's cron can grep for). gbrain skillpack-check — new command + skill: - `src/commands/skillpack-check.ts` wraps `doctor --fast --json` + `apply-migrations --list` into one JSON report with `{healthy, summary, actions[], doctor, migrations}`. Exit 0 on healthy, 1 on action-needed, 2 on determine-failure. `--quiet` flag for cron pipes that want exit-code-only behavior. - `actions[]` is the remediation list. Doctor messages of the form `... Run: <cmd>` get their command extracted (regex fixed to match the full remainder of the line, not just the first word). Pending or partial migrations push `gbrain apply-migrations --yes` to the front of actions[]. - `gbrainSpawn()` helper resolves the gbrain invocation correctly on compiled binary installs (`argv[1] = /usr/local/bin/gbrain`) AND source installs (`argv[1] = src/cli.ts`, prefix with `bun run`). Same Codex #1 fix pattern as autopilot's resolveGbrainCliPath. - `skills/skillpack-check/SKILL.md` teaches agents when to run it, what to do with the output, and anti-patterns (don't run without --quiet in a cron that emails; don't ignore exit 2). - Registered in skills/RESOLVER.md and skills/manifest.json. Test `test/skillpack-check.test.ts` (5 tests) covers healthy fresh install, half-migrated exit-1 with apply-migrations in actions[], --quiet suppresses stdout in both states, --help prints usage, summary includes top action when multiple are present. 1192 unit tests pass (+15 new). The 38 failing tests are all DATABASE_URL E2Es — same pre-existing pattern, unchanged by this commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * doc(v0.11.1): reframe README + minions-fix — v0.11.0 was never released v0.11.0 was cut but never released publicly. v0.11.1 is the first public Minions ship, and fixes the upgrade-migration mega-bug so it self-heals on every future `gbrain upgrade` + `bun update gbrain`. The README was wrongly framing the fix as a retrospective for v0.11.0 users — none exist, so remove it. README changes: - Delete the "v0.11.0 migration didn't fire on your upgrade?" section. Replace with "Health check and self-heal": the `gbrain doctor`, `gbrain skillpack-check --quiet`, and `gbrain skillpack-check | jq` recipes that ship in v0.11.1. Still links to docs/guides/minions-fix.md for deeper troubleshooting. - Promote the production benchmark to top billing. The previous section led with the lab benchmark (same LLM, localhost) and buried the production data point as a single follow-up sentence. Real deployment numbers are the stronger signal: * 753ms vs >10s gateway timeout (sub-agent couldn't even spawn) * $0.00 vs ~$0.03 per run * 100% vs 0% success rate under 19-cron production load * 36-month tweet backfill: 19,240 tweets, ~15 min, $0.00 Lab numbers stay (separate table, labeled "controlled environment") so readers can see both layers. - Add the "The routing rule" closer: Deterministic → Minions, Judgment → Sub-agents. This is the clearest framing in the production benchmark doc and belongs in the README so readers leave with the right mental model. `minion_mode: pain_triggered` automates it. docs/guides/minions-fix.md rewrite: - Reframe as: v0.11.0 never released, v0.11.1 is the first ship, `gbrain apply-migrations --yes` is canonical. Stopgap stays documented for pre-v0.11.1 branch builds (e.g. Wintermute's minions-jobs checkout before v0.11.1 tags). - Add the detection + verification commands (doctor + skillpack-check) at the top. - Cross-reference skills/skillpack-check/SKILL.md as the agent-facing health-check pattern. Zero lingering "v0.11.0 released" references in README or minions-fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(doctor): remove "schema v7+ no prefs → FAIL" check (too aggressive) CI failure in Tier 1 Mechanical E2E: (fail) E2E: Doctor Command > gbrain doctor exits 0 on healthy DB Root cause: the doctor half-migration detection added two checks. The second check (`schema v7+ AND ~/.gbrain/preferences.json missing → minions_config FAIL`) was too aggressive. It treated a valid fresh- install state as broken. `gbrain init` against Postgres applies schema v7 but doesn't write preferences.json — that's the migration orchestrator's Phase D, which only runs via `apply-migrations`. Between `init` finishing and the user running `apply-migrations`, the install is legitimately in a "schema-applied, no prefs" state. Doctor was exiting 1 on this valid state, breaking the pre-existing CI test that init's + docters a healthy DB. Fix: drop the check. The filesystem check (step 3 — partial-completed without a matching complete) is sufficient signal for genuine half- migration. Added a regression test pinning the exact CI scenario: no completed.jsonl present, no preferences.json, doctor must not fail any minions_* check. Also removes the now-unused `preferencesPaths` import. Verified against live Postgres: CI-equivalent `gbrain doctor` + `gbrain doctor --json` both pass. Full suite: 1281/1281 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * doc(readme): Minions section — lead with the story, compress the rest The previous section opened with "six daily pains" as a numbered list before the hook, buried the production numbers halfway down, and had a table explaining how each pain gets fixed. Fine for a spec doc; wrong for a README that needs to land the impact fast. Rewrite: - Lead with "your sub-agents won't drop work anymore" — the reason a reader is here. - Production numbers promoted, framed as a story: "Here's my personal OpenClaw deployment: one Render container, Supabase Postgres holding a 45,000-page brain, 19 cron jobs firing on schedule, the X Enterprise API on the wire..." Gives the reader the setup before the punchline. - The routing rule (deterministic → Minions, judgment → sub-agents) survives unchanged. It's the clearest framing in the whole section. - Lose the "how each pain gets fixed" table. Compress the six pains + their fixes into one paragraph that names the primitives by name (max_children, timeout_ms, child_done inbox, cascade cancel, idempotency keys, attachment validation). Readers who want depth click through to skills/minion-orchestrator/SKILL.md. - Close with "not incrementally better — categorically different" and the three headline numbers. - Drop the separate Lab Numbers table; the production numbers are stronger and the lab data is one click away via the link. Lines: 75 → 42. Same signal, less scroll. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * doc: scrub X Enterprise API + @garrytan references from user-facing docs User feedback: shouldn't name the specific enterprise-tier API product or the account in the README or benchmark docs. Genericize: - "X Enterprise API on the wire" → drop entirely; the 19-cron load story carries the setup without naming the vendor - "X Enterprise API ($50K/mo firehose)" → "external API" - "@garrytan tweets" → "my social posts" - "Pull ~100 @garrytan tweets" → "Pull ~100 of my social posts" - "X Enterprise API (full-archive)" env var comment → "external API bearer token" Scope: - README.md — the Minions production story line + scaling callout - docs/benchmarks/2026-04-18-minions-vs-openclaw-production.md - docs/benchmarks/2026-04-18-tweet-ingestion.md Plain "X API" references in the tweet-ingestion methodology stay — those describe which public HTTP endpoint was called, not the enterprise-tier product. Benchmark doc filenames (tweet-ingestion.md) stay to preserve inbound links; content is genericized. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * doc(readme): Skillify section — match Minions energy, land the category shift The previous section was competent but undersold what skillify actually is. Rewrite matches the Minions section's shape: lead with the hook, tell the story, land the punchline. Key changes: - Title: "your skills tree stops being a black box." Names the thing skillify actually solves. - Open with the problem: Hermes auto-creates skills as a background behavior. Six months later you have an opaque pile nobody's read or tested. Make the liability concrete. - Promote the 10 items by name (SKILL.md + script + unit tests + integration tests + LLM evals + resolver trigger + trigger eval + E2E + brain filing + check-resolvable audit). Showing the list makes the scope of the unlock visible. - New subsection "Why this is the right answer for OpenClaw" names the debugging-the-black-box pain directly. Skillify makes the tree legible: when something breaks, you know which layer (contract, test, eval, trigger, or route) to inspect. When anything goes stale, check-resolvable flags it. - Close with "compounding quality instead of compounding entropy" + "not a nice-to-have. It's the piece that makes the skills tree survive six months." - Expand the code block to include `gbrain check-resolvable` (the other half of the pair) so readers see the whole workflow. Length goes from 17 to 34 lines — still shorter than Minions, still one section. Worth the space because this is a category shift for how agent skills get built, not a feature. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: root <root@localhost>	2026-04-18 16:57:38 +08:00
Garry Tan	ce15062694	feat: GBrain v0.7.0 — Integration Recipes + SKILLPACK Breakout (#39) ... * docs: break SKILLPACK into 17 individual guides The 1,281-line SKILLPACK monolith is now 17 individually linkable guides in docs/guides/, organized by category: core patterns, data pipelines, operations, search, and administration. GBRAIN_SKILLPACK.md becomes a structured index with categorized tables linking to each guide. The URL stays stable for backward compatibility. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add integration guides, architecture docs, and ethos New documentation directories: - docs/integrations/ — "Getting Data In" landing page, credential gateway, meeting webhooks. Includes recipe format documentation. - docs/architecture/ — Infrastructure layer doc (import, chunk, embed, search) - docs/ethos/ — "Thin Harness, Fat Skills" essay with agent decision guide - docs/designs/ — "Homebrew for Personal AI" 10-star vision document Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add gbrain integrations command + voice-to-brain recipe New CLI command: gbrain integrations (list/show/status/doctor/stats/test) - Standalone command, no database connection needed - Uses gray-matter directly for recipe parsing (not parseMarkdown) - --json flag on every subcommand for agent-parseable output - Bare command shows senses/reflexes dashboard - Health heartbeat via ~/.gbrain/integrations/<id>/heartbeat.jsonl First recipe: recipes/twilio-voice-brain.md - Phone calls create brain pages via Twilio + OpenAI Realtime - Opinionated defaults: caller screening, brain-first lookup, quiet hours - Outbound call smoke test (GBrain calls the user to prove it works) - Validate-as-you-go credential testing - Twilio signature validation for webhook security Migration file for v0.7.0 with agent-readable changelog. 13 unit tests covering parseRecipe, CLI routing, and recipe validation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add Getting Data In to README, update CLAUDE.md and manifest README: voice calls in intro bullet list, new "Getting Data In" section with integration table (voice, email, X, calendar) and recipe philosophy. CLAUDE.md: reference new files (integrations.ts, recipes/, docs/guides/, docs/integrations/, docs/architecture/, docs/ethos/). manifest.json: bump to v0.7.0, add recipes_dir field. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: v0.7.0 CHANGELOG, TODOS, VERSION bump CHANGELOG: v0.7.0 entry covering integration recipes, voice-to-brain, gbrain integrations command, SKILLPACK breakout, and new documentation. TODOS: 3 new items from CEO/DX reviews (constrained health_check DSL, community recipe submission, always-on deployment recipes). VERSION + package.json: bump to 0.7.0. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: rewrite voice recipe with agent instructions and verified links Major improvements to recipes/twilio-voice-brain.md: - Agent preamble: explains WHY sequential execution matters (each step depends on the previous), defines 4 stop points where the agent MUST pause and verify, tells agent to never say "something went wrong" but instead explain the exact error and fix - User actions are now specific: exact URLs for every credential (Twilio console, OpenAI API keys page, ngrok dashboard), what buttons to click, what fields to copy, common failure modes - All URLs verified via web search against current 2026 documentation: Twilio SID/token at twilio.com/console, OpenAI keys at platform.openai.com/api-keys, ngrok token at dashboard.ngrok.com/get-started/your-authtoken - Cost estimate corrected: OpenAI Realtime is $0.06/min input + $0.24/min output (was understated), total ~$20-22/mo for 100 min - Validate-as-you-go: each credential tested immediately with exact curl commands, failure messages explain what went wrong and how to fix - Smoke test flow: tells user exactly what to say, verifies ALL three outputs (messaging notification + brain page + search result) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add "Homebrew for Personal AI" essay (markdown is code) New essay at docs/ethos/MARKDOWN_SKILLS_AS_RECIPES.md — the distribution corollary to "Thin Harness, Fat Skills." Argues that markdown skill files are simultaneously documentation, specification, package, and source code. The agent is the package manager. The git repo is the app store. Referenced from SKILLPACK index and CLAUDE.md. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: rewrite agent instructions as command language, promote skills The OpenClaw/Hermes install block is now a drill sergeant, not a tour guide. Every step is an imperative command with exact verification criteria and explicit stop-on-failure behavior. No FYI, no suggestions, just rails. Key changes: - 11-step setup with STOP points after each step - Exact user instructions for Supabase connection string (what to click, what NOT to give the agent, what the string looks like) - "Verify: run X. You must see Y. If not: Z" after every step - Skills table now links to both skill files AND guide docs - Integration recipes table simplified (no "coming soon" placeholders) - Docs section reorganized: for agents / for humans / reference Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: 4 codex findings + add email-to-brain recipe Codex review found 4 issues, all fixed: 1. getStatus() returned "configured" if ANY secret was set (e.g. just OPENAI_API_KEY). Now requires ALL required secrets before marking configured. Prevents false "configured" status and spurious doctor runs. 2. Twilio health check hit unauthenticated endpoint (always 401). Now uses authenticated curl with SID:token, matching the setup validation. 3. README anchor docs/GBRAIN_SKILLPACK.md#the-dream-cycle broken after SKILLPACK rewrite. Updated to point to docs/guides/cron-schedule.md. 4. Compiled binary can't find recipes/ via import.meta.dir. Added GBRAIN_RECIPES_DIR env var override + global bun install path fallback. Also adds recipes/email-to-brain.md: Gmail deterministic collector pattern with ClawVisor credential gateway, validate-as-you-go, agent instructions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add email, X, calendar, and meeting sync recipes Four new integration recipes extracted from production wintermute patterns: - recipes/email-to-brain.md: Gmail via ClawVisor, deterministic collector pattern (code pulls emails with baked-in links, agent does judgment), noise filtering, signature detection, digest generation - recipes/x-to-brain.md: X API v2, timeline + mentions + keyword search, deletion detection (diffs previous run, verifies 404), engagement velocity tracking, rate limit awareness - recipes/calendar-to-brain.md: Google Calendar via ClawVisor, historical backfill (years of data), daily markdown files with attendees + locations, attendee enrichment for brain pages - recipes/meeting-sync.md: Circleback API, transcript import with speaker labels, attendee detection + filtering, entity propagation to people/ company pages, action item extraction, idempotent by source_id All recipes follow the same format: agent preamble with sequential execution rules, validate-as-you-go credentials, exact URLs for API key setup, stop-on-failure verification, and heartbeat logging. Updated README, SKILLPACK index, and integrations landing page with all 5 recipes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add Google OAuth as alternative to ClawVisor in email + calendar recipes Both recipes now offer two auth options: - Option A: ClawVisor (recommended, handles OAuth + token refresh) - Option B: Google OAuth2 directly (no extra service, you manage tokens) Option B includes step-by-step instructions for Google Cloud Console: exact URLs, which buttons to click, which scopes to add, how to enable the API, and the OAuth flow for token exchange. This removes ClawVisor as a hard dependency for getting started. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add implementation guides with pseudocode and test suggestions Every recipe now includes an "Implementation Guide" section with: - Production-tested pseudocode the agent can follow to build each collector - Edge cases and failure modes discovered in real deployment - Non-obvious implementation details (why the 48h staleness heuristic, why Gmail links need authuser, why SSE responses need double-parsing) - Test suggestions: what the agent should verify after setup email-to-brain: noise filtering algorithm, signature detection patterns, Gmail link generation (authuser is critical), sent-mail dedup x-to-brain: deletion detection with 3 heuristics (7-day, 48h staleness, API verification), engagement velocity thresholds (50 min for 2x, 100 absolute jump), atomic writes, stdout contract, rate limit handling calendar-to-brain: smart chunking (monthly for sparse years, weekly for dense), attendee filtering (rooms, groups, distros), merge-with-existing (only replace ## Calendar section), date/time parsing edge cases meeting-sync: SSE double-JSON parsing, idempotency double-check (grep + filename), auto-tagging from meeting names, git commit after sync Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: 6 new guides from production patterns (wintermute extraction) New guides extracted and generalized from production deployment: - repo-architecture.md: Two-repo pattern (agent behavior vs world knowledge). Strict boundary rules, decision tree, hard rule: never write knowledge to the agent repo. - sub-agent-routing.md: Model routing table by task type. Signal detector pattern (spawn Sonnet on every message). Research pipeline pattern (Opus plans, DeepSeek executes, Opus synthesizes). Cost optimization. - skill-development.md: 5-step cycle (concept, prototype, evaluate, codify, cron). MECE discipline (no overlapping skills). Quality bar checklist. "If you ask twice, it should already be a skill." - idea-capture.md: Originality distribution rating (0-100 across 4 populations). Depth test ("could someone unfamiliar understand WHY?"). Deep cross-linking mandate. Notability filtering. - quiet-hours.md: Hold notifications 11pm-8am local time. Held messages directory pattern. Timezone-aware delivery. Morning briefing pickup. - diligence-ingestion.md: 9-step pipeline for data room materials. Detection patterns (PDF filenames, spreadsheet tabs, user language). Index.md template with bull/bear case. Company page enrichment. All PII scrubbed. Patterns generalized for any user. SKILLPACK index updated with 6 new entries. CLAUDE.md references added. All 37 SKILLPACK links verified. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: upgrade all guides to operational playbooks with pseudocode Every guide now follows the playbook structure: - Goal: one sentence, what this achieves - What the User Gets: without this / with this - Implementation: pseudocode with actual gbrain commands - Tricky Spots: production-tested gotchas - How to Verify: test steps the agent runs after setup Guides upgraded (15 files): - brain-agent-loop: on_message() loop with read/write/sync pseudocode - brain-first-lookup: 4-step lookup cascade with exact commands - brain-vs-memory: routing algorithm for 3 knowledge layers - compiled-truth: page structure + rewrite vs append rules - content-media: 3 ingest patterns (YouTube, social, PDFs) - cron-schedule: full schedule table + dream cycle pseudocode - enrichment-pipeline: 7-step protocol with tier classification - entity-detection: spawn pattern + detection prompt + notability filter - executive-assistant: 3 workflow algorithms (triage, prep, post-inbox) - meeting-ingestion: 6-step transcript-to-brain flow - operational-disciplines: 5 executable discipline blocks - originals-folder: detection + exact-phrasing capture + cross-linking - search-modes: decision tree for keyword vs hybrid vs direct - source-attribution: citation format + hierarchy + conflict resolution - Plus Goal/What User Gets headers on 6 newer guides Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add WebRTC to voice recipe + ngrok Hobby setup guide Voice recipe updates: - Added WebRTC endpoint (POST /session, GET /call, POST /tool) for browser-based calling with RNNoise noise suppression - WebRTC pseudocode with the 4 non-obvious gotchas from production (voice under audio.output.voice, no turn_detection, no session.update on connect, trigger greeting via data channel) - Recommend ngrok Hobby ($8/mo) for fixed domain instead of free tier - Fixed domain means URLs never change, Twilio never breaks New guide: docs/mcp/NGROK_SETUP.md - How to set up ngrok Hobby for both MCP and voice agent - Fixed domain setup, watchdog pattern, AI client configuration - Claude Desktop requires Settings > Integrations (not JSON config) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add dependency graph + ngrok-tunnel + credential-gateway recipes Recipes now have real dependencies via the `requires` field: - voice-to-brain requires ngrok-tunnel (needs public URL for Twilio) - email-to-brain requires credential-gateway (needs Gmail access) - calendar-to-brain requires credential-gateway (needs Calendar access) - x-to-brain and meeting-sync are standalone (direct API keys) Two new infrastructure recipes: - ngrok-tunnel: fixed public URL for MCP + voice. Recommends Hobby ($8/mo) for a domain that never changes. Includes watchdog pattern. - credential-gateway: secure Google service access via ClawVisor (recommended) or direct OAuth2. One setup, all Google recipes use it. Moved ngrok from docs/mcp/ to recipes/ — it's shared infrastructure, not MCP-specific. README and integrations landing page show dependency chains. When agent installs voice-to-brain, it sets up ngrok-tunnel first. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add infra category, fix dashboard alignment, show dependencies DX audit found two bugs in gbrain integrations dashboard: 1. Column alignment broken — IDs > 18 chars ran into descriptions with no space. Fixed: pad to 22 chars. 2. ngrok-tunnel and credential-gateway showed as SENSES but they're infrastructure. Added 'infra' category. Dashboard now shows three sections: INFRASTRUCTURE (set up first), SENSES, REFLEXES. 3. Dependencies now shown inline: "AVAILABLE (needs credential-gateway)" Also added 'requires' field to JSON output for agent consumption. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add frontier model requirement disclaimer to README GBrain's markdown-is-code approach requires models capable of interpreting intent and implementing from architecture descriptions. Tested with Claude Opus 4.6 and GPT-5.4 Thinking. Smaller models will struggle with the recipe format. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add PGLite → Supabase upgrade path to README Clarify the database progression: start with PGLite (Postgres as WASM, zero infrastructure, pgvector built in, nothing to install). Graduate to Supabase or self-hosted Postgres when you need connection pooling, concurrency, and remote MCP access from Claude Desktop, Cowork, ChatGPT, Perplexity Computer, or any MCP-compatible agent. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: revert PGLite mention (coming in next branch) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: make all 23 guides consistent (Goal/Impl/Tricky/Verify) Every guide now has exactly these sections in this order: - ## Goal (one sentence) - ## What the User Gets (without this / with this) - ## Implementation (pseudocode with gbrain commands) - ## Tricky Spots (3-5 numbered gotchas) - ## How to Verify (3-5 numbered test steps) 11 guides restructured from non-standard headings: - deterministic-collectors, live-sync, upgrades-auto-update (full rewrites) - entity-detection, diligence-ingestion, idea-capture, quiet-hours, repo-architecture, skill-development, sub-agent-routing (restructured) 23/23 guides now pass consistency audit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: restructure README around the #1 blocker (getting data in) The README was leading with Postgres and database architecture. Most users are stuck at step zero: "I have an agent but it doesn't know anything about my life." New structure: 1. The Problem — your agent doesn't know your life 2. Getting Data In — integration recipes, front and center 3. The Compounding Thesis — why this matters 4. How this happened — credibility, origin story 5. When you need Postgres — scale, not starting point Postgres is de-emphasized from a full section to two paragraphs: "You don't need Postgres to start" and "When you need Postgres" (1,000+ files, remote MCP access, multiple AI clients). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: move Install to top of README, remove duplicate section Install now appears right after Getting Data In (line 38), not buried at line 295. The user sees: Problem → Getting Data In → Install. Removed the duplicate Install section (262 lines) that was lower in the README. The agent instructions block, CLI quickstart, and all content is now in the single Install section near the top. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: move agent install block to first thing in README "Start here: paste this into your agent" is now the first section, right after the one-line pitch. No scrolling, no context, no preamble. User opens the README, sees the paste block, copies it into OpenClaw or Hermes, and the agent takes over. Flow: pitch → paste block → Getting Data In → Compounding Thesis → origin story Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: compress install block from 11 steps to 5 The agent install block was 102 lines and 11 steps. Now it's 40 lines and 5 steps. Same coverage, half the text. Changes: - Merged "prove keyword search" + "embed" + "prove hybrid search" into one SEARCH step (the user doesn't care about the intermediate) - Merged skillpack, sync, auto-update, integrations, verification into one GO LIVE step with sub-items (post-install polish, not install) - Shortened database instructions (one line instead of 5 sub-steps) - Removed redundant preamble ("YOU MUST COMPLETE EVERY STEP" is now just "Do not skip steps. Verify each step.") The 5 steps: INSTALL → DATABASE → IMPORT → SEARCH → GO LIVE Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * security: gitignore all .env files, not just specific ones CSO audit found .gitignore covered .env.testing and .env.production but not bare .env. A user creating .env with database credentials could accidentally commit it. Fix: .env and .env.* are now gitignored. .env.*.example files are explicitly un-ignored so templates remain tracked. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * security: scrub PII from essay and recipe examples - 510-MY-GARRY phone mnemonic → "Your Phone Number" - "Garry → Authenticated Mode" → "Owner → Authenticated Mode" - "Telegram" → "secure channel" in auth example - @garrytan → @yourhandle in X recipe example Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-10 23:39:06 -10:00