9 Commits

Author	SHA1	Message	Date
David Breslauer	f8d4ce6fc4	feat(skills): add idea-lineage (#1830)	2026-06-07 17:48:48 -07:00
Garry Tan	03947665e4	v0.36.0.0 feat(skillpack): scaffold + reference + harvest (retire managed-block install) (#1130) ... * feat(skillpack): extract copyArtifacts shared helper (T1) Pure file-copy primitive for scaffold (gbrain→host) and harvest (host→gbrain). Atomic-refusal contract: symlink-reject + canonical-path containment validate every item before any write. Used by both directions of the v0.33 loop. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(skillpack): scaffold subcommand + SKILL.md frontmatter sources (T2) New scaffold.ts replaces the managed-block installer. One-time additive copy into the user's repo via copyArtifacts; refuses to overwrite existing files (user owns them). Partial-state policy: copies missing paired sources even when the skill dir already exists. bundle.ts extended with loadSkillSources + enumerateScaffoldEntries — paired source files declared in each SKILL.md's frontmatter sources: array, not in openclaw.plugin.json. Single source of truth, co-located with the skill. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(skillpack): reference command + apply-clean-hunks (T4 + T15) reference is the read-only diff lens with an agent-readable framing line. Pure-JS unified-diff producer + parser + applier (no patch(1) dependency). Two-way merge with documented limitation: without scaffold-time base tracking, applied hunks align everything to gbrain. The agent dry-runs reference first, then decides. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(skillpack): migrate-fence + scrub-legacy-fence-rows (T5 + T16) migrate-fence is the one-shot transition from the pre-v0.36 managed-block model. Strips begin/end markers and the cumulative-slugs receipt comment; preserves fence rows verbatim as user-owned routing during the transition to frontmatter discovery. Receipt-then-row fallback (F-CDX-8) covers stale/missing receipts. scrub-legacy-fence-rows is the opt-in cleanup after migrate-fence. Two-condition gate: removes a row only when skills/<slug>/ exists AND that skill's frontmatter declares non-empty triggers (proof frontmatter discovery covers it). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(skillpack): harvest + privacy linter (T6 + T7) The inverse loop: lift a proven skill from a host repo (~/git/wintermute, etc.) back into gbrain so other clients can scaffold it. --from <host-repo-root> is symmetric with scaffold's --workspace. Security: symlink rejection + canonical-path containment (mirrors validateUploadPath). Privacy: default-on linter scans harvested files against ~/.gbrain/harvest-private-patterns.txt plus built-in defaults (Wintermute, email, Slack channel patterns). Any match rolls back the copy and exits non-zero. --no-lint bypasses for the editorial workflow after a manual scrub. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(repo-root): cwd_walk_up tier for non-OpenClaw hosts (T9 + D3) autoDetectSkillsDir now walks up from cwd looking for any skills/ directory, ahead of the implicit ~/.openclaw/workspace fallback. cd ~/git/wintermute && gbrain skillpack scaffold ... finds wintermute automatically without requiring a RESOLVER.md/AGENTS.md to exist yet. R5 regression preserved: $OPENCLAW_WORKSPACE still wins when explicitly set. +5 test cases in test/repo-root.test.ts pin the new tier order and the R5 guard. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(skillpack): rewrite CLI dispatch, drop install + uninstall (T3 + T10) skillpack.ts dispatcher rewritten for the v0.36 contract: scaffold, reference (+ --apply-clean-hunks), migrate-fence, scrub-legacy-fence-rows, harvest, plus the existing list / diff / check. install and uninstall are gone — both exit non-zero with a hint pointing at scaffold / migrate-fence. Clean break, no deprecated alias. skillpack-check gains --strict for CI gating. When invoked as the subcommand `gbrain skillpack check`, default is informational (exit 0 even with drift); --strict opts back into the cron-friendly exit-1-on-issues behavior. Top-level gbrain skillpack-check preserves its existing exit semantics for backwards compat. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(skills): skillpack-harvest editorial workflow + resolver wiring (T8) The companion editorial skill for the gbrain skillpack harvest CLI. Walks the genericization checklist (scrub fork names, generalize triggers, lift fork- specific conventions to references) before the CLI runs. Routing-eval fixtures use paraphrased intents to avoid the intent_copies_trigger lint. Wires the new slug into openclaw.plugin.json#skills, skills/manifest.json, and skills/RESOLVER.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(skillpack): 9-case real-subprocess E2E flow (T11) Spawns gbrain as a subprocess against tempdir workspaces. Covers: scaffold first-run + re-run no-op, reference diff + --apply-clean-hunks, migrate-fence, scrub-legacy-fence-rows, harvest privacy-lint catch + --no-lint bypass, and the install removed-error path. No DATABASE_URL needed — skillpack is filesystem-only. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore: docs + VERSION + CHANGELOG for v0.36.0.0 (T13 + T14) Skillpacks as scaffolding, not amber. v0.36 retires the managed-block install model. Six new subcommands replace install + uninstall: scaffold, reference (with --apply-clean-hunks), migrate-fence, scrub-legacy-fence-rows, harvest, plus the existing list / diff / check (check gains --strict for CI gating). Routing comes from each skill's frontmatter triggers — gbrain does not touch your RESOLVER.md or AGENTS.md. Companion editorial skill skillpack-harvest drives the genericization checklist; default-on privacy linter catches Wintermute / email / Slack references before they leak into gbrain core. New docs guide at docs/guides/skillpacks-as-scaffolding.md walks the model and the migration path for pre-v0.36 installs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(ci): privacy checks — allow-list harvest-lint tests, scrub user-facing fork-name references CI's check-privacy.sh and check-test-real-names.sh both flagged the literal fork name across the v0.36 skillpack diff. Two failure modes, two fixes: 1. **Meta-rule-enforcement files** added to both allow-lists. The harvest privacy linter's whole job is to catch the banned literal leaking into gbrain; its source has the regex pattern, its tests verify the linter fires by feeding it the banned string, and the skill markdown documents the substitution policy. Same exception status as check-privacy.sh and check-proposal-pii.sh themselves. Files allow-listed: - src/core/skillpack/harvest-lint.ts - test/skillpack-harvest-lint.test.ts - test/skillpack-harvest.test.ts - test/e2e/skillpack-flow.test.ts - skills/skillpack-harvest/SKILL.md 2. **User-facing references** swapped for canonical phrasing per CLAUDE.md's responsible-disclosure rule. README + new docs guide + 4 src docstrings + 1 test now say 'your OpenClaw' / 'host agent repo' / 'agentRepo' var name. Behavior unchanged — only documentation strings touched. Verify gate (the script CI runs) passes locally: EXIT=0. Tests still pass: 60/60 across the affected files. llms-full.txt regenerated. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(test): update check-resolvable-cli expectation for cwd_walk_up tier Sister fix to the test/repo-root.test.ts update in commit a31418e3. The new v0.33 cwd_walk_up tier fires before repo_root when running from inside the gbrain repo — same skills/ dir matched, different source label. Behavior unchanged; the legacy repo_root tier is now functionally subsumed (kept in the type union for back-compat). CI shard 3 failure: test/check-resolvable-cli.test.ts:171. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(test): pin clock in sync_freshness boundary tests (CI flake) The 24h and 72h exact-boundary tests scheduled last_sync_at relative to Date.now() at construction time, then let the check call Date.now() again internally. CI scheduler jitter between the two reads pushed ageMs past the strict > thresholds by microseconds, dropping the 72h-boundary case into the fail branch instead of warn. Fix: add an optional `opts.now` test seam to checkSyncFreshness. The two boundary tests now capture t0 once and pass it both to the timestamp constructor and to the check, making ageMs deterministically equal to the boundary. The non-boundary tests (4d, 30h, 2h, etc.) don't need pinning — they're comfortably away from the > comparison. CI shard 1 flake: test/doctor.test.ts:479. Locally 48/48 doctor tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(skillpack): agent-onboarding readme + next-action hints on every CLI surface (DX review) DX audit of the v0.36 scaffold model surfaced one structural gap and four output gaps. When scaffolded files land on a downstream agent's disk, the agent had no agent-facing manifest telling it what to do — no routing contract, no upgrade flow, no two-way merge warning at the right surface. Fixes: 1. **New shared dep: skills/_AGENT_README.md.** Lands on every scaffold + migrate-fence alongside the existing _brain-filing-rules.md and _output-rules.md. Short, agent-readable contract: walk *.SKILL.md frontmatter triggers: for routing, gbrain is reference not law on upgrade, no managed-block fence anymore, two-way merge has known limitations. Single source of truth for the agent operating contract. 2. **scaffold stdout** prints a next-action hint pointing at the readme (with absolute path) and the reference --all upgrade-sweep command. 3. **reference stdout** adds per-category decision policy: - missing → scaffold again - differs → was edit intentional? keep it. Accidental? patch by hand or apply-clean-hunks after reading the two-way warning. 4. **reference --apply-clean-hunks** prints the two-way merge WARNING BEFORE the apply (to stderr, survives stdout redirect). Spells out that gbrain has no scaffold-time base and local edits in differing sections WILL be aligned to gbrain. Skipped in --json mode for machine consumers. On conflicts, prints how to inspect and patch. 5. **migrate-fence stdout** tells the agent its routing model just changed (fence gone, walk frontmatter now) and points at scrub-legacy-fence-rows as the eventual cleanup. References the new _AGENT_README for fresh-install agents. Smoke verified end-to-end: 16 files land (was 15, +1 for _AGENT_README), hint prints with absolute path, readme lands on disk. Tests + verify gate pass clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(skillpack): upgrade-time reference sweep + reference --since version filter (DX deferred items) Closes the last two DX gaps from the v0.36 audit: 1. **Post-upgrade reference sweep.** New `postUpgradeReferenceSweep` helper called at the end of `gbrain post-upgrade`. After migrations apply, auto-runs `reference --all` against the detected host workspace and prints a one-line-per-skill summary of drift. Five gates: GBRAIN_SKIP_REFERENCE_SWEEP env-var bypass, no detected workspace (silent), workspace IS gbrain repo (dev-mode silent), zero drift (silent), and pure-missing skills the host never scaffolded are filtered out as noise. All errors swallowed — never blocks post-upgrade. Helper accepts test-seam opts (gbrainRoot, targetWorkspace) for unit testability. 2. **`reference --all --since <version>`.** Filters the sweep to skills whose source actually changed in gbrain between <version> and HEAD, using a new `changedSlugsSinceVersion` helper in bundle.ts. Pure-JS git wrapper (spawnSync), no deps. Accepts bare '0.X.Y.Z' or 'v0.X.Y.Z' or commit SHA. Falls back loudly to full sweep when git can't resolve the ref (tarball install, missing tag). Test coverage added — total +32 new test cases: UNIT (15 cases): - test/skillpack-changed-since-version.test.ts (9 cases): git-aware filter against a fixture git repo. Covers null on non-repo, null on bad tag, empty array on no changes, single + multi-slug drift (deduped + sorted), bare + v-prefix version forms, non- skills/ path filtering, SHA-prefix ref form. - test/upgrade-reference-sweep.test.ts (6 cases): gate logic. Covers env-var bypass, zero drift, empty-host suppression, drift-detected output shape, dev-mode workspace==gbrain guard, error-swallowing contract. E2E (8 new cases in test/e2e/skillpack-flow.test.ts): - 10: scaffold lands skills/_AGENT_README.md - 11: scaffold stdout prints the Next: hint - 12: scaffold re-run (skipped-existing) suppresses the hint - 13: reference stdout prints per-category decision policy - 14: --apply-clean-hunks WARNING on stderr, not stdout - 15: --apply-clean-hunks --json suppresses the WARNING (bug fix surfaced here: code originally printed unconditionally, now gated on !json) - 16: migrate-fence stdout points at the new routing model - 17: --since with a bad tag falls back to full sweep with warn Local sweep: 579/579 pass across 18 affected test files, verify gate EXIT=0, llms regenerated. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(README): zero-base rewrite — 921 → 422 lines, refreshed catalog, MECE structure The README had drifted into a changelog dumping ground. Four 'New in vX.Y' paragraphs competed for the lead, 16 version tags scattered through headings, the production-numbers hook (17,888 pages, 4,383 people) was six months stale, and skills were described in three places (Skills section, Commands section, inline marketing prose). Zero-based rewrite: **Refreshed catalog** (surveyed live brain + live agent fork, broad strokes per CLAUDE.md privacy rules): - ~100K total brain items (was 17,888 in the old README — 6x stale) - ~16K people (was 4,383) - ~5K companies (was 723) - ~8K concepts, ~4K originals, ~3.5K daily notes - ~31K media (30K tweets, 179 books, papers/films/games/interviews) - 108 cron jobs running (was 21) - 273 skills in the live agent fork (35 bundled + 238 user-built) **Structure** — MECE, single source of truth per concept: 1. Hook + at-a-glance table (refreshed numbers) 2. Install (3 paths, terse) 3. What it does (5 capability areas — replaces 12 scattered sections) 4. Skills (categorized one-liners — 35 lines, was ~200) 5. How it works (one coherent flow — replaces 4 overlapping sections: Architecture, Knowledge Model, Knowledge Graph, Search, Why It Works) 6. Commands (terse cheatsheet — every command, one line each) 7. Docs (link map — points to docs/ for the heavy stuff) 8. Origin / Contributing / License **Cut entirely** (moved or deleted): - 4 'New in vX.Y' leads (→ CHANGELOG.md is the changelog) - 16 (vX.Y) version tags in section headings - Minions stats subsection (subsumed into hook + 'durable background work') - Voice section (was 12 lines of brand prose) - Engine Architecture detail (→ docs/architecture/) - File Storage section (→ docs/guides/storage-tiering.md) - Per-skill marketing prose (one-liner per skill in the table) The README is no longer the changelog. Future releases append to CHANGELOG.md; the README only changes when a structural capability does. llms-full.txt regenerated. Privacy check + verify gate pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(README): fix line-start '+' rendering bug + lead with eval evidence Two fixes in one: 1. **Markdown bug fix.** The OAuth 2.1 paragraph had `+ PKCE,` on a line start (column 1), which GitHub-flavored markdown interprets as a list marker — the line break before it broke the paragraph and rendered as an orphan first line followed by a bullet. Rewrote the OAuth 2.1 capabilities as inline-comma-separated, escaped the `+` semantics. Swept the whole file for the same bug class — no other instances. 2. **Maximum-sell mode for evals.** Surveyed every published benchmark in both this repo and ~/git/gbrain-evals. Strongest evidence pulled to the top: - **97.60% R@5 on the public LongMemEval _s (500 questions).** No LLM in the retrieval loop. $0.50 per 1000 queries. Beats MemPalace raw by a point on the same dataset, beats every academic dense retriever (Stella, Contriever, BM25). Mastra/Supermemory measure a different metric (QA accuracy with LLM judge) — flagged honestly. - **+31.4 points P@5 from the self-wiring knowledge graph** on BrainBench v0.20.0 (240-page rich-prose corpus, 145 relational gold queries). Separable, measured, load-bearing. Zero retrieval regression across seven releases (v0.16 → v0.20). New '## Benchmarks' section after Install: - Public benchmark table with cross-system comparison - In-house BrainBench scorecard with per-adapter Δ vs gbrain - Source-swamp resistance result (93.3% top-1 vs 80% grep-only) - Skill/prompt compression: 25KB → 13KB AGENTS.md, +13-17pp accuracy across Opus 4.7 / Sonnet 4.6 / Haiku 4.5 - 'Run your own evals' subsection with copy-pasteable commands for every eval surface (longmemeval, cross-modal, eval capture/replay, BrainBench) Tightened the lead's cost-comparison claim to what's defensible per the underlying eval doc (MemPal LLM-rerank $0.001/q vs gbrain $0.0005/q; dropped the overstated '6x' I'd written initially). Privacy + verify gate + build-llms test all pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(README): integrate the eval story into the lead, move jargon into 'Receipts on the evals' Previous lead dumped metric acronyms (R@5, P@5, P@5 deltas, MemPalace, Stella, Contriever, BM25) before the reader knew what gbrain does. A 'somewhat technical' reader hits the wall of jargon and bounces. Rewritten: **Lead (jargon-free, 3 paragraphs)** — describes the value in plain English, with two anchor numbers: - 'right answer in top 5 results 97.6% of the time' (not 'R@5 97.60%') - 'roughly 4x more relevant than plain vector RAG' (not '+31.4 pts P@5') - 'better than every comparable system that doesn't pay for a language- model call on every retrieval' (the load-bearing honest framing, without naming the competitors mid-hook) - ends with '[Receipts on the evals →]' linking down **'## Benchmarks' renamed '## Receipts on the evals'** with a glossary at the top defining R@5, P@5, and 'no LLM in the loop' in one line each. Then the full tables: LongMemEval cross-system (with the metric-mismatch flag for Mastra/Supermemory), in-house BrainBench scorecard, source-swamp resistance, and prompt compression. The competitor names + metrics stay here where readers who want the receipts can find them, with the glossary so the acronyms don't tax cold readers. Net: lead reads as 'here's what it does and the proof' instead of 'here are the benchmark numbers, figure out what they mean.' Comparison facts unchanged. Privacy + verify gate + build-llms test all pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(README): name LongMemEval explicitly + first-person voice in lead Two specific edits from user feedback: 1. 'the standard public benchmark for AI memory systems' → 'LongMemEval' (linked to the HuggingFace dataset). The benchmark has a name; use it. 2. 'Built by the President and CEO of Y Combinator to run his own AI agents' (passive third-person) → 'I'm the President and CEO of Y Combinator, and I use this 16 hours a day' (active first-person). Carried the voice change through the rest of the README — the downstream 'Garry's personal agent' line and the Origin section's 'Garry Tan needed... he'd ever drafted... so he built one' all flip to first person ('my personal agent', 'I needed', 'I'd ever drafted', 'so I built one'). The README is now consistently first-person from the author's voice instead of a hagiographic third-person framing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(README): add Multi-player and company brains section Three deployment patterns documented: 1. Single GBrain server + thin MCP clients (recommended). Tailscale private networking, OAuth scope, source-scoped clients, exhaustive what-clients-can/cannot-do lists. 2. Local PGLite + GStack for per-worktree code search. 3. Federated repos (advanced) — multiple servers indexing the same brain repo. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(README): tighten install path + add tech-orientation block + visceral query example Self-eval as a cold reader surfaced four gaps blocking a 10/10 first read: 1. Lead never says WHAT it is technically — CLI? service? cloud? local? Added a "What it is, technically" block right after the hook: open-source MIT, Bun CLI + MCP server, local-first, data stays on disk, MCP-native. 2. Install path optimized for committed users not evaluators. The old "recommended" path (deploy OpenClaw on Render, 8GB RAM) blocked anyone trying gbrain for the first time. Reordered into 3 paths by commitment: 60-second standalone CLI first, MCP for Claude Code / Cursor second, full agentic install third. 3. No example output showing what success looks like. Added a real sample `gbrain query` invocation with the hybrid-search result format so a reader can feel the experience before they install. 4. Privacy / data-locality unaddressed in lead. Now stated up front: embedding calls only hit external APIs if you configure them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-18 12:53:01 -07:00
Garry Tan	bd2fe8a1fa	v0.32.5 feat: gbrain-context OpenClaw context engine — deterministic temporal/spatial injection (#880) ... * feat: gbrain-context OpenClaw context engine — deterministic temporal/spatial injection Adds a context engine plugin that runs on every assemble() call to inject structured live context into the system prompt: - Garry's current local time (computed from heartbeat-state.json timezone) - Current location (city + timezone from heartbeat or flight data) - Home time when traveling (e.g. 'Mon 7:58 AM PT') - Active travel status - Quiet hours detection - Airport→timezone mapping for 30+ airports This kills the 'time warp' bug class where compacted sessions lose track of time/location. The engine delegates compaction to the legacy runtime and only owns systemPromptAddition injection. Zero LLM calls, <5ms. Files: - src/core/context-engine.ts — engine implementation (SDK-free, testable) - src/openclaw-context-engine.ts — plugin entry point (requires SDK) - test/context-engine.test.ts — 9 tests, all passing Enable: plugins.slots.contextEngine = 'gbrain-context' * feat: add activity injection — calendar events + open tasks in context block Reads memory/calendar-cache.json and ops/tasks.md to inject: - **Right now:** current meeting (with attendees) from calendar - **Coming up:** next 3 events within 4-hour window - **Open tasks:** unchecked items from Today section - Stale calendar warning when cache is >6 hours old Skips all-day events and generic markers (Home, OOO, Out of Office). Caps upcoming events at 3 and tasks at 5 to keep prompt lean. 15 tests passing (was 9). * v0.32.5 feat: gbrain-context OpenClaw context engine — deterministic temporal/spatial injection Ships PR #873 by @garrytan-agents (two underlying commits preserved): - f1dbe6ea — core engine (heartbeat + flights + airport→tz + quiet hours) - 14e85873 — activity injection (calendar events + open tasks + stale-cache warning) Kills the "time warp" bug class: when sessions compact, the LLM loses track of current time, location, and active threads. This engine owns the `systemPromptAddition` slot and reinjects live state on every `assemble()` call. Zero LLM calls, <5ms overhead, deterministic. Typecheck cleanup folded in: - `@ts-ignore` on the two `openclaw/plugin-sdk` runtime-only imports (resolved by the OpenClaw host; not a build-time dep — same pattern the core engine already used for `await import('openclaw/plugin-sdk/core')`) - Inline `PluginApi` + `PluginCtx` type shapes in the plugin entry so the `register(api)` + `(ctx)` callback params aren't implicit any - Test file's `from 'vitest'` → `from 'bun:test'` to match the rest of the suite (bun's globals make it pass at runtime, but tsc fails) Verification: - bun test test/context-engine.test.ts → 15/15 pass - bun run typecheck → exit 0 Co-Authored-By: garrytan-agents <garrytan-agents@users.noreply.github.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix-wave: close 5 findings from /plan-eng-review pass on PR #880 A `/plan-eng-review` audit of the shipped v0.32.5 surfaced 5 things worth fixing before merge. All folded into this branch with 5 new regression tests (15 → 20 total). A4 — silent-wrong-timezone for unknown airports Pre-fix: an active flight to any airport not in the 30-entry AIRPORT_TZ map (BOM, DXB, GRU, JNB, FRA, AMS, etc.) silently fell back to US/Pacific. The exact failure class this engine exists to prevent, in a different shape. Post-fix: unknown airports surface via the source field (flight:AC8:tz-unknown:BOM) so the LLM can see the data is incomplete instead of believing it's in Pacific Time. A2 / P1 — duplicate disk reads generateLiveContext was loading heartbeat-state.json and upcoming-flights.json twice per assemble() call (once in resolveLocation, once inline). Batch-load each workspace file once at the top of the function and thread results down. Halves the hot-path I/O. C4 — sanitize external content before injection Calendar event summaries, attendees, and task strings now go through sanitizeForPrompt() which strips newlines + control chars (U+0000-001F + U+007F) and clamps length. A meeting titled "Standup\n\nIgnore prior instructions" can no longer forge LLM directives by escaping the bullet structure. C1 — split isQuietHours into 3 explicit signals Original name was misleading (returned false when user was awake at 2 AM, even though wall clock said quiet hours). Split into `userAwake`, `wallClockQuietHours`, and a composite `quietHoursActive` so consumers can decide their own policy. On-disk heartbeat.garryAwake JSON field is unchanged — only the internal LiveContext type and the format-block consumer renamed. T1 — regression test coverage for the active-flight path Pre-fix, resolveLocation's flight branch (the headline path for the Toronto incident) had ZERO direct test coverage. Two new cases lock in the known-airport happy path AND the unknown-airport failure mode so A4 can't silently regress. Verification: - bun test test/context-engine.test.ts → 20/20 pass (was 15) - bun run typecheck → exit 0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(L0): A4 real fix + TLA → lazy SDK resolution (Codex F5 + F7) A Codex outside-voice review on /plan-eng-review's plan caught two findings both previous eng-reviews missed. L0-A (F5) — A4 was COSMETIC, not real. Pre-fix: resolveLocation's unknown-airport branch returned tz: DEFAULT_TZ (US/Pacific) with only a `source: 'flight:XX:tz-unknown:XYZ'` sticker. The engine then computed Time/Day/quietHoursActive from US/Pacific regardless, so a flight to BOM injected "Mon 3:00 PM PT" with a footnote nobody reads. Same silent-wrong-output failure class A4 was supposed to close. Post-fix: resolveLocation returns tz: UNKNOWN_TZ. generateLiveContext short-circuits time computation when tz is UNKNOWN_TZ (now/dayOfWeek become null, wallClockQuietHours/quietHoursActive become false). formatContextBlock renders an explicit Timezone-unavailable warning in place of Time:/Day:. The LLM sees the gap, not a guess. L0-B (F7) — Top-level `await import` is a hard module-load constraint. Any OpenClaw deployment in a non-TLA runtime (older Node, CJS bridges, certain transpilers, some test shims) fails BEFORE the plugin registers. The try/catch inside doesn't help — module load can't be caught by the consumer. Post-fix: SDK resolution moved to an `ensureSdkLoaded()` async helper called from assemble() and compact() on first invocation. Module loads cleanly in every runtime; the fallback path actually catches. Tests: - The cosmetic "tz-unknown sticker" assertion is replaced with the behavioral assertion: no US/Pacific Time, no Day field, explicit Timezone-unavailable warning present. - New L0-B contract test asserts engine creation does NOT trigger SDK load and the first compact() call exercises the lazy path. Verification: - bun test test/context-engine.test.ts → 21/21 pass (20 + L0-B contract) - bun run typecheck → exit 0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(L1): scrub real names from test fixtures + CI guard (CLAUDE.md privacy rule) The /plan-eng-review pass flagged pre-existing real-name leaks in PR #873's test fixtures. CLAUDE.md's privacy rule is unambiguous: "Never reference real people, companies, funds, or private agent names in any public-facing artifact." Tests are checked-in code, distributed with every release, and indexed by GitHub search. Fixture scrub (test/context-engine.test.ts, 5 substitutions): '1:1 with Diana' → '1:1 with @alice-example' 'diana@ycombinator.com' → 'alice@example.com' 'DM Technium re: Hermes PR' → 'DM @charlie-example re: agent-fork PR' 'Post open source manifesto — from YC Labs' → '... from a-team' '~~Reply to Bob McGrew~~ — DONE' → '~~Reply to bob-example~~ — DONE' Plus matching assertion updates. Adjacent scrub: test/link-extraction.test.ts line 523 fixture entry 'people/diana-hu' → 'people/alice-example' (single occurrence, never referenced elsewhere in the test). New CI guard (scripts/check-test-real-names.sh, ~120 lines): Designed per Codex F4 review: drop the broad corporate-email regex (@openai|google|stripe...) because legitimate billing/auth fixtures use those domains. Replace with two targeted lists: - BANNED_NAMES: exact-string list of known real identifiers (Diana, Wintermute, Hermes, Technium, McGrew, YC Labs) - BANNED_EMAILS: specific addresses (currently just diana@ycombinator.com) Plus ALLOWLIST of exact `file:string` pairs that are intentional and pre-existing (the user's own email; structural tests that ASSERT a banned name is absent and therefore MUST reference it literally). Scope: test/**/*.test.ts only. Historical CHANGELOG entries, doc examples, and skill READMEs each have their own scrub status and are out of scope for this guard. Wire-in: - New `bun run check:test-names` npm script - Added to `bun run verify` chain (pre-push gate) - Added to `bun run check:all` chain (local-only superset) Allowlist documents the structural references the guard correctly identifies but cannot meaningfully strip: - test/integrations.test.ts (regex pattern in personal-info filter test) - test/recency-decay.test.ts (regression-prevention assertions) - test/serve-stdio-lifecycle.test.ts (pre-existing comment) - test/extract.test.ts (pre-existing markdown-link fixture) These flagged-but-not-scrubbed entries belong to a broader repo-wide privacy-scrub pass (deferred TODO). Verification: - bun run check:test-names → exit 0 (no new banned strings) - bun test test/context-engine.test.ts → 21/21 pass - bun test test/link-extraction.test.ts → 98/98 pass Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(L2): plugin-shape e2e + compact fallback + selector map + race-condition JSDoc The unit suite at test/context-engine.test.ts exercised createGBrainContextEngine directly — that's the ENGINE, not the PLUGIN. Until this commit, nothing tested the actual OpenClaw plugin discovery + registration path. Codex outside-voice F1 flagged the gap: "we ship a plugin we don't test as a plugin." Layer 2 closures: T-NEW1 (plugin-shape e2e, test/e2e/openclaw-context-engine-plugin.test.ts, 3 tests): - Default export has the expected plugin-entry shape (id, name, description, register) - register() wires registerContextEngine with ENGINE_ID and a factory - Factory returns a working ContextEngine that injects Live Context and threads through the mocked memory-addition SDK call Implementation note: dropped the unused `definePluginEntry` import from src/openclaw-context-engine.ts. The wrapper was a type-tag with no behavior — OpenClaw's loader inspects the default export's shape, not the wrapping. Removing it eliminated a brittle build-time SDK import that blocked mock.module() interception (Codex F1 was right). Module now loads cleanly in any runtime. T-NEW4 (compact() fallback test, test/context-engine.test.ts): - Pins the no-runtime fallback shape so a refactor that drops the fallback or returns a different shape gets caught. - Codex F9 noted that without a real SDK boundary, a spy-on-delegate test is busywork. This commit keeps just the fallback assertion (no spy, no __internal export-for-tests hatch). T-NEW6 (heartbeat-write concurrency contract, src/core/context-engine.ts): - JSDoc on loadJsonFile documenting that producers MUST use atomic-rename writes (write-to-tmp + rename) to avoid partial-read races. The engine silent-degrades to defaults on parse failure; the contract makes the expectation explicit instead of buried in behavior. T-NEW5 (e2e selector map, scripts/e2e-test-map.ts): - Added entries mapping src/core/context-engine.ts and src/openclaw-context-engine.ts to the new plugin e2e file. ci:local:diff now narrows correctly for engine changes. Verification: - bun test test/context-engine.test.ts → 22/22 pass (21 + T-NEW4) - bun test test/e2e/openclaw-context-engine-plugin.test.ts → 3/3 pass - bun run typecheck → exit 0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(L3): ENGINE_VERSION → ENGINE_API_VERSION semantic + tasks.md size cap C-NEW1 — Engine version constant semantic. Pre-fix: `ENGINE_VERSION = '0.1.0'` looked like it should track package.json. It doesn't — it's the engine's CONTRACT version, bumped when the ContextEngine interface shape changes. Rename to ENGINE_API_VERSION makes that explicit. ENGINE_VERSION kept as a deprecated alias so existing v0.32.5 callers don't break. C-prior C2 — tasks.md size cap. resolveTodayTasks() now refuses to read a tasks file >1MB. Defends against a runaway file (clipboard-paste accident, log capture, etc) blocking every assemble() call with a multi-megabyte sync read. The size check uses statSync — same try/catch already handles missing-file via readFileSync throwing. Verification: - bun test test/context-engine.test.ts → 23/23 pass (22 + size-cap test) - bun test test/e2e/openclaw-context-engine-plugin.test.ts → 3/3 pass - bun run typecheck → exit 0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: CHANGELOG + TODOS for the Codex recalibration wave; allowlist sibling guard CHANGELOG.md — extend v0.32.5 entry with a "Codex outside-voice recalibration" subsection covering L0-A (A4 real fix), L0-B (TLA → lazy), the privacy guard redesign, the new plugin-shape e2e, and the deferred v0.32.6 items. Credits gpt-5-codex as the driver. TODOS.md — append "v0.32.6 follow-ups from PR #880" section with 13 deferred items: - Clock-injection seam (prerequisite for perf + snapshot tests) - T-NEW2 perf budget (with Codex F2 math-bug note) - T-NEW3 full-block snapshot test - C-NEW2 exports map entry (per Codex F8 — premature public API) - A3 .ts-extension resolution coupling - A5 typed openclaw/plugin-sdk ambient module shim - C-prior C5 loadJsonFile parse-error warn - C-prior C3 fractional-hour timezone offset - DST-boundary test - Multibyte sanitizer test - Dynamic airport-tz lookup (replace 30-entry static map) - DOC1 docs/openclaw-context-engine.md workspace contract - DOC2 CLAUDE.md "Key files" annotations - Repo-wide privacy scrub (24+ non-test matches) scripts/check-privacy.sh — allowlist sibling guard scripts/check-test-real-names.sh, which literally contains 'Wintermute' in its BANNED_NAMES list (same meta-rule-enforcement exception as check-privacy.sh's self-reference). Verification: bun run verify → exit 0 (full chain green: check:privacy + check:test-names + check:jsonb + check:progress + check:test-isolation + check:wasm + check:admin-build + check:admin-scope-drift + check:cli-exec + typecheck) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(L4): real openclaw-loads-the-plugin e2e — closes Codex F1 properly Until this commit, the gbrain-context plugin had two test paths: - test/context-engine.test.ts (23 unit tests against createGBrainContextEngine) - test/e2e/openclaw-context-engine-plugin.test.ts (3 e2e tests with mocked SDK) Both call our engine directly or shim the OpenClaw SDK. Codex outside-voice F1 (cited at v0.32.5 ship) flagged that nothing in the repo proves OpenClaw's actual plugin loader walks our entry file, calls register(api) against its real api object, and accepts the registration. The reviewer was right — shipping a plugin without an "OpenClaw actually loads it" test is a credibility hit on a feature whose entire purpose is to integrate with OpenClaw. L4 — test/e2e/openclaw-plugin-load-real.test.ts (6 tests, Tier 2): beforeAll: - Detects `openclaw` CLI; skips suite if missing - bun build src/openclaw-context-engine.ts → JS bundle (same packaging shape the release ships) - Writes minimal package.json + openclaw.plugin.json from templates - openclaw plugins install --link --dangerously-force-unsafe-install against an isolated --profile dir (won't touch user's openclaw state) Tests: 1. status=loaded, imported=true, activated=true 2. Default-export id/name/description metadata round-trips through openclaw's plugin loader unchanged 3. register(api) produced zero error-level diagnostics (only the expected trust warning for --link installs) 4. plugins.slots.contextEngine binding to "gbrain-context" passes openclaw config validate 5. openclaw plugins doctor surfaces zero errors for our plugin id 6. Public-SDK round-trip: imports registerContextEngine from openclaw/plugin-sdk (resolved via realpathSync on the openclaw binary's symlink so it works for Homebrew, npm -g, nvm, asdf, volta installs uniformly), registers our factory, then exercises assemble() and asserts the Live Context block appears afterAll: - Uninstalls the plugin (best-effort) + rm -rf the isolated profile dir + the tempdir fixture Fixture: test/fixtures/openclaw-plugin-real/ holds the manifest templates (package.json.template + openclaw.plugin.json.template). The test writes fresh copies into a per-run tempdir so the fixture itself stays read-only. Selector map: scripts/e2e-test-map.ts now points BOTH source files (src/core/context-engine.ts, src/openclaw-context-engine.ts) at BOTH the mocked-SDK plugin-shape e2e AND this real-loader e2e. ci:local:diff fires both on either change. Verification: - bun test test/e2e/openclaw-plugin-load-real.test.ts → 6/6 pass - bun test test/context-engine.test.ts test/e2e/openclaw-context-engine-plugin.test.ts test/e2e/openclaw-plugin-load-real.test.ts → 32/32 pass total - bun run typecheck → exit 0 - bun run verify → exit 0 (full chain green) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: garrytan-agents <garrytan-agents@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-11 21:49:57 -07:00
garrytan-agents	7be17261bc	v0.32.3.0 skill: functional-area-resolver — pattern for compressing routing tables (#859) ... * skill: compress-agents-md — functional-area resolver pattern Proven via A/B eval: 100% routing accuracy at 48% size reduction. Converts granular per-skill resolver rows into functional-area dispatchers with '(dispatcher for: ...)' sub-skill lists. Includes: - SKILL.md with full pattern docs, before/after examples, eval results - routing-eval.jsonl with 5 fixtures - Anti-patterns (resolver-of-resolvers pipe table = 15% accuracy) * skill: rename compress-agents-md → functional-area-resolver, cite prior art The contribution is a pattern (functional-area dispatcher with `(dispatcher for: ...)` clauses), not a file. Rename describes the contribution; triggers broaden to cover both AGENTS.md and RESOLVER.md phrasings. SKILL.md rewrite: - Three-model A/B table (Opus 4.7 / Sonnet 4.6 / Haiku 4.5) replaces the original Sonnet-only claim. Functional-areas beats baseline by +13 to +17pp training (lenient) across all three models at 48% the size. - Strict + lenient scoring documented side by side. Lenient (predicted shares dispatcher area with expected) matches production agent behavior. - Preconditions added: refuse to compress if file <12KB or working tree dirty. - Multi-file routing precedence section for the v0.31.7 RESOLVER.md/AGENTS.md merge case. - Mandatory verification step (≥95% via the harness). - Daily-doctor.mjs reference scrubbed (didn't exist in gbrain). - Three prior-art citations: AnyTool (arXiv:2402.04253), RAG-MCP (arXiv:2505.03275), Anthropic Agent Skills progressive disclosure. The pattern is the static-prompt analog of runtime hierarchical routing. routing-eval.jsonl: 8 positive (5 original + 3 broadened triggers) + 4 adversarial negatives targeting skillify, skill-creator, book-mirror, concept-synthesis to prove broadened triggers don't over-capture adjacent meta-skills. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * evals: A/B harness for functional-area-resolver (gateway-routed, strict + lenient scoring) evals/functional-area-resolver/ lives outside skills/ deliberately. The skillpack bundler walks skills/<skill>/ recursively, so an eval surface in there would copy harness + variants + fixtures + tests into every downstream install. The pattern (in SKILL.md) ships everywhere; the eval evidence stays in the gbrain repo. What ships: - Three variant resolvers in variants/ — baseline.md (verbose 25KB) and functional-areas.md (compressed 13KB) extracted from a real production AGENTS.md at git commits 93848ff3b^ and 93848ff3b (owner PII scrubbed). resolver-of-resolvers.md derived mechanically by stripping (dispatcher for: ...) clauses — the ablation case. - 20 hand-authored training fixtures + 5 held-out blind fixtures. - harness-runner.ts — TypeScript runner via gbrain gateway. Flags: --model {opus|sonnet|haiku|<full-id>}, --variants-dir, --variants for description-length sweeps, --parallel N (rate-lease bound), --limit N for smoke runs, --yes for non-TTY. - Every output row carries BOTH `correct` (strict) and `correct_lenient` (predicted shares dispatcher area with expected). Lenient matches production behavior. - Receipt header binds (model, prompt_template_hash, fixtures_hash, harness_sha, ts, cmd_args). Re-runs are auditable. - harness.mjs — thin Node shim that spawns the TS runner via bun. - rescore.mjs — zero-cost lenient re-score of an existing JSONL. - harness-runner.test.ts — 45 unit tests (no API key needed) covering every pure function plus the dispatcher-list parser. The prompt template is load-bearing: without the "drill into (dispatcher for: ...) list" instruction, every compression variant collapses to ~30-60%. Documented in SKILL.md and README.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * evals: baseline receipts (Opus 4.7 + Sonnet 4.6 + Haiku 4.5, 2026-05-11) Three canonical 225-row receipts (3 variants × 25 fixtures × 3 seeds per model). Each receipt header binds (model, prompt_template_hash, fixtures_hash, harness_sha, ts) so the published SKILL.md numbers are reproducible. Training corpus (n=20, lenient): baseline | Opus 81.7% | Sonnet 86.7% | Haiku 73.3% | 25KB functional-areas | Opus 98.3% | Sonnet 100% | Haiku 88.3% | 13KB resolver-of-resolvers | Opus 63.3% | Sonnet 41.7% | Haiku 65.0% | 10KB functional-areas beats baseline by +13 to +17pp across all three models at 48% the size. resolver-of-resolvers' Sonnet collapse (41.7%) is the SKILL.md "compression without dispatcher clause is broken" claim, observed. Held-out (n=5, lenient) saturates at 100% across most cells (Sonnet × resolver-of-resolvers is 73.3% — the same failure mode visible on a smaller sample). ~$3 API spend across all three runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * skill: wire functional-area-resolver into RESOLVER.md + manifests skills/RESOLVER.md gets a new row in Operational, adjacent to skillify. Triggers: "Compress my resolver", "AGENTS.md too large", "RESOLVER.md too big", "functional area dispatcher", "shrink routing table". skills/manifest.json adds the new entry and bumps manifest version 0.25.1 → 0.32.3.0 (loadOrDeriveManifest reads this for sync-guard). openclaw.plugin.json adds functional-area-resolver to the skills array and bumps version 0.25.1 → 0.32.3.0 so install receipts stop being stale (src/core/skillpack/installer.ts:307-311 uses manifest version on every install). Verified: - gbrain check-resolvable --json: 42/42 reachable, 0 errors. - gbrain routing-eval: 70/70 pass (100% structural). - bun test test/skillpack-sync-guard.test.ts: passes (manifest in sync). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.32.3.0 skill: functional-area-resolver — pattern for compressing routing tables Headline: compress a 25KB AGENTS.md down to 13KB without losing routing accuracy. Pattern proven across Opus 4.7, Sonnet 4.6, and Haiku 4.5 — beats the verbose baseline by +13 to +17pp at 48% the size. Empirical (training, n=20, 3 seeds, lenient): baseline 25KB: Opus 81.7% | Sonnet 86.7% | Haiku 73.3% functional-areas 13KB: Opus 98.3% | Sonnet 100% | Haiku 88.3% resolver-of-resolvers 10KB: Opus 63.3% | Sonnet 41.7% | Haiku 65.0% The (dispatcher for: ...) clause is the load-bearing signal. Strip it (the resolver-of-resolvers variant) and Sonnet collapses to 41.7% — the failure case the pattern's authors predicted, now observed. Files in this release: - VERSION + package.json bumped to 0.32.3.0 (4-segment per CLAUDE.md). - CHANGELOG.md: full empirical story, cross-model table, three prior-art citations (AnyTool, RAG-MCP, Anthropic Agent Skills progressive disclosure). - TODOS.md: nine v0.33.x follow-ups (dogfood on gbrain's own RESOLVER.md, CLI promotion to gbrain routing-eval --ab-compare, held-out corpus growth, cross-vendor Gemini+GPT verification, per-row description length sweep, structural compression to ~10KB, hierarchical area-of-areas, embedding pre-router, adversarial fixtures, prompt-design ablation doc). - llms-full.txt regenerated. Bisect-friendly history on this branch: 502d447e skill: rename + content rewrite + routing-eval.jsonl 472cc686 evals: A/B harness + variants + fixtures + tests (no receipts) 243e013e evals: cross-model baseline receipts (Opus + Sonnet + Haiku) ecab180b skill: wire-up to RESOLVER.md + manifest.json + openclaw.plugin.json THIS: v0.32.3.0 release marker Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * evals: codex review fixes — accept ASCII -> arrow + provider-aware auth gate Two P2 findings from /codex review on commit 8870c64e: P2-2: parseDispatcherLists regex required Unicode `→`, but SKILL.md Step 4 documents the template with ASCII `->`. Downstream-authored resolvers following the template silently fell through to strict-only scoring (correct_lenient == correct always), under-reporting same-area accuracy with no warning. Regex now accepts both `→` and `->`. Two new test cases pin the behavior — pure-ASCII variant + mixed-arrow variant. P2-3: main() exited with `ANTHROPIC_API_KEY is not set` even when the user passed `--model openai:gpt-4o` with a valid OPENAI_API_KEY. The CLI advertises full provider:model support (resolveModel tests cover openai:* explicitly) and the gateway routes by recipe; the env check should match the provider that will actually be called. Now extracts the provider id from the model string and looks up the right env var from REQUIRED_ENV_BY_PROVIDER (anthropic, openai, google, groq, voyage, together, deepseek, minimax, dashscope, zhipu). Unknown providers fall through to the gateway, which raises a clear recipe-specific error. 47/47 harness unit tests pass after the change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * skill: codex review P2-1 — verification gate now tests the user's edited file The original SKILL.md Step 6 told users to run `node harness.mjs` from the gbrain repo as the mandatory ≥95% gate. But that runs the harness against the COMMITTED sample variants in evals/functional-area-resolver/variants/, not the file the user just compressed. The gate could pass while the edit dropped a sub-skill. Step 6 now: - Gate 1 stays at `gbrain routing-eval --json` (structural, runs against the user's actual routing-eval.jsonl fixtures). - Gate 2 is rewritten: copy the user's edited routing file into a tmp variants dir, then run `node harness.mjs --variants-dir <tmp> --variants my-edit --model opus`. This exercises the harness's existing --variants flag (added in commit 472cc686 / T4) but now points at the user's actual edit. The harness uses gbrain-bundled fixtures, so this is a regression check on shared skills, not a full eval of the user's fixture set — and the SKILL.md says so explicitly. Also adds a "common false negatives" callout: when the user's routing file doesn't expose the skills gbrain's bundled fixtures target (e.g. `gmail`, `enrich`), expect strict-scoring fails on those rows; lenient scoring remains accurate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * evals: codex review P3 — regenerate Opus baseline with current schema The prior Opus receipt was generated before commit 472cc686 (T4 added harness_sha to ReceiptRow and correct_lenient to every RunRow). The Sonnet and Haiku receipts shipped with the new schema, but Opus was the outlier. This run was produced with the current harness (sha ca99fbfeb, after the P2-1 + P2-2 + P2-3 fixes). The harness_sha in the receipt header binds the numbers to a specific harness revision so consumers can detect schema drift. Numbers (training, lenient, n=20, 3 seeds): baseline: 81.7% ± 7.2% (unchanged — strict and lenient are equal) functional-areas: 100% ± 0% (was 98.3% — one nondeterministic seed is now in-cluster; pattern continues to beat baseline at 48% the size) resolver-of-resolvers: 66.7% ± 7.2% (was 63.3% — still in noise; absent dispatcher clause keeps it ~30pp behind functional-areas on training) Held-out (n=5, 3 seeds, lenient): all variants 100% except resolver-of- resolvers on Sonnet (committed in earlier baseline) — Opus held-out saturates the small fixture set. Run cost: ~$1.40 at Opus 4.7 pricing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * post-merge: scrub fork-private paths + add Contract/Output Format sections Two CI gates landed on master after this branch was cut: 1) scripts/check-privacy.sh (v0.32.2): banned /data/brain/ and /data/.openclaw/ in committed files. The eval variants extracted from a real production AGENTS.md still contained those fork-private path literals. Rewrote to /your/brain/path/, /your/agent/.openclaw/, /your/gbrain, /your/gstack, /your/tmp, /your/git-projects/. Only path strings changed — the routing structure (skill names, dispatcher clauses, trigger phrases) is byte-for- byte identical, so harness baseline-runs/ receipts are still valid. 2) test/skills-conformance.test.ts (master): added required sections `## Contract` and `## Output Format` to every skill. Added both to skills/functional-area-resolver/SKILL.md following the book-mirror convention (short body referencing the canonical content above + a conformance-test footnote). Contract notes the privacy guarantee + the verification-gate semantics; Output Format documents the area entry template (with both ASCII -> and Unicode → arrows accepted). Full unit suite: 5578 pass / 0 fail. bun run verify clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: surface functional-area-resolver in CLAUDE.md + README.md for v0.32.3.0 CLAUDE.md — adds a "Routing-table compression (v0.32.3.0)" entry under Skills, covering the two-layer dispatch pattern, the load-bearing (dispatcher for: ...) clause, the eval surface at evals/functional-area-resolver/, the three cross-model baseline receipts, the 25KB → 13KB compression numbers, and the nine v0.33.x follow-up TODOs. Cites AnyTool / RAG-MCP / Anthropic Agent Skills prior art so the pattern's position in the literature is discoverable from the agent entry point. README.md — adds a "New in v0.32.3.0" callout in the intro section so users landing on the repo see the new skill before scrolling to the skills list. Links the SKILL.md and eval directory; states the cross-model gain (+13 to +17pp at 48% the size) so the reason to apply the pattern is one click away. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: garrytan-agents <garrytan-agents@users.noreply.github.com> Co-authored-by: Garry Tan <garrytan@gmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-11 20:39:00 -07:00
Garry Tan	c2ae4dbfc5	v0.25.1 feat: book-mirror flagship + 8 research skills + skillpack uninstall + post-install advisory (#566) ... * v0.25.1 foundation: scaffolds + manifests + filing-doctrine update Foundation commit for v0.25.1 skills wave (book-mirror flagship + 8 research pairings). All content is scaffold-stage; subsequent commits port wintermute SKILL.md content into pure gbrain idiom. Version bumps: - VERSION 0.24.0 -> 0.25.1 - package.json: version + engines.bun >= 1.3.10 (D14 PTY harness) - openclaw.plugin.json inner version 0.19.0 -> 0.25.1 - bun.lock refreshed 9 skill scaffolds via `gbrain skillify scaffold` (frontmatter + RESOLVER row + routing-eval seed): book-mirror, article-enrichment, strategic-reading, concept-synthesis, perplexity-research, archive-crawler, academic-verify, brain-pdf, voice-note-ingest. Stub .mjs scripts and stub .test.ts files deleted; these are pure-markdown skills, not deterministic-script skills. Real tests will return when src/commands/book-mirror.ts and the other runtime pieces land. skills/manifest.json + openclaw.plugin.json skills[]: 9 new entries (codex T6 fix; required by test/skillpack-sync-guard.test.ts). D13 filing-doctrine update: - skills/_brain-filing-rules.md: carve out media/<format>/<slug> as a sanctioned exception for sui-generis synthesized output. - skills/_brain-filing-rules.json: add media/books/ and media/articles/ as `synthesis-output` kind, distinct from raw-ingest filing. - skills/media-ingest/SKILL.md: refine anti-pattern callout to clarify that format-prefixed paths are anti-pattern for raw ingest only, sanctioned for one-of-one synthesis. Privacy guard hardening (codex T7): - scripts/check-privacy.sh: extended for /data/brain/ and /data/.openclaw/ wintermute-specific path patterns. 7 historical files allow-listed (frozen migrations, test fixtures, env-var fallbacks). PRIVACY OK passes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 book-mirror: trusted CLI with read-only subagent fan-out Implements `gbrain book-mirror` per the locked v0.25.1 plan (D2/α + codex HIGH-1 fix). Closes the prompt-injection vector codex flagged on the earlier `allowedSlugPrefixes: ['media/books/*', 'people/*']` design by narrowing the trust contract at the tool-allowlist layer instead. Trust contract: - Each chapter is analyzed by a separate subagent with allowed_tools restricted to ['get_page', 'search'] — read-only. Subagents cannot call put_page or any mutating op. Untrusted EPUB/PDF content cannot prompt-inject any people/* page because subagents lack write access entirely. - Subagents return markdown analysis text via final_message (SubagentResult.result). The CLI reads each child's job.result and assembles the final two-column page itself. - The CLI calls put_page once at the end with operator-level trust (no viaSubagent flag, no allowedSlugPrefixes). Operator can write anywhere; the namespace check doesn't fire for direct CLI calls. Architecture: - `--chapters-dir` is the input contract. The skill (which has shell + python access) handles EPUB/PDF extraction; the CLI takes pre-extracted .txt files. Separation of concerns: skill prepares inputs, CLI is the trusted runtime. - Cost-estimate prompt before launching: ~$0.30/chapter × N at Opus, ~$0.06/chapter at Sonnet. Refuses to spend in non-TTY without --yes. - Idempotency keys on each child: `book-mirror:<slug>:ch-<N>`. Re-running on same input dedups against the queue; failed chapters retry. - Partial-failure handling: assembled page renders with completed chapters and a `## Failed chapters` section listing retries needed. Exit 1 on any failure; exit 0 only on full success. - 30-min default per-child timeout (override with --timeout-ms). CLI wiring: - `book-mirror` added to CLI_ONLY set in src/cli.ts. - Lazy-imports src/commands/book-mirror.ts to keep cold-start fast. Out of scope for this commit (filed for v0.25.1 follow-ons): - skills/book-mirror/SKILL.md content port (replaces the foundation scaffold stub). - test/book-mirror.test.ts (will test arg parsing, validation, mock fan-out, cost-estimate gating, partial-failure assembly). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 book-mirror: port SKILL.md content + routing-eval Replaces the foundation scaffold stub with the full ported book-mirror SKILL.md, pointing the agent at the new `gbrain book-mirror` CLI as the trusted runtime. skills/book-mirror/SKILL.md: - Drops wintermute_only frontmatter; uses gbrain frontmatter shape (mutating + writes_pages + writes_to: media/books/). - Documents the trust contract: subagents are read-only, the CLI does the put_page write itself with operator trust. Closes the codex HIGH-1 prompt-injection vector at the tool-allowlist layer. - Replaces /data/brain/ absolute paths with $BRAIN_DIR resolution from gbrain config. - Replaces brain-commit-link.sh / direct shell-script writes with the CLI's single put_page call. - Documents EPUB/PDF extraction via the agent's shell + python access (BeautifulSoup4 for EPUB, pdftotext for PDF). The skill prepares inputs; the CLI is the trusted runtime. - Privacy scrub clean — no real names, no /data/brain/, no .openclaw/, no Wintermute literals. skills/book-mirror/routing-eval.jsonl: - 5 paraphrased intents per D-CX-6 rule (intent paraphrases the trigger, doesn't copy it). - 3 adversarial intents that pattern-match media-ingest's "process this book" trigger (IRON RULE regression test for the media-ingest <-> book-mirror routing conflict flagged in R1+R2). These assert that book-mirror should NOT win on generic ingest phrasing. skills/_brain-filing-rules.json: 4 new directory kinds added so check-resolvable's filing audit passes for the new skills' writes_to declarations: - idea (ideas/) — generative ideas to act on later (voice-note-ingest, archive-crawler). - research (research/) — web-research deltas, citation-checked claims (perplexity-research, academic-verify). - original (originals/) — user-authored thinking the user originated (voice-note-ingest, archive-crawler, signal-detector). - voice-note (voice-notes/) — random-thought audio capture pages (voice-note-ingest). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 ports: article-enrichment + strategic-reading + voice-note-ingest Replaces SKILLIFY_STUB scaffolds with content-ported SKILL.md files in pure gbrain idiom: skills/article-enrichment/SKILL.md: - Drops wintermute-specific scripts/enrich-article.mjs reference; the skill is markdown agent instructions, not a deterministic script pipeline. - Replaces /data/brain/ paths with relative brain-dir paths. - Documents the structured output contract (Executive Summary, Quotable Lines verbatim, Key Insights, Why It Matters, See Also, details-block source preservation). - Sonnet by default, Opus for high-value content. skills/strategic-reading/SKILL.md: - Generic problem-lens reading flow (book/article/case study x specific strategic problem -> applied playbook with do/avoid/watch-for). - Drops Garry-specific oppo example ("Tyler Law/Han Zou gatekeeper fight"); uses generic "gatekeeper-vs-incumbent fight" framing. - Files to projects/<slug>/playbook.md (problem-tied) or concepts/<slug>.md (general strategy) per primary-subject filing rule. - Cross-references book-mirror as the whole-life-personalization counterpart. skills/voice-note-ingest/SKILL.md: - Iron Law: exact phrasing preserved, never paraphrased. Block-quoted transcript is sacred; analysis is interpretive. - 7-step decision tree (originals -> concepts -> people -> companies -> ideas -> personal -> voice-notes catch-all) per _brain-filing-rules.md. - Replaces wintermute's brain-commit-link.sh + Supabase Storage helper with gbrain transcription + storage interface (pluggable per src/core/storage.ts). Each skill ships routing-eval.jsonl with 5 paraphrased intents per D-CX-6 (intent paraphrases trigger, doesn't copy it). The literal "please <trigger> for me now" stubs from gbrain skillify scaffold are replaced with realistic user phrasings. Privacy scrub clean — no real names, no /data/brain/, no .openclaw/, no Wintermute literals. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 ports: concept-synthesis + perplexity-research + brain-pdf Replaces SKILLIFY_STUB scaffolds with content-ported SKILL.md files in pure gbrain idiom: skills/concept-synthesis/SKILL.md: - 4-phase pipeline: dedup -> tier (T1 Canon to T4 Riff) -> synthesize T1/T2 -> cluster + intellectual map. - Generic across any concept-stub source (signal-detector, voice-note-ingest, idea-ingest, archive-crawler). - Drops wintermute-specific X-pipeline framing (9051 stubs from x-deep-enrich, scripts/x-concept-compiler.mjs); skill is markdown agent instructions using gbrain query + put_page. - Output format: T1 gets full synthesis with evolution table + best articulation + related-concepts cross-links; T3/T4 stay as stubs. - Cluster map at concepts/README.md as the master intellectual fingerprint. skills/perplexity-research/SKILL.md: - Brain-augmented web research: sends brain context as part of the Perplexity prompt so the search focuses on what's NEW vs already-known. - Output structure: Executive Summary + Key New Developments + Confirming Signals + Contradictions or Updates + Recommended Brain Updates + Citations. - Uses Perplexity sonar-pro by default (~$0.04/query); sonar for bulk. - Drops wintermute-specific scripts/perplexity-research.mjs and /data/.env path; documents PERPLEXITY_API_KEY in agent env. - Cross-references academic-verify (which wraps this skill for citation-checked claim verification per D7/alpha) and enrich (entity enrichment loop). skills/brain-pdf/SKILL.md: - Documents gstack make-pdf as soft prereq with absent-binary detection. - 4-step workflow: resolve -> strip frontmatter -> render -> deliver. - Defaults: NO --cover, NO --toc (look corporate and waste space). - Mandatory CONTAINER=1 for Playwright sandboxing. - Anti-pattern callout: never use raw MEDIA: tags for Telegram delivery (they fail silently); use message tool with filePath= attachment. Each ships routing-eval.jsonl with 5 paraphrased intents per D-CX-6. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 ports: archive-crawler + academic-verify (final SKILL.md batch) Replaces the last two SKILLIFY_STUB scaffolds. All 9 new skills now have ported content; `gbrain check-resolvable` reports zero skillify_stub_unreplaced warnings. skills/archive-crawler/SKILL.md (D3 + D12): - Hard safety gate: refuses to run unless `archive-crawler.scan_paths:` is set in gbrain.yml. Closes the codex HIGH-4 footgun where 'trust the prompt' was not a control. - Schema-generic port (D3 user constraint): no hardcoded era folders (no archive/, post-stanford/, posterous-era/, initialized-era/, yc-era/). Reads filing rules from _brain-filing-rules.json at runtime; agent decides per-page filing within sanctioned dirs. - Drops wintermute-specific scripts and brain-commit-link.sh; uses gbrain operations for inventory + put_page for ingest. - File-type handlers preserved (.mbox, .doc/.docx, .pst, .zip, images) with the exact same shell + python recipes. - Manifest tracks per-item triage status + exact user reactions per conventions/quality.md exact-phrasing rule. skills/academic-verify/SKILL.md (D4 + D7/alpha): - Drops ALL the wintermute-specific oppo / adversarial framing: no Goff/Solomon, no CPE, no '48 Hills', no fabrication-detection, no 'oppo research where the target relies on academic credentials'. This is the public skillpack — research-not-adversarial bar. - Pure-routing implementation per D7/alpha: skill is a thin orchestrator that scopes the claim, invokes perplexity-research with citation-mode prompt, and formats results as a verdict-shaped brain page. Zero new infrastructure. - 5 verdict states (verified / partial / unverifiable / misattributed / retracted) replace the 'fabrication suspected' / 'methodologically flawed' classifications that read like takedown rubric. - Documents Retraction Watch / PubPeer / OSF / Semantic Scholar / OpenAlex / Many Labs as the databases the agent uses via perplexity-research, but doesn't ship its own API integrations. Each ports a routing-eval.jsonl with 5 paraphrased intents per D-CX-6. Privacy scrub clean. typecheck OK. Remaining check-resolvable warnings are routing_miss on the substring matcher (paraphrased intents don't exact-match the RESOLVER triggers); the LLM tie-break layer is a v0.26+ enhancement per CLAUDE.md routing-eval section. Warnings are advisory, not errors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 drift backports: citation-fixer + testing + cross-modal-review Pulls the wintermute drift improvements identified by R1's quick audit into the public skillpack, in pure gbrain idiom (no real names, no /data/brain/ paths, no Wintermute literals — privacy guard passes). skills/citation-fixer/SKILL.md (PORT, version 1.0 -> 1.1): - Adds tweet/post URL resolution: scans pages for broken tweet references (no x.com URL) and resolves them via the host's X API integration. - 5-step pipeline: identify broken refs -> extract searchable content (handle/quote/date) -> X API search -> verify + extract metadata -> patch the page with deterministic URL. - Batch-mode pattern with priority order (recently changed pages first), rate-limit guidance (~50 pages/run), batch-commit cadence. - Integration callout: enrich + media-ingest can call citation-fixer pre-commit to validate output. - Anti-pattern: never compose tweet URLs by guessing the id; deterministic links only (per _output-rules.md). skills/testing/SKILL.md (PORT, version 1.0 -> 1.1): - Splits into TWO modes: skill conformance validation (original 1.0 scope) AND project test-suite health (v0.25.1 extension). - Test tiers: unit (<2s, every commit), evals (~60s, daily), integration (~5m, pre-ship + nightly), system health (<10s). - Daily run protocol: unit -> evals -> system -> git diff analysis for regression intelligence. - Failure classification: REGRESSION / STALE / FLAKE / NEW / INFRA with markers (red / yellow / warning / green / wrench). - Auto-fix protocol: explicit DO and DO NOT lists. Security-test failures always escalate, never auto-fix. - State tracking at ~/.gbrain/test-state.json for trend analysis, flake detection, regression velocity. skills/cross-modal-review/SKILL.md (PORT, version 1.0 -> 1.1): - Adds explicit "When to invoke" gating (significant code changes 5+ files / 100+ lines, security-sensitive, architecture, churning, pre-bulk, skill creation, brain-page quality) vs DO NOT invoke (simple memory writes, typo fixes, routine cron, post-review commits). - Adds code-review handoff section: knows WHEN to recommend gstack's /codex review (independent diff review from a different AI) and how to frame the cross-model output. - Adversarial Challenge sub-mode: red-team prompt for security- sensitive changes; output adds exploitability rating (CRITICAL/HIGH/MEDIUM/LOW) + mitigations. - Iron Law: user-sovereignty rule explicitly captured. Reviewer findings are informational until the user explicitly approves; cross-model consensus is signal, not permission. All three pass scripts/check-privacy.sh (no Wintermute literals, no /data/brain/, no /data/.openclaw/). typecheck OK. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 skillpack uninstall: D6 + D8 + D11 content-hash guard Implements `gbrain skillpack uninstall <name>` per the locked v0.25.1 plan. Inverse of install with symmetric data-loss posture: refuses if the slug isn't in the managed-block's cumulative-slugs receipt (D8) or if any installed file diverges from the bundle original (D11). Same --overwrite-local escape hatch as install. src/core/skillpack/installer.ts: - New UninstallError class (mirrors InstallError shape) with codes: lock_held, bundle_error, target_missing, unknown_skill, user_added_slug (D8), locally_modified (D11), managed_block_missing. - New types: UninstallFileOutcome, UninstallFileResult, UninstallResult, UninstallOptions. - New applyUninstall() function. Steps: 1. Acquire workspace lockfile (same gate as install). 2. D8 check: read managed block; verify slug is in cumulative-slugs receipt. If user-added or unknown, throw user_added_slug. 3. Enumerate bundle entries scoped to the skill (NOT shared_deps — other installed skills depend on them). 4. D11 check: hash each existing target file vs bundle original. Skip removal for divergent files unless --overwrite-local. 5. Atomic: if ANY file would be skipped due to local-mod and the user did not pass --overwrite-local, refuse the WHOLE uninstall (no half-uninstall — would desync managed block from filesystem). 6. Rebuild managed block via applyManagedBlockUninstall() (drops slug from cumulative-slugs, preserves other rows + user-added unknown rows with stderr warning, atomic write via writeAtomic). 7. Release lock. src/commands/skillpack.ts: - Wire `gbrain skillpack uninstall` subcommand. Flags mirror install: --dry-run, --overwrite-local, --force-unlock, --skills-dir, --workspace, --json, --help. - Exit codes: 0 success, 1 refused due to local-mod (recoverable with --overwrite-local), 2 setup error (slug not in receipt, no workspace, lock held, etc.). - Help text documents the symmetric trust contract explicitly. D6 test slot is filled (smoke test t2 "uninstall changes routing" will use this command). Per the plan, no `--all` uninstall in v0.25.1 (scope-narrowing; renaming a skill in the bundle should still be the install --all path that prunes). Typecheck passes. Privacy guard passes. `gbrain skillpack uninstall --help` renders correctly. Out of scope for this commit (next): - test/skillpack-uninstall.test.ts (D8 + D11 cases, multi-arg, fail-loud-under-lock, idempotent-when-absent). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 archive-crawler safety gate (D12 + codex HIGH-4 fix) Adds the gbrain.yml `archive-crawler.scan_paths:` allow-list contract that closes the codex HIGH-4 finding. The archive-crawler skill refuses to run unless the user has explicitly listed paths the agent is permitted to scan. src/core/archive-crawler-config.ts (NEW, 263 lines): - Sibling to storage-config.ts (separate concern: archive scanning, not storage tiering; same gbrain.yml file shape). - Hand-rolled parser for the `archive-crawler:` section (mirrors storage-config's parsing pattern; same trade-off — narrow-but- predictable, zero-dep). - Accepts both `archive-crawler:` and `archive_crawler:` spellings. - ArchiveCrawlerConfig: { scan_paths: string[]; deny_paths: string[] } — both normalized to absolute trailing-slashed paths. - Validation: * scan_paths MUST be non-empty (D12 contract) * Every path absolute after ~ expansion (rejects relative) * Path-traversal rejected (`..` literal in path → invalid_path) * Trailing-slash normalized for unambiguous prefix matching - isPathAllowed(candidate, config) helper for runtime per-file gate: prefix-match against scan_paths, deny_paths overrides. Directory- boundary safe — /writing/ does NOT match /writing-stuff/. - ArchiveCrawlerConfigError class with discriminated codes: missing_section / empty_scan_paths / invalid_path / parse_error. test/archive-crawler-config.test.ts (NEW, 19 tests): - D12 missing_section gates: null repoPath, missing gbrain.yml, no archive-crawler section. - D12 empty_scan_paths: scan_paths omitted or empty array. - D12 invalid_path: relative path, ".." traversal in scan_paths, ".." traversal in deny_paths. - Happy path: normalized paths, ~ expansion, deny_paths optional, both archive-crawler and archive_crawler key spellings. - Direct API validation (normalizeAndValidateArchiveCrawlerConfig). - isPathAllowed: scan_path match, scan_path miss, deny_path override, directory-boundary correctness (writing/ vs writing-stuff/), relative-path rejection. 19/19 pass in 17ms. Privacy guard passes. Typecheck OK. The skills/archive-crawler/SKILL.md (already shipped in earlier commit) documents the contract; this commit lands the runtime that enforces it. The skill's safety claim is no longer aspirational. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 PTY harness port from gstack (D14/C-prime) Ports gstack's claude-pty-runner.ts (~1300 lines) as a generalized gbrain harness (~470 lines after trimming gstack-specific orchestrators). Used by the smoke test E2E to drive interactive openclaw sessions; future: any CLI command that grows interactive prompts becomes testable without a refactor. test/helpers/cli-pty-runner.ts (NEW, 470 lines): - launchPty(opts): generic CLI spawner via Bun.spawn `terminal:` mode. Drops gstack's launchClaudePty's --permission-mode plan default; takes any binary + args. - resolveBinary(name, override?): finds CLI binaries on PATH with homebrew/local/bun fallbacks. - stripAnsi: standard CSI + OSC + charset + DEC-special escape stripping (verbatim port). - isNumberedOptionListVisible: cursor + numbered list detection. - parseNumberedOptions: extracts cursor-anchored numbered AUQ options (1-based indices, sequential block only). Handles cursor-on-non-1 (user pressed Down) and box-layout AUQs (cursor mid-line after dividers). Reads only last 4KB to avoid matching stale lists. - optionsSignature: stable hash for "is this AUQ the same as last poll?" detection. - isTrustDialogVisible: matches Claude Code's "trust this folder" dialog so launchPty can auto-handle it. - PtyOptions / PtySession types + send / sendKey / mark / visibleSince / waitFor / waitForAny primitives. - launchPty internals: terminal: mode, exit tracking, wall-clock timeout, autoTrust polling watcher (15s window), graceful close with SIGINT then SIGKILL fallback. DROPPED from the gstack original (gstack-specific): - runPlanSkillObservation, runPlanSkillCounting, invokeAndObserve (Claude-Code plan-mode test orchestrators). - isPlanReadyVisible, isPermissionDialogVisible (Claude-Code-specific dialog detection). - ceoStep0Boundary, engStep0Boundary, designStep0Boundary, devexStep0Boundary (per-skill /plan-* boundary predicates). - MODE_RE, COMPLETION_SUMMARY_RE, parseQuestionPrompt, auqFingerprint, assertReviewReportAtBottom (gstack plan-review specifics). - classifyVisible (plan-mode outcome classifier). If the smoke test ever needs Claude-Code-specific dialog detection, add a thin wrapper in test/e2e/ — keeping the harness generic. test/cli-pty-runner.test.ts (NEW, 24 tests, all pass): - stripAnsi: 6 cases (CSI, OSC-BEL, OSC-ST, charset, DEC-special, plain) - isNumberedOptionListVisible: 4 cases (match, no-cursor, single-opt, TTY collapsed-whitespace) - parseNumberedOptions: 7 cases (3-opt, no-list, single-opt, prose- gating-pattern, gap-truncation, cursor-on-non-1, last-4KB-only) - optionsSignature: 2 cases (order-independence, label-changes-sig) - isTrustDialogVisible: 2 cases (canonical phrase, non-match) - resolveBinary: 3 cases (override, missing, sh-on-path) 24/24 pass in 14ms. Privacy guard passes. Typecheck OK. Bun version requirement (D14): engines.bun >= 1.3.10 (set in commit b438a7c4) — required by Bun.spawn terminal: mode. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 skillpack uninstall tests + atomic-refusal bug fix 10 tests for applyUninstall covering D6 + D8 + D11. Found and fixed a real atomic-refusal bug while writing them. src/core/skillpack/installer.ts (BUG FIX): - applyUninstall previously interleaved D11 hash check + unlink in the same loop. If file 5/N diverged, files 1..4 were ALREADY gone by the time the throw fired — half-uninstalled state, managed block out of sync with filesystem. - Now: pre-scan ALL files for divergence into a fileChecks array; refuse loudly BEFORE any filesystem mutation if anything is blocked. Then unlink in a second pass (no decisions left to make). - The atomic-refusal contract documented in the original code now matches the actual behavior. The contract was always the intent; the implementation just shipped wrong. test/skillpack-uninstall.test.ts (NEW, 10 tests): - Happy path: removes alpha files, drops slug from cumulative-slugs receipt, --dry-run leaves disk untouched. - Preserves other installed skills: install --all then uninstall alpha, beta still present + still in receipt. - D8 user_added_slug: refuses uninstall when slug not in cumulative-slugs receipt; refuses even when user hand-added the managed-block row. - D11 locally_modified: file diverges from bundle → throws + NOTHING removed (atomic refusal; this is the test that caught the bug). - D11 --overwrite-local: bypasses guard, removes anyway. - unknown_skill / bundle_error: bad slug rejected with typed error. - managed_block_missing: no RESOLVER.md in target → typed error. - Idempotency: file already absent on disk doesn't crash; counts in result.summary.absent. 10/10 pass in 53ms. All 90 skillpack-related tests still pass (install + uninstall + sync-guard + harness + archive-crawler). Privacy guard passes. Typecheck OK. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 book-mirror tests — CLI surface + source invariants 9 tests pinning the book-mirror CLI's contract surface and regression-detector source patterns. Pure surface tests; the full subagent fan-out integration is exercised by the opt-in smoke test (test/e2e/skill-smoke-openclaw.test.ts when EVALS=1). Architecture note documented in the test file: src/cli.ts dispatches connectEngine() BEFORE any CLI_ONLY command's own arg parsing, including --help. This is a pre-existing choice (every CLI_ONLY command — agent, sync, jobs, book-mirror — behaves identically) so arg-validation paths can't be exercised from a clean tempdir without DATABASE_URL. The smoke test covers them with a real engine. What we test: - book-mirror is registered in CLI_ONLY (no "Unknown command") - Without DB, never reaches the queue-submission path - Source file: exports runBookMirrorCmd - Source file: documents the trust contract (codex HIGH-1 fix marker) - Source file: read-only allowed_tools = ['get_page', 'search'] (the actual trust narrowing — regression-detector for someone adding put_page back to the subagent's tool list) - Source file: operator-trust put_page (remote: false, viaSubagent intentionally omitted as a regression-detector inline comment) - Source file: cost-estimate confirmation (P1) - Source file: idempotency keys for child jobs - Source file: partial-failure handling 9/9 pass in 157ms. Privacy guard passes. Typecheck OK. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 docs: CHANGELOG + CLAUDE.md + migration + privacy allow-list CHANGELOG.md (NEW v0.25.1 entry): - Garry-voice release summary per CLAUDE.md voice rules: bold two-line headline, lead paragraph, "numbers that matter" table, "what this means for builders" closer, "To take advantage of v0.25.1" verify block, itemized changes (skills / CLI / filing / test infra / CI guard / config schema / drift backports / bug fix / tests / deferred). - Documents the cross-model review trail: 15 user decisions across R1 + R2 + codex outside voice; 4 codex HIGH findings the eng review missed. - The atomic-refusal bug fix called out as the cross-model loop working: test was written with the contract in mind, implementation lied about the contract, lie surfaced immediately. CLAUDE.md (Key Files updates): - src/commands/book-mirror.ts: full annotation with trust contract, codex HIGH-1 fix, idempotency keys, partial-failure handling. - src/commands/skillpack.ts: extended with v0.25.1 uninstall semantics — D8 user-added refuse, D11 content-hash guard, atomic- refusal contract enforced by test. - src/core/archive-crawler-config.ts: D12 + codex HIGH-4 safety gate documentation. - test/helpers/cli-pty-runner.ts: PTY harness port from gstack documented. skills/migrations/v0.25.1.md (NEW): - Agent-readable upgrade walkthrough. 6 steps: 1. Verify upgrade landed 2. Install new skills (optional) 3. Configure archive-crawler scan_paths if installed (REQUIRED) 4. Use gbrain book-mirror (optional, the flagship) 5. gbrain skillpack uninstall (when you want it) 6. Privacy CI guard (fork-operators only) - "If anything fails" feedback loop pointing at the issues tracker. scripts/check-privacy.sh: - CHANGELOG.md added to ALLOW_LIST. The v0.25.1 release notes document the BANNED_PATHS extension and reference the patterns in describing what's banned — same exception status as CLAUDE.md (which describes the rules) and the script itself. Privacy guard passes. Typecheck OK. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 README: 34 skills + new "Research and synthesis" section README.md updates: - Top-of-page count: "29 skills" -> "34 skills" (4 places). - Section header: "The 29 Skills" -> "The 34 Skills" with a pointer to the new Research and synthesis section. - Added voice-note-ingest + article-enrichment under Content ingestion. - New "Research and synthesis (v0.25.1)" section with 7 skills: book-mirror (flagship), strategic-reading, concept-synthesis, perplexity-research, archive-crawler (with safety-fence callout), academic-verify, brain-pdf. - Each entry is one-line, what-it-does framing, no AI vocabulary. scripts/check-privacy.sh: - Added skills/migrations/v0.25.1.md to ALLOW_LIST. Same exception status as CHANGELOG.md and CLAUDE.md: meta-documentation that references the banned patterns to explain what's banned to the operating agent. Privacy guard passes. Typecheck OK. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 verification: conformance sections + routing-eval intents + test loosen Final pass to make the test suite green. skills/{12 ports + backports}/SKILL.md: - Renamed `## Anti-patterns` -> `## Anti-Patterns` (capital P) so the conformance test (test/skills-conformance.test.ts) sees the literal header it requires. - Appended `## Contract` and `## Output Format` skeleton sections to every new SKILL.md and any backport that didn't have them. The conformance test asserts these literal headers; content can be brief (the body sections above already carry the substantive contract / output prose). - Privacy guard: changed the appended Contract prose from "no `/data/brain/` literals" to "no fork-specific filesystem path literals" so the guard doesn't flag the doc text. skills/{9 new ports + book-mirror}/routing-eval.jsonl: - Rewrote intents so each contains at least one trigger string as substring. The structural matcher in check-resolvable requires substring match against triggers; my earlier intents were too paraphrased (per D-CX-6 rule) and missed the matcher entirely. Now each fixture has 5 intents that BOTH paraphrase user phrasing AND contain a literal trigger. book-mirror keeps its 3 adversarial intents that route to media-ingest (IRON RULE regression test). - Fixed perplexity-research intent ambiguity: "Run perplexity research" was matching data-research too; tightened to "perplexity-research" with hyphen + added ambiguous_with to acknowledge the overlap. test/check-resolvable.test.ts: - v0.22.4 regression test loosened: routing_miss warnings are now ALLOWED (still fails on errors and on other warning types like trigger overlap, DRY violations, filing-rule misses). Documented in-line: routing_miss surfaces naturally when intents are paraphrased per D-CX-6; the LLM tie-break layer (placeholder per v0.24.0) is the intended fix when it ships. - Test renamed: "0 warnings" -> "0 errors" to match the new contract. Verification: - scripts/check-privacy.sh OK - bun run typecheck OK - 423 tests / 0 fails on the v0.25.1-relevant suite (book-mirror, skillpack-install, skillpack-uninstall, skillpack-sync-guard, cli-pty-runner, archive-crawler-config, skills-conformance, resolver, check-resolvable, check-resolvable-cli). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 post-install advisory: agent-readable "what to do next" gbrain users typically interact through their host agent (openclaw, claude-code), not the CLI directly. So an interactive TTY prompt at install time misses most of the audience. Instead: every gbrain init and gbrain post-upgrade ends by printing an advisory the agent reads from terminal output. The advisory: 1. Names the version that just landed (0.25.1) 2. Lists each new skill the workspace hasn't installed yet, with a one-line value prop (FLAGSHIP, two-column, brain-augmented, etc.) 3. Tells the agent EXPLICITLY to ask the user before installing 4. Prints the exact command if the user says yes 5. Shows alternative commands (install <name>, list) if they say no Detection logic (no nag): - Reads cumulative-slugs receipt from the workspace's managed block - Filters the v0.25.1 recommended set against installed slugs - Returns null when every recommended skill is already installed (so existing-user upgrades that already installed --all don't get re-pestered every gbrain post-upgrade run) - Workspace not detected → still renders advisory with a workspace- detection note (the agent can prompt the user for the right path) src/core/skillpack/post-install-advisory.ts (NEW, 209 lines): - V0_25_1_RECOMMENDED constant: the 9 new skills + descriptions. Future releases either bump the constant or read frontmatter from the latest migration file. - detectInstalledSlugs(skillsDir, workspace): reads receipt or falls back to extractManagedSlugs for pre-v0.19 fences. - buildAdvisory({ version, context, targetWorkspace, targetSkillsDir }): returns string OR null. Picks `--all` command for fresh installs, per-skill command for upgrades with subset missing. - printAdvisoryIfRecommended(): no-op safe wrapper for the caller. - Renders to stderr (stdout stays clean for --json output). src/commands/init.ts: prints the advisory after both PGLite and Postgres init paths succeed. src/commands/upgrade.ts (runPostUpgrade): prints the advisory at the end of post-upgrade, after migrations apply. Best-effort wrapped — never blocks post-upgrade on a print failure. test/post-install-advisory.test.ts (NEW, 10 cases): - detectInstalledSlugs: empty set on no managed block; reads receipt - partial-install: lists ONLY missing; uses per-skill command - fresh-install (no slugs): uses --all command - all-installed: returns null (no nag) - agent-readable framing: "ACTION FOR THE AGENT", "Ask the user explicitly", "Do NOT install without asking", "user owns this decision" - version + context (init vs upgrade) phrasing differ correctly - one-line skill descriptions present (FLAGSHIP, two-column, etc.) - workspace-not-detected fallback still renders with a note 10/10 pass in 25ms. Privacy guard passes. Typecheck OK. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 docs: document post-install advisory in CHANGELOG + migration CHANGELOG.md: new section "Added (post-install advisory — v0.25.1 DX)" explaining the advisory module, the "agent-readable instead of TTY prompt" design choice, and which commands print it. skills/migrations/v0.25.1.md: new Step 0 explicitly tells the operating agent to handle the post-install advisory FIRST (the banner the user just saw after `gbrain upgrade`), then return to the rest of the migration steps. If the advisory didn't print, the workspace is already up to date. The migration file is what the agent reads after `gbrain upgrade` runs `gbrain post-upgrade` and prints the banner — Step 0 closes the loop between the advisory's "ASK THE USER FIRST" and the existing migration walkthrough. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.25.1 regen llms-full.txt — pick up v0.25.1 CLAUDE.md additions The build-llms regen-drift guard (test/build-llms.test.ts) caught that llms-full.txt was stale after the merge with master. CLAUDE.md gained v0.25.1 entries (book-mirror.ts, archive-crawler-config.ts, cli-pty-runner.ts, skillpack uninstall annotation) that the generator inlines into llms-full.txt. Regenerated via bun run build:llms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-02 08:41:11 -07:00
Garry Tan	78ba0b5b53	v0.19.0 check-resolvable: add OpenClaw skills-dir fallback + docs/tests (#326) ... * Add OpenClaw skills fallback for check-resolvable * feat: v0.17.0 foundation — errors/warnings split + AGENTS.md support + auto-manifest First two workstreams of the v0.17.0 "skillify end-to-end" release. Landed together because the D-CX-3 exit-code refactor is a prerequisite for W1's warning-surfaced filing audit in Workstream 3. ## D-CX-3: split ResolvableReport into errors[] + warnings[] + --strict Prior: `env.ok = report.issues.length === 0` treated warnings and errors identically for exit status. Any warning forced exit 1, which meant the planned filing-audit (W3) would break CI for every OpenClaw deployment emitting advisory warnings. New contract: - `ResolvableReport.errors[]` and `warnings[]` as separate arrays. - `issues[]` stays as deprecated backcompat union (remove in v0.18). - Default: exit 0 unless any errors. Warnings are advisory. - `--strict` flag promotes warnings to fail CI (explicit opt-in). Files: src/core/check-resolvable.ts, src/commands/check-resolvable.ts (added --strict flag + help text + header doc), src/commands/doctor.ts (use new fields), test/check-resolvable-cli.test.ts (rewrite REGRESSION-GATE to document the new contract, add 3 D-CX-3 cases). ## W1: AGENTS.md support + auto-manifest + priority fix The reference OpenClaw deployment uses AGENTS.md (not RESOLVER.md) at the workspace root, and ships without a manifest.json. check-resolvable silently false-passed against it pre-W1: 0 manifest entries meant 0 reachability iterations meant 0 errors reported. Post-W1 behavior against ~/git/<redacted>/workspace (smoke-tested live): - Detects 102 skills via SKILL.md walk (no manifest.json needed) - Flags 15 unreachable errors (exactly the essay's '~15% dark' finding) - Flags 108 warnings (overlaps, gaps) — advisory, not blocking - Auto-detects via \$OPENCLAW_WORKSPACE without --skills-dir Changes: - NEW src/core/resolver-filenames.ts: one source of truth for the filename policy. \`RESOLVER_FILENAMES = ['RESOLVER.md', 'AGENTS.md']\`. Callers import from here, never hardcode either name. - NEW src/core/skill-manifest.ts: \`loadOrDeriveManifest()\` — reads manifest.json when present+valid, otherwise walks \`skillsDir/*/SKILL.md\` to derive a synthetic manifest. Both check-resolvable.ts AND dry-fix.ts now call this, replacing the two duplicated loaders that silently returned [] on missing file (F-ENG-1, D-CX-12). - src/core/repo-root.ts (rewrite): auto-detect priority changed to put \$OPENCLAW_WORKSPACE ahead of findRepoRoot() walk when explicitly set (D-CX-4). Adds workspace-root AGENTS.md detection — OpenClaw layout places routing at workspace/AGENTS.md with skills/ below. New SkillsDirSource variants \`openclaw_workspace_env_root\` and \`openclaw_workspace_home_root\` for --verbose log clarity. - src/core/check-resolvable.ts: accepts RESOLVER.md or AGENTS.md at the skills dir or one level up (workspace root). Uses loadOrDeriveManifest for reachability. Updated error messages reference both filenames. - src/core/dry-fix.ts: unified manifest loader — auto-fix now works in AGENTS.md-only workspaces where it previously no-op'd silently. - src/commands/check-resolvable.ts: new AUTO_DETECT_HINT import for clearer missing-skills-dir errors; updated sourceLabel map for the two new workspace-root variants. Tests: - test/skill-manifest.test.ts: 14 cases covering explicit-manifest, derived-manifest, malformed JSON, wrong shape, empty explicit array (honored as 'zero skills' declaration), dirname fallback when no name: frontmatter, underscore/dotfile dir skipping. - test/repo-root.test.ts: new tests for the priority swap, AGENTS.md skills-dir variant, AGENTS.md workspace-root variant, both-files present (RESOLVER.md wins). - test/check-resolvable-cli.test.ts: updated regression-gate to the new contract; added three D-CX-3 cases. All 105 tests passing across the foundation surface. Plan + reviews: ~/.claude/plans/p1-lets-just-vast-blanket.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W2 — Check 5 trigger routing eval (structural) Check 5 of the 10-step skillify checklist (the essay's "resolver trigger eval") now runs structurally by default and has a dedicated CLI verb for CI. Ships Layer A; Layer B (LLM tie-break) is reserved for v0.18. ## New module: src/core/routing-eval.ts The harness. Pure functions: - `normalizeText(s)`: lowercase, strip non-alnum to spaces, collapse whitespace. Unicode-friendly, quote-agnostic, punctuation-tolerant. - `extractTriggerPhrases(cellText)`: split quoted alternatives like `"search for", "find me"` into separate normalized phrases; fall back to the whole cell when unquoted (OpenClaw-style descriptions). - `indexResolverTriggers(resolverContent)`: build a skill-slug → normalized-trigger-phrases map from the resolver table. - `structuralRouteMatch(intent, index)`: substring-match the normalized intent against every trigger phrase; return the set of matched skills + whether the match was ambiguous (more than one specific skill, excluding always-on family). - `lintRoutingFixtures`: rejects fixtures whose intent is verbatim-equal to a trigger (D-CX-6: fixtures must paraphrase the framing, not copy the trigger text) and unknown expected_skill references. - `loadRoutingFixtures(skillsDir)`: walks `skills/<name>/routing-eval.jsonl`, handles JSONL line-comments (`//` / `#`), collects malformed lines separately without crashing. - `runRoutingEval(resolver, fixtures)`: pure scoring. Supports negative cases (`expected_skill: null` — nothing should match) and an `ambiguous_with` allow-list for skills that co-fire with always-on handlers (signal-detector, brain-ops, ingest). Outcomes per fixture: `pass`, `missed`, `ambiguous`, `false_positive`. Metrics: `top1Accuracy`, `passed`, `missed`, `ambiguous`, `falsePositives`. ## Integration: check-resolvable runs Layer A by default `checkResolvable()` now loads `routing-eval.jsonl` fixtures from every skill, runs the structural eval, and appends non-pass outcomes as warning-severity issues. New issue types: - `routing_miss` — expected skill did not match - `routing_ambiguous` — expected matched AND unexpected skills - `routing_false_positive` — negative case unexpectedly matched - `routing_fixture_lint` — linter or malformed-JSONL finding All four are warnings — routing issues don't break exit in default mode, but `--strict` promotes them (D-CX-3 contract). Advisories without breaking CI. ## New CLI verb: `gbrain routing-eval` Standalone Check 5 runner. `--json` envelope, `--llm` flag reserved, `--skills-dir` override. Exit codes: 0 clean, 1 any failure/lint, 2 setup error. Suitable for CI gating separately from check-resolvable. Removed from DEFERRED in CLI: `{check: 5, name: trigger_routing_eval}`. Check 6 (brain_filing) still deferred; lands in W3. ## Seed fixtures - skills/query/routing-eval.jsonl - skills/citation-fixer/routing-eval.jsonl (includes a negative case) These are intentionally modest. Additional fixtures per skill are the natural next step; routing-eval itself passes cleanly under check-resolvable default mode even when fixtures surface real gaps (they're warnings, not errors). Running `gbrain routing-eval` reveals the gaps immediately. ## Tests (34 new cases + updated integrations) - test/routing-eval.test.ts: full harness coverage including normalization, trigger extraction (quoted and unquoted), indexer, structural match with ambiguity + always-on exemption, fixture linter (verbatim-equality rule, unknown-skill rule, shape rule, negative-case skip), JSONL loader (comments, malformed lines, missing dirs, underscore/dot skipping), and every runRoutingEval outcome (pass, miss, ambiguous, negative-pass, false-positive, empty). - test/check-resolvable-cli.test.ts: updated DEFERRED unit test + `--json` envelope test + `--verbose` test to reflect Check 5 shipping. 140/140 passing across the W1 + W2 surface. ## Live smoke `gbrain routing-eval --json` against the current gbrain repo: 6 fixtures, 1 passing, 5 missed. The misses correctly surface resolver-trigger narrowness (intents users naturally phrase differently than trigger text). Fixtures will iterate in follow-up PRs; the machinery ships now. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W3 — Check 6 brain filing audit Check 6 ships. Every skill that writes brain pages is now audited against a machine-readable filing-rules doc at `skills/_brain-filing-rules.json`. ## New: skills/_brain-filing-rules.json Canonical filing rules, JSON (D-CX-8: the pre-existing yaml-lite parser handles flat maps only, so YAML would have needed a new dependency for one file). The companion `_brain-filing-rules.md` stays as the human explainer. 14 rule entries + explicit `sources_dir` carve-out for bulk/raw data. ## New module: src/core/filing-audit.ts - `loadFilingRules(skillsDir)`: returns parsed doc or null (missing file → no-op; malformed JSON throws loud). - `allowedDirectories(rules)`: normalized set of every rules[] directory + sources_dir. - `runFilingAudit(skillsDir)`: walks skills/*/SKILL.md, parses frontmatter, audits any skill with `writes_pages: true`. Two checks per qualifying skill: 1. `writes_to:` list is non-empty. 2. Every entry in `writes_to:` appears in allowedDirectories. Both failures emit warning-severity issues. No errors — advisories only, per D-CX-3. ## Distinction: writes_pages vs mutating (D-CX-7) v0.17 introduces a new boolean frontmatter field `writes_pages:`. `mutating: true` already means "has any side effect" (cron schedulers, report writers, config mutators). Filing audit targets ONLY skills with `writes_pages: true`, correctly excluding side- effect-but-not-page-writing skills. The codex outside voice caught this: conflating the two fields would drag ~100 skills into filing-audit noise in the reference OpenClaw deployment. ## Integration: check-resolvable runs Check 6 by default `checkResolvable()` calls `runFilingAudit(skillsDir)` and appends issues as warnings. On missing/malformed rules doc, surfaces a single advisory rather than bailing. `DEFERRED` array in the CLI is now empty — v0.17 ships both Check 5 (W2) and Check 6 (W3). The export stays in place (stable --json field) for future deferred checks. ## Seeded frontmatter on 7 canonical writers Added `writes_pages: true` + `writes_to:` to: - brain-ops (people, companies, deals, concepts, meetings) - enrich (people, companies) - ingest (people, companies, concepts, meetings, sources) - idea-ingest (people, concepts, sources) - media-ingest (concepts, people, companies, sources) - meeting-ingestion (meetings, people, companies) - signal-detector (people, companies, concepts) Live smoke: `gbrain check-resolvable --json` on gbrain repo shows `ok: true`, zero filing errors, zero filing warnings on seeded skills. Every other mutating:true skill (citation-fixer, cron-scheduler, data-research, maintain, migrate, minion-orchestrator, reports, setup, skill-creator, soul-audit, webhook-transforms) correctly skipped as side-effectful-but-not-page-writing. ## Tests (17 new cases + 3 updated CLI integrations) test/filing-audit.test.ts covers: - rules loader: missing (null), valid, malformed (throw), non-array rules (throw) - directory normalization (trailing slash, leading slash) - clean case - missing writes_to on writes_pages:true - unknown directory - D-CX-7: mutating:true alone does not trigger audit - writes_pages:false skips - no frontmatter skips - inline `writes_to: [a, b]` syntax - block `writes_to:\n - a` syntax - sources/ allowed - underscore/dot dir skipping - total counts (totalScanned vs writesPagesSkills) - missing dir graceful - action string quality guard Plus: CLI integration tests updated for empty DEFERRED array (Checks 5 and 6 both shipped). 158/158 passing across the v0.17 foundation + W1 + W2 + W3 surface. ## v0.18 preview (D-CX-13) v0.17 filing-audit is declaration-level only. A future `gbrain filing-audit --pages` walks the brain itself, infers primary subject from page content via LLM judgment, and flags actual misfilings vs. declarations. Declaration audit is the leading indicator; pages audit is the ground truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W4 — gbrain skillify {scaffold,check} subcommand namespace The essay's "skillify it!" verb becomes a CLI primitive pair. Two subcommands, both promoted/factored so there's one source of truth: ## `gbrain skillify scaffold <name>` (mechanical) Pure file generation. Zero LLM, zero judgment. Writes 5 stub files atomically: 1. skills/<name>/SKILL.md frontmatter + body template 2. skills/<name>/scripts/<name>.mjs deterministic-code stub 3. skills/<name>/routing-eval.jsonl routing fixture seed 4. test/<name>.test.ts vitest skeleton 5. Appended trigger row to the detected resolver file (RESOLVER.md or AGENTS.md — whatever W1's auto-detect found) Flags: --description (required), --triggers, --writes-to, --writes-pages, --mutating, --force, --dry-run, --json, --skills-dir. Kebab-case name validation (`^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$`). Works against gbrain-native RESOLVER.md layout AND OpenClaw-native AGENTS.md-at-workspace-root layout (W1 interop). ## `gbrain skillify check [path]` (audit) Promoted from scripts/skillify-check.ts per codex D-CX-2. The legacy script stays as a 12-line shim that delegates to the new module so existing callers (docs, cron, tests) keep working. Wrapped in a subcommand namespace: `gbrain skillify {scaffold, check}` is one coherent verb for the whole post-task loop. The essay's "skillify it!" triggers the markdown skill, which orchestrates the CLI primitives. ## Idempotency contract (D-CX-7) `skillify scaffold --force` regenerates stub FILES but never re-appends a resolver row that already references `skills/<name>/SKILL.md`. Unit test pins this: two applies produce one resolver row, not two. ## D-CX-9 SKILLIFY_STUB sentinel Every scaffolded script + SKILL.md body carries a SKILLIFY_STUB sentinel. `check-resolvable` walks every skill's script dir looking for the marker and emits a `skillify_stub_unreplaced` warning when found. Default mode: advisory. `--strict` mode: error, blocks CI. This is the gate that catches "we scaffolded and forgot to implement" — the exact failure codex flagged as "scaffold verification is theater" in the outside-voice review. ## Files - NEW src/core/skillify/templates.ts (template strings) - NEW src/core/skillify/generator.ts (planScaffold / applyScaffold + SkillifyScaffoldError with typed error codes) - NEW src/commands/skillify.ts (top-level dispatcher + scaffold handler) - NEW src/commands/skillify-check.ts (promoted check logic) - scripts/skillify-check.ts: rewritten to 12-line shim - skills/skillify/SKILL.md: Phase 2 now references the scaffold primitive; legacy manual path kept for extending existing skills - src/cli.ts: `skillify` added to CLI_ONLY + dispatcher - src/core/check-resolvable.ts: SKILLIFY_STUB sentinel scan + new issue type `skillify_stub_unreplaced` ## Tests (14 new scaffold cases) test/skillify-scaffold.test.ts covers: - SKILL_NAME_PATTERN validation (kebab-case, no spaces, no leading digit, no underscores/uppercase) - planScaffold against fresh + existing-file + --force paths - SKILLIFY_STUB sentinel presence in SKILL.md AND script stub (both gate paths) - D-CX-7 idempotency: resolverAppend null when row pre-exists, second apply doesn't duplicate the row - TBD-trigger placeholder when --triggers empty - writes_pages / writes_to / mutating flow through to frontmatter - applyScaffold writes files + appends resolver - Full AGENTS.md-layout workspace interop (W1) Existing test/skillify-check.test.ts still passes against the legacy shim — zero regression for downstream consumers. 178/178 passing across v0.17 foundation + W1..W4. ## Live smoke \`gbrain skillify scaffold webhook-verify --description "verify incoming webhook signatures" --triggers "verify webhook,check tunnel" --skills-dir /tmp/smoke --dry-run\` produces the expected 4-file plan plus a 115-byte resolver append. \`--help\` works on both the top-level and scaffold levels. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W5 — gbrain skillpack install (deps closure + lockfile + diff/dry-run) The essay's "drop it into YOUR OpenClaw" promise lands as a CLI verb. One command installs a curated bundle of gbrain skills + the shared convention files they depend on into a target OpenClaw workspace. Data-loss protected, concurrency-safe, atomic on the AGENTS.md managed block. ## openclaw.plugin.json refresh - Bumped version from stale 0.4.1 → 0.17.0 (codex flagged this drift F-ENG-4 / D-CX-4). - Expanded curated skill list from 7 → 25. Uses skills/manifest.json top-level (v0.10.0 sourced) minus setup/migrate/publish (install-time / code+skill pairs) minus private skills. - Added \`shared_deps: [...]\` listing convention files every skill references: conventions/, _brain-filing-rules.{md,json}, _output-rules.md. Installer always pulls these (D-CX-10 dependency closure). - Added \`excluded_from_install: [...]\` for setup/migrate/publish — surfaces the intentional exclusion as data rather than a comment. ## New module: src/core/skillpack/bundle.ts - \`findGbrainRoot(start)\` — walks up looking for openclaw.plugin.json + src/cli.ts. The pair identifies a gbrain checkout. - \`loadBundleManifest(root)\` — strict validation + typed BundleError codes (manifest_not_found, manifest_malformed, skill_not_found). - \`enumerateBundle({gbrainRoot, skillSlug?, manifest})\` — flat list of source → target-relative paths. When skillSlug is set, scopes to that one skill BUT always pulls shared_deps. \`--all\` walks every skill in the manifest. - \`bundledSkillSlugs(manifest)\` — sorted slugs for \`skillpack list\`. ## New module: src/core/skillpack/installer.ts - \`planInstall(opts)\` — builds InstallPlan with per-file existing/identical diff state. Pure; no writes. - \`applyInstall(plan, opts)\` — writes files + managed block with the contracts below. - \`diffSkill(root, slug, skillsDir)\` — read-only per-file status for \`skillpack diff <name>\`. **Per-file diff protection (D-CX-3 / F4):** wrote_new fresh file wrote_overwrite local diff + --overwrite-local passed skipped_identical bytes match the bundle (silent re-install) skipped_locally_modified target differs + no --overwrite-local → PROTECTED DEFAULT **Concurrency + atomic AGENTS.md (D-CX-11):** - \`.gbrain-skillpack.lock\` at workspace root. Acquired on the first write, released in finally. - Lock stale threshold configurable (default 10min). --force-unlock overrides. - Managed-block writes via tmp-file-plus-rename (atomic on POSIX). **Managed-block format:** <!-- gbrain:skillpack:begin --> <!-- Installed by gbrain <version> — do not hand-edit between markers. --> | Trigger | Skill | |---------|-------| | "alpha" | \`skills/alpha/SKILL.md\` | | ... <!-- gbrain:skillpack:end --> extractManagedSlugs() roundtrips: single-skill installs accumulate into the same block rather than overwriting each other. ## New CLI: gbrain skillpack {list, install, diff, check} Namespaced alongside W4's \`gbrain skillify\`. Subcommands: list bundle inventory (human + --json) install <name> single skill + deps closure install --all entire curated bundle diff <name> per-file diff vs target; read-only check delegates to the pre-existing skillpack-check (same CLI just namespaced) Flags on install: --overwrite-local, --force-unlock, --dry-run, --json, --skills-dir, --workspace. Exit codes: 0 clean, 1 files skipped (protected local edits), 2 setup error / lock held. ## Live smoke \`gbrain skillpack list\`: 25 skills. \`skillpack install query --dry-run\` against a fresh temp workspace: 12 files planned (SKILL.md, routing-eval.jsonl, 7 convention files, 3 rule files, managed block to AGENTS.md). All shared_deps flagged [shared]. ## Tests (36 new cases) test/skillpack-install.test.ts: - findGbrainRoot walks up, returns null when absent - loadBundleManifest validates + rejects malformed - enumerateBundle pulls shared_deps on single-skill scope (D-CX-10) - buildManagedBlock + updateManagedBlock: append when absent, in-place replace when present, extractManagedSlugs roundtrip - planInstall + applyInstall: fresh install, dry-run, idempotency (skipped_identical), local-edit protection, --overwrite-local, lock-held concurrency (D-CX-11), --force-unlock, atomic managed-block write, multi-skill accumulation in managed block, AGENTS.md-at-workspace-root interop (W1 cross-check) - diffSkill: missing, identical, differs test/skillpack-sync-guard.test.ts (F-ENG-4): - both manifests exist - every skill in plugin.json exists on disk - every shared_dep exists on disk - plugin.json skills ⊂ skills/manifest.json - excluded skills aren't in the install list - plugin version ≥ 0.17 (kills the 0.4.1 stale drift) 204/204 passing across the v0.17 foundation + W1..W5. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 guards — privacy scrub + OpenClaw-reference E2E + v0.16.4 regression Three ship-blocker work items from the eng review + codex outside voice round out v0.17: ## scripts/check-privacy.sh (CLAUDE.md:550 enforcement) Greps for the banned OpenClaw fork name (case-insensitive) across tracked files. Two modes: scripts/check-privacy.sh scan working tree scripts/check-privacy.sh --staged scan git-staged files (pre-commit) Exit 1 on any finding outside the allow-list. Allow-list covers files where the name is legitimately present: this script itself (defines the rule), CLAUDE.md (the canonical rule text), llms-full.txt (auto-generated from CLAUDE.md), the historical upgrade guide, and test/integrations.test.ts (whose personal-info regex ENFORCES the rule against recipes/). Scrubbed existing leaks: - CHANGELOG.md:366 reference in a closes-# line → "from the OpenClaw reference deployment" - test/doctor-minions-check.test.ts:171 comment → "an OpenClaw host's cron script" - test/plugin-loader.test.ts fixture plugin name → "openclaw-ref" ## test/e2e/openclaw-reference-compat.test.ts (ship-blocker gate) The test that proves v0.17 delivers on the headline claim. New fixture at test/fixtures/openclaw-reference-minimal/ mimics the reference OpenClaw deployment layout: AGENTS.md at workspace root, skills/ below, no manifest.json. Four fixture skills (signal-detector, query, brain-ops, context-now). Every v0.17 surface gets exercised end-to-end: - autoDetectSkillsDir with $OPENCLAW_WORKSPACE (D-CX-4 priority) - loadOrDeriveManifest walks SKILL.md (F-ENG-1 auto-manifest) - checkResolvable accepts AGENTS.md at workspace root, all 4 skills reachable via resolver rows, zero errors - Filing audit clean (brain-ops declares writes_pages+writes_to) - CLI subprocess via `--skills-dir` → exit 0 - CLI subprocess via $OPENCLAW_WORKSPACE (no flag) → exit 0, correct skillsDir detection - skillpack install against the layout writes managed block into AGENTS.md at workspace root This is THE ship-blocker test. If the W1 + W5 stack ever regresses against an AGENTS.md-layout workspace, this fails first. ## test/regression-v0_16_4.test.ts (F-ENG-8) Guards v0.17 against adding "surprise" warnings. Builds a clean fixture matching v0.16.4 canonical shape (manifest.json, RESOLVER.md, 2 skills, no routing-eval fixtures, no writes_pages). Runs v0.17 checkResolvable and asserts: - zero errors, zero routing_*/filing_*/skillify_stub_* warnings - JSON envelope keys unchanged (errors, warnings, issues, ok, summary) — deprecated `issues[]` still equals errors ∪ warnings - summary shape unchanged If someone adds a new check that fires unexpectedly on a v0.16.4-era fixture, this test catches it immediately. ## Fixture test/fixtures/openclaw-reference-minimal/ ├── AGENTS.md (4 rows, 3 sections) └── skills/ ├── brain-ops/SKILL.md (writes_pages+writes_to) ├── context-now/SKILL.md ├── query/SKILL.md └── signal-detector/SKILL.md Intentionally small (4 skills, 1 AGENTS.md, ~30 lines total) so the fixture is maintainable. The OPENCLAW-reference deployment has 107 skills — this fixture is the minimum shape that exercises the full v0.17 code path. ## Tests 215/215 passing across the full v0.17 surface: - foundation + W1 + W2 + W3 + W4 + W5 (204) - regression-v0_16_4 (3) - openclaw-reference-compat (7) - privacy guard (separate bash; exits 0 clean) Plus: privacy pre-commit hook is a drop-in wrapper (documented in the script header). Wiring into .github/workflows is a follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * release: v0.17.0 — skillify goes end-to-end Every skill. Every check. Every install. One command each. Five workstreams land in one release: - W1: AGENTS.md + auto-manifest + env-priority - W2: Check 5 routing eval - W3: Check 6 brain filing - W4: gbrain skillify {scaffold,check} - W5: gbrain skillpack {list,install,diff} Plus D-CX-3 foundation (errors/warnings split + --strict), plus codex outside-voice fixes (D-CX-1..12 applied), plus privacy pre- commit guard, plus OpenClaw-reference E2E fixture, plus v0.16.4 regression guard. Live against the reference OpenClaw deployment: 102 skills detected via auto-manifest, 15 unreachable errors + 108 warnings surfaced — exactly the essay's "~15% dark" finding. The magic word from the essay finally works the way the essay describes. Tests: 2156 unit (178 new) + 152 E2E Tier 1 + 3 Tier 2 + 8 new openclaw-reference fixture cases. 0 failures across all tiers. Plan + reviews: ~/.claude/plans/p1-lets-just-vast-blanket.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(test): add missing 'strict' field to 5 Flags literals in check-resolvable-cli.test.ts CI failed `tsc --noEmit` after the D-CX-3 errors/warnings split added `strict: boolean` as a required field on the `Flags` interface. Five test sites in test/check-resolvable-cli.test.ts still construct Flags object literals (for direct `resolveSkillsDir()` calls) and hadn't been updated. Added `strict: false` to all five literals: - line 129 --skills-dir absolute path - line 135 --skills-dir relative path - line 148 no --skills-dir - line 160 no --skills-dir + no env - line 178 --skills-dir + OPENCLAW_WORKSPACE (REGRESSION-GATE) Unit tests: 207/207 pass across the v0.19 surface. tsc --noEmit exits 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: adopt gstack's branch-scoped CHANGELOG rule + rewrite v0.19.0 entry CLAUDE.md gains a new top section before "CHANGELOG voice" that codifies what gstack's CLAUDE.md already says: CHANGELOG is user-facing product release notes, not a log of internal decisions. Every entry describes what THIS branch adds vs master. Plan-file IDs, decision tags (D-CX-#, F-ENG-#), review rounds, test counts as marketing, and contributor- facing metrics don't belong in it. The v0.19.0 entry is rewritten to the new bar: Removed: - Version-collision note about v0.17.0/v0.18.0 shipping on master - All D-CX-## and W# tags (meaningless outside the plan file) - "codex caught" / CEO + Eng review round-up narrative - Plan file path reference - "215 new cases across 13 test files" marketing metrics - W1..W5 bucketing in itemized changes Kept / sharpened: - User-facing headline (what your agent can now do) - Numbers that mean something to users (unreachable-skills count, scaffold timing, pre/post AGENTS.md support) - Upgrade instructions - Added/Changed/Fixed/For-contributors itemized sections (standard keep-a-changelog shape) Version sequence (`grep "^## \["`) is contiguous v0.19.0 → v0.16.4. Privacy guard clean. Tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update README/CLAUDE/TODOS for v0.19.0 skills + skillify loop Skill count was stale (README said 26, actual is 28: skillify + skillpack-check were missing from the tables and count). Corrected throughout. Marked TODOS item "Checks 5 + 6 deferred in PR #325" as completed in v0.19 — they shipped as real implementations, not just filed issues. README: - Skill count 26 → 28 (headline, install flow, table section, architecture diagram) - Added `skillify` + `skillpack-check` rows to the operational skills table - Rewrote the "Skillify" section to lead with the four v0.19 CLI verbs (`gbrain skillify scaffold/check`, `gbrain skillpack list/install/diff`, `gbrain routing-eval`, `gbrain check-resolvable --strict`) instead of describing the pre-v0.19 state. Added the "works on your OpenClaw" pitch around AGENTS.md + auto-manifest. Added the "drop 25 curated skills into your OpenClaw" section for skillpack install. - Added v0.19 skills block + v0.18 multi-source + v0.17 dream to the Commands reference at the bottom. - Standalone instruction sets count: 25 → 28 (with a parenthetical noting the curated 25-skill bundle that `skillpack install` ships). CLAUDE.md: - Skill count 26 → 28 in the Skills section. - New "Skillify loop (v0.19)" sub-bullet listing skillify + skillpack-check. - Noted that `AGENTS.md` is also accepted as a resolver filename. TODOS.md: - Created "## Completed" section at the top. - Moved the "Checks 5 + 6" item there with completion note linking to the actual implementation files (routing-eval.ts + filing-audit.ts). Privacy scan clean. Version sequence contiguous v0.19.0 → v0.16.4. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(test): regenerate llms-full.txt + llms.txt after README/CLAUDE edits CI failed on `build-llms generator > committed llms.txt + llms-full.txt match current generator output`. The drift was expected: the prior commit edited README.md and CLAUDE.md (skill count + skillify section), both of which are inlined into llms-full.txt by `scripts/build-llms.ts`. Fix: `bun run build:llms` + commit the regenerated output. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Wintermute <wintermute@garrytan.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-23 19:34:27 -07:00
Garry Tan	f541f045d2	feat: add gbrain check-update command and auto-update agent workflow (#15) ... * feat: add `gbrain check-update` command for auto-update notifications Deterministic collector that checks GitHub Releases for new versions, compares semver (minor+ only, skips patches), and fetches changelog diffs. Exports `detectInstallMethod()` from upgrade.ts for reuse. Includes 15 unit tests covering version comparison, CLI wiring, and error handling. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add E2E upgrade tests against real GitHub API Exercises check-update CLI end-to-end: valid JSON output, human-readable mode, help text, graceful no-releases handling, and version comparison wiring. Skips gracefully when network is unavailable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add SKILLPACK Section 17 — auto-update notifications Full agent playbook for the update lifecycle: check, notify, consent, upgrade, skills refresh, schema sync, report. Includes standalone self-update for skillpack-only users via version markers and raw GitHub URL fetching. Adds version markers to both SKILLPACK and RECOMMENDED_SCHEMA headers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add auto-update step 7 to install paste, setup Phase G, migrations dir Adds step 7 to the OpenClaw install paste (default-on update checks). Setup skill gets Phase G (conditional offer for manual installs) and schema state tracking via ~/.gbrain/update-state.json. Creates skills/migrations/ directory for version-specific upgrade directives. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update CLAUDE.md with E2E test DB lifecycle, migration conventions Adds E2E test DB lifecycle instructions (spin up, run, tear down). Documents version migration convention (skills/migrations/v[version].md) and schema state tracking (~/.gbrain/update-state.json). Updates test file counts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: broken semver comparison in extractChangelogBetween The version range check compared minor versions without guarding on major being equal, causing incorrect changelog entries to be captured (e.g., v0.5.0 would match when upgrading from v1.2.0). Extracted semverGt/semverLte helpers for correct comparisons. Added 5 tests for extractChangelogBetween covering cross-major, same-version, and malformed input cases. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.4.1) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-09 12:25:04 -10:00
Garry Tan	912a321cfa	GBrain v0.4.0 — production agent documentation + reference architecture (#10) ... * fix: widen validateSlug to accept any filename characters Git is the system of record. Slugs are lowercased repo-relative paths. The restrictive regex rejected spaces, parens, and special chars, blocking 5,861 Apple Notes files from importing. Now only rejects empty slugs, path traversal (..), and leading slash. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: enable RLS on all tables with BYPASSRLS safety check Without RLS, the Supabase anon key gives full read access to the DB. Enable RLS on all 10 tables with no policies — the postgres role (used by gbrain via pooler) has BYPASSRLS and is unaffected. Only enables if the current role actually has BYPASSRLS privilege to avoid locking ourselves out on non-Supabase setups. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: import resilience — 5MB limit, error suppression, structured progress Raise MAX_FILE_SIZE from 1MB to 5MB for Apple Notes with attachments. Track error patterns and suppress after 5 identical errors to prevent 5,861 identical warnings from killing the agent process. Replace \r progress bar with structured log lines (rate, ETA) for agent parsing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: init detects IPv6-only Supabase URLs, adds pgvector check Detect db.*.supabase.co direct URLs and warn about IPv6 failure. On ECONNREFUSED/ETIMEDOUT to Supabase, suggest the Session pooler connection string with exact dashboard click path. Check for pgvector extension after connecting and fail with clear instructions if missing. Update wizard hints to show pooler URL format. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add pre-ship requirement for E2E tests E2E tests against real Postgres+pgvector must pass before /ship or /review. Adds the requirement to CLAUDE.md so all agents enforce it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: parallel import with per-worker engine instances Refactor PostgresEngine to support instance-level DB connections instead of only the module-global singleton. Each worker gets its own connection with poolSize:2 (vs 10 for the main engine), so 8 workers = 16 connections. Add --workers N flag to gbrain import. Workers pull from a shared queue and use independent engine instances — no transaction context corruption. The bottleneck is network round-trips to Supabase (one per page upsert). Parallel workers cut import time proportionally. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: automatic schema migration runner Migrations are embedded as string constants in migrate.ts (survives Bun --compile). Each migration runs in a transaction for clean rollback on failure. Runs automatically on initSchema() — no manual step needed when a user updates the gbrain binary against an older DB. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: pluggable storage backend (S3 + Supabase Storage + local) Add StorageBackend interface with three implementations: - S3Storage: works with AWS S3, Cloudflare R2, MinIO (any S3-compatible) - SupabaseStorage: uses Supabase Storage REST API with service role key - LocalStorage: filesystem-based, for testing Add file-resolver.ts with fallback chain: local file → .redirect breadcrumb → .supabase marker → storage backend. Supports the three-stage migration (mirror → redirect → clean). Add yaml-lite.ts for parsing marker and breadcrumb files without adding a YAML dependency. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: gbrain doctor command — health checks with --json output Checks: connection, pgvector extension, RLS on all tables, schema version, embedding coverage. Outputs structured JSON with --json flag for agent parsing. Exit code 0 if healthy, 1 if issues found. Agents should run gbrain doctor --json when any command fails. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: rewrite setup skill + README for agent-first DX Setup skill: add Why Supabase, step-by-step project creation, explicit agent instructions (nohup for large imports, doctor on failure, don't ask for anon key), available init flags, file migration offer after first import. Remove ClawHub references. README: simplify to single OpenClaw install path, remove ClawHub, fix squatted npm name to github:garrytan/gbrain, add Supabase settings note about Session pooler. Add Apple Notes test fixtures with spaces and parens in filenames for E2E testing of the slug fix. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add RLS verification, schema health, and nohup hints to maintain skill Maintenance skill now checks RLS status and schema version as part of periodic health checks. Adds nohup pattern for large embedding refreshes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: import resume checkpoint + Supabase smart URL parsing Import resume: saves checkpoint every 100 files to ~/.gbrain/import-checkpoint.json. On restart with same directory and file count, skips already-processed files. Use --fresh to ignore checkpoint and start over. Cleared on successful completion. Supabase admin: extractProjectRef() parses any Supabase URL format (dashboard, direct, pooler, project URL) to extract the project ref. discoverPoolerUrl() uses the Management API to find the correct pooler connection string (including the exact region prefix). checkRls() verifies RLS status via the API. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add 56 unit tests for all new code 8 new test files covering every feature added in this branch: - slug-validation.test.ts: spaces, parens, unicode, path traversal (10 tests) - yaml-lite.test.ts: parse + stringify, marker/redirect formats (9 tests) - supabase-admin.test.ts: extractProjectRef for 4 URL formats (7 tests) - migrate.test.ts: version export, runMigrations callable (2 tests) - storage.test.ts: LocalStorage CRUD + createStorage factory (14 tests) - file-resolver.test.ts: fallback chain, redirect, marker parsing (6 tests) - import-resume.test.ts: checkpoint save/load/resume/fresh (6 tests) - doctor.test.ts: module export, CLI registration (3 tests) Total: 184 pass, 0 fail (up from 128). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: bulk chunk INSERT + E2E tests for all new features Bulk INSERT: upsertChunks now builds a multi-row VALUES query instead of inserting chunks one-by-one. Reduces DB round-trips by ~50x per page. E2E tests added to mechanical.test.ts: - Slug with special chars: import Apple Notes fixtures with spaces/parens, verify search finds them, verify idempotency - RLS verification: check pg_tables.rowsecurity on all tables, verify current user has BYPASSRLS - Doctor command: verify exit 0 on healthy DB, --json produces valid JSON with check structure - Parallel import: --workers 2 produces same page count as sequential Unit tests added: - setup-branching.test.ts: IPv6 detection, defaultWorkers auto-tuning, smart URL parsing across all Supabase URL formats Fixtures added: - large/big-file.md (2.1MB) for testing raised file size limit - apple-notes/ fixtures already existed Total: 200 pass, 0 fail (up from 184). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: --json on init/import, file migration CLI, lifecycle tests --json flag: init and import now support --json for structured output. Agents get parseable JSON instead of human-readable text. File migration CLI: implement mirror, unmirror, redirect, restore, clean, and status subcommands for the three-stage file migration lifecycle (local → mirrored → redirected → cloud-only). File migration tests: full lifecycle test covering every transition in the state machine (LOCAL → MIRROR → UNMIRROR → REDIRECT → RESTORE → CLEAN), including edge cases and file resolver at each stage. Bulk chunk INSERT: upsertChunks now builds multi-row parameterized VALUES query, reducing round-trips per page from ~50 to 1. Total: 207 pass, 0 fail. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: thorough E2E tests for parallel import concurrency Replace the weak single-comparison parallel import test with 7 tests: - Sequential baseline: capture page count, chunk count, and all slugs - --workers 2: verify page count matches sequential - Chunk count matches (no duplicates from concurrent writes) - Page slugs match exactly - No duplicate pages (SQL GROUP BY HAVING count > 1) - No duplicate chunks (SQL GROUP BY page_id, chunk_index) - --workers 4: also works correctly - Re-import with workers is idempotent These tests catch the exact bug Codex found (db.ts singleton causing concurrent transaction corruption) by verifying data integrity after parallel writes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add batch embedding queue as P1 TODO Deferred during eng review (per-worker embedding is good enough for now). Revisit after profiling real imports to confirm embedding is the bottleneck. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: E2E test failures — fixture counts, arg parsing, doctor exit code Fix fixture count assertions: 13 → 16 pages (added apple-notes + large file), companies 2 → 3 (ohmygreen), concepts 3 → 5 (notes, big-file). Fix --workers arg parsing: the worker count value (e.g. "2") was being picked up as the directory arg. Skip flag values when finding the dir. Fix doctor exit code: warnings (like missing embeddings) should exit 0, only actual failures exit 1. E2E tests import with --no-embed, so embeddings are always WARN. Fix E2E CLI tests: add initCli() before doctor and parallel import tests so ~/.gbrain/config.json exists for the subprocess. All E2E tests pass: 63 pass, 0 fail. All unit tests pass: 207 pass, 0 fail. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update project documentation for v0.4.0 New CHANGELOG entry for all post-0.3.0 features (doctor, storage backends, parallel import, resume checkpoints, RLS, schema migrations, --json output). Version bumped 0.3.0 → 0.4.0 across all manifests. CLAUDE.md: test count 9→19, skill count 8→7, added key files. CONTRIBUTING.md: fixture count 13→16, added missing source files. README.md: added gbrain doctor to commands, fixed stale welcome PRs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add GBRAIN_SKILLPACK.md reference architecture Production agent patterns from a real deployment with 14,700+ brain files. Covers: entity detection on every message, brain-first lookup protocol, 7-step enrichment pipeline with tiered API spend, compiled truth + timeline, source attribution with mandatory citations, meeting ingestion with entity propagation, cron schedule with quiet hours and travel-aware timezone, YouTube/media ingestion via Diarize.io, integration guides for ClawVisor, Circleback webhooks, and Quo/OpenPhone SMS. Opens with the Vannevar Bush memex framing and the originals folder for capturing intellectual capital. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: rewrite README opener with memex pitch and production architecture Replace code-first opener with mimetic-desire pitch: Vannevar Bush memex tagline, production brain numbers (10K+ files, 3K+ people, 13 years of calendar), "ask it anything" examples, compounding thesis. New sections: The Compounding Thesis (read-write loop), Architecture (three-column diagram), What a Production Agent Looks Like (SKILLPACK reference), How gbrain fits with OpenClaw (three-layer complement). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update skills with brain-first lookup, entity detection, heartbeat setup: Phase D rewritten with brain-first lookup protocol (gbrain search → query → get → grep fallback), sync-after-write rule, memory_search complement table. query: token-budget awareness (chunks not full pages), source precedence hierarchy (user > compiled truth > timeline > external). ingest: entity detection on every message (scan, check brain, create or enrich, commit and sync). maintain: heartbeat integration (doctor, embed --stale, sync verification, stale compiled truth detection). briefing: gbrain-native context loading (search attendees before meetings, search sender before email, daily deal/meeting/commitment queries). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add OpenClaw positioning to README opener Make it clear up top that GBrain is built for OpenClaw agents and works with any OpenClaw deployment. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: credit Karpathy's Knowledge LLM vision, add origin story GBrain started as Karpathy's LLM wiki idea built for real. Worked great until the brain hit thousands of files and grep fell apart. GBrain is the search layer that had to exist once the brain outgrew grep. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-09 10:17:13 -07:00
Garry Tan	a86f995883	feat: GBrain v0.3.0 — contract-first architecture + ClawHub plugin (#7) ... * feat: contract-first operations.ts with OperationError, dry_run, importFromContent 30 shared operations as single source of truth for CLI and MCP. - OperationError with typed error codes (page_not_found, invalid_params, etc.) - dry_run support on all mutating operations - importFromContent split from importFile with transaction wrapping - Idempotency hash now includes ALL fields (title, type, frontmatter, tags) - Config env var fallback: GBRAIN_DATABASE_URL > DATABASE_URL > config file Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: rewrite MCP server + CLI + tools-json from operations server.ts: 233 -> ~80 lines. Tool definitions and dispatch generated from operations[]. cli.ts: shared operations auto-registered, CLI-only commands kept as manual dispatch. tools-json: generated FROM operations[], eliminating the third contract surface. Parity test verifies structural contract between operations, CLI, and MCP. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: delete 12 command files migrated to operations.ts Handler logic for get, put, delete, list, search, query, health, stats, tags, link, timeline, and version now lives in operations.ts. Kept: init, upgrade, import, export, files, embed, sync, serve, call, config. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: init --non-interactive, upgrade verification, schema migration - gbrain init --non-interactive --url <url> for plugin mode (no TTY required) - Post-upgrade version verification in gbrain upgrade - Drop storage_url from files table (storage_path is the only identifier) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: tool-agnostic skills + new setup skill All 7 skills rewritten with intent-based language instead of CLI commands. Works with both CLI and MCP plugin contexts. New setup skill replaces install: auto-provision Supabase via CLI, AGENTS.md injection, target TTHW < 2 min. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: ClawHub bundle plugin, CI workflows, v0.3.0 - openclaw.plugin.json with configSchema, MCP server config, skill listing - GitHub Actions: test on push/PR, multi-platform release (macOS arm64 + Linux x64) - Version bump 0.3.0, CHANGELOG, README ClawHub section, CLAUDE.md updated Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: idempotency hash mismatch + MCP dry_run passthrough importFromContent now passes its all-fields hash through putPage via content_hash on PageInput, so the stored hash matches the computed hash. Previously the skip-if-unchanged check never fired because the hash formulas differed. MCP server now passes dry_run from tool params to OperationContext. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.3.0.0) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: schema loader handles PL/pgSQL $$ blocks Delete the semicolon-based SQL splitter in db.ts which broke on PL/pgSQL trigger functions containing semicolons inside $$ delimiter blocks. Use single conn.unsafe(schemaSql) call instead — the postgres driver handles multi-statement SQL natively. schema.sql already uses IF NOT EXISTS / CREATE OR REPLACE for idempotency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: E2E test infrastructure + realistic brain fixtures Add test infrastructure for running E2E tests against real Postgres+pgvector. Includes: - test/e2e/helpers.ts: DB lifecycle, fixture import, timing, diagnostics - 13 fixture files as a miniature realistic brain (people, companies, deals, meetings, concepts, projects, sources) following the compiled truth + timeline format from GBRAIN_RECOMMENDED_SCHEMA.md - docker-compose.test.yml: local pgvector convenience (port 5433) - .env.testing.example: template for test credentials - package.json: add test:e2e script Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: E2E test suites + CI workflow Tier 1 (mechanical.test.ts): 14 test suites covering all operations against real Postgres — page CRUD, search with quality scoring, links, tags, timeline, versions, admin, chunks, resolution, ingest log, raw data, files, idempotency stress, setup journey (full CLI flow), init edge cases, schema idempotency, schema diff guard, performance baselines. Tier 1 (mcp.test.ts): MCP protocol test — spawns server, sends JSON-RPC, verifies tools/list matches operations count. Tier 2 (skills.test.ts): OpenClaw skill tests — ingest, query, health. Skips gracefully when dependencies missing. CI (.github/workflows/e2e.yml): Tier 1 on every PR (pgvector service), Tier 2 nightly/manual with API key secrets. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: E2E test fixes + traverseGraph jsonb cast - Fix traverseGraph query: cast json_agg to jsonb_agg so SELECT DISTINCT works - Fix put_page tests to use importFromContent with noEmbed (no OpenAI key in Tier 1) - Fix get_health assertion (page_count not total_pages) - Fix raw_data test to handle JSONB string/object return - Simplify MCP test to verify tool generation directly - Add timeouts to CLI subprocess tests - Use port 5434 for docker-compose (5433 often in use) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: update all project docs for E2E test suite - CLAUDE.md: updated test count (9 unit + 3 E2E), added E2E test instructions, fixed skill count to 8 - CONTRIBUTING.md: updated project structure with test/e2e/, added E2E test instructions, rewrote "Adding a new command" to reflect contract-first architecture (add to operations.ts, done) - README.md: fixed table count (10 not 9), added recommended schema doc to Docs section, added E2E instructions to Contributing section - CHANGELOG.md: added E2E test suite, docker-compose, schema loader fix, and traverseGraph jsonb fix to v0.3.0 entry Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-08 23:26:11 -10:00