From d798d81462d421545f000d3788ca046c795e1199 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Sat, 11 Apr 2026 21:26:30 -1000 Subject: [PATCH] feat: v0.9.0 migration tells agents to swap scripts for built-in commands Migration file now: - Lists all 5 new deterministic commands with usage examples - Includes a script-to-command replacement table (old -> new) - Tells the agent to find custom script references in AGENTS.md, skills, and cron jobs and replace with gbrain commands - Adds recommended cron jobs for daily backlink fix + weekly lint - References the Thin Harness, Fat Skills thread --- skills/migrations/v0.9.0.md | 244 +++++++++++++++++++++++++----------- 1 file changed, 171 insertions(+), 73 deletions(-) diff --git a/skills/migrations/v0.9.0.md b/skills/migrations/v0.9.0.md index c65b205..16357d2 100644 --- a/skills/migrations/v0.9.0.md +++ b/skills/migrations/v0.9.0.md @@ -1,133 +1,231 @@ --- version: 0.9.0 feature_pitch: - headline: "Large files, smart uploads, production-grade skills" - description: "Files over 100 MB auto-upload to cloud storage via TUS resumable upload. .redirect.yaml pointers keep your git repo lean. All skills upgraded with battle-tested patterns from a production deployment." + headline: "5 new deterministic tools, smart file uploads, production-grade skills" + description: "gbrain publish, backlinks, lint, report, and upload-raw. Code+skill pairs -- deterministic TypeScript does the work, skills tell the agent when to use it. Plus TUS resumable uploads, .redirect.yaml pointers, and battle-tested skill patterns." tiers: - - name: "Core (no storage)" - description: "Upgraded skills only. Back-linking, filing rules, enrichment protocol, media ingest, citations." + - name: "Core tools (everyone)" + description: "publish, backlinks, lint, report -- zero external deps, work immediately." setup: "gbrain check-update && gbrain upgrade" - name: "With Supabase Storage" - description: "Full file management. Large files in cloud, .redirect.yaml pointers in git, signed URLs, TUS resumable upload." - setup: "Configure storage backend, then gbrain files mirror + redirect for existing binaries." + description: "upload-raw, signed-url, file migration lifecycle. Large files in cloud, git stays lean." + setup: "Configure storage backend, then gbrain files mirror + redirect." --- -# v0.9.0 Migration: Smart File Storage + Production Skills +# v0.9.0 Migration: Deterministic Tools + Smart File Storage -This migration covers both the file storage infrastructure (code changes) and -the skill patterns (latent space changes). No database schema changes required. +This is a major upgrade. GBrain now ships deterministic tools alongside skills -- +code for data, LLMs for judgment. No database schema changes required. -## What's New +## What's New: 5 Deterministic Commands -### File Storage Infrastructure (Code) +These commands run without LLM calls. They are the "code" half of the +[Thin Harness, Fat Skills](https://x.com/garrytan/status/2042925773300908103) pattern. -**Smart upload routing:** -- `gbrain files upload-raw --page --type ` auto-routes by size -- < 100 MB text/PDF: stays in git (returns `{storage: "git"}`) -- >= 100 MB or media: uploads to cloud storage, creates `.redirect.yaml` pointer -- Files >= 100 MB use TUS resumable upload (6 MB chunks with retry/backoff) +### 1. `gbrain publish` -- shareable HTML from brain pages -**New commands:** -- `gbrain files upload-raw` -- smart upload with size routing and pointer creation -- `gbrain files signed-url ` -- generate 1-hour signed URL for private files -- `gbrain files status` -- show migration state of directories +```bash +gbrain publish brain/people/jane-doe.md # local HTML +gbrain publish brain/people/jane-doe.md --password # auto-generated pw +gbrain publish brain/people/jane-doe.md --password "pw" # custom pw +gbrain publish brain/people/jane-doe.md --out share.html # custom output +``` -**Upgraded redirect format:** -- Old: `.redirect` (5 fields: moved_to, bucket, path, moved_at, original_hash) -- New: `.redirect.yaml` (10 fields: target, bucket, storage_path, size, size_human, - hash, mime, uploaded, source_url, type) -- File resolver supports BOTH formats for backward compatibility +Strips private data (frontmatter, citations, confirmations, brain links, timeline). +Optional AES-256-GCM encryption with client-side decryption. Dark/light mode, +mobile-optimized. Self-contained HTML, no server needed. -**File migration lifecycle:** -1. `gbrain files mirror ` -- upload to cloud, keep local copies -2. `gbrain files redirect ` -- replace local with `.redirect.yaml` (verifies remote first) -3. `gbrain files restore ` -- download back from cloud -4. `gbrain files clean --yes` -- remove pointers (cloud is sole source) +**Skill:** `skills/publish/SKILL.md` tells the agent when to publish, defaults +(always encrypt), and sharing workflows (local file, cloud upload + signed URL, +static hosting). -### Skill Patterns (Latent Space) +### 2. `gbrain backlinks check/fix` -- enforce the Iron Law -All patterns from v0.8.1 are included plus: -- Skills reference actual `gbrain files` commands (not theoretical patterns) -- Maintain skill includes file storage health checks -- Setup skill includes storage configuration phase with migration guidance -- Voice recipe uses `gbrain files upload-raw` for call audio -- Filing rules document the `.redirect.yaml` format and upload commands +```bash +gbrain backlinks check --dir /path/to/brain # report missing back-links +gbrain backlinks fix --dir /path/to/brain # create them +gbrain backlinks fix --dir /path/to/brain --dry-run # preview +``` + +Scans all pages for entity mentions (links to people/ and companies/), checks +if those entity pages link back. Creates timeline entries for missing back-links. + +### 3. `gbrain lint [--fix]` -- catch LLM slop + +```bash +gbrain lint /path/to/brain # report issues +gbrain lint /path/to/brain --fix # auto-fix what's fixable +gbrain lint brain/people/jane.md # lint single file +``` + +Catches: LLM preambles ("Of course! Here is..."), wrapping code fences, +placeholder dates (YYYY-MM-DD, XX-XX), missing frontmatter, broken citations, +empty sections. `--fix` strips the fixable ones automatically. + +### 4. `gbrain report` -- audit trail + +```bash +gbrain report --type enrichment-sweep --title "Weekly Sweep" --content "..." +echo "report body" | gbrain report --type meeting-sync --title "Daily Sync" +``` + +Creates: `brain/reports/{type}/{YYYY-MM-DD-HHMM}.md` with frontmatter. +Used by the maintain skill for enrichment sweeps, backlink checks, lint runs. + +### 5. `gbrain files upload-raw` -- smart file uploads + +```bash +gbrain files upload-raw --page --type +gbrain files signed-url +``` + +Auto-routes by size: < 100 MB text stays in git, >= 100 MB or media goes to +cloud storage via TUS resumable upload (6 MB chunks with retry). Creates +`.redirect.yaml` pointers in the brain repo. ## Agent Instructions ### Step 1: Verify upgrade ```bash -cat VERSION # should show 0.9.0 +gbrain --version # should show 0.9.0 gbrain doctor --json ``` -### Step 2: Configure storage backend (optional but recommended) +### Step 2: Replace custom scripts with built-in commands + +**This is the key step.** If your agent uses custom scripts for any of these +operations, swap them for the built-in gbrain commands: + +| Old pattern (custom script) | New pattern (built-in) | +|----------------------------|----------------------| +| `node scripts/brain-publish.mjs ` | `gbrain publish --password` | +| `node scripts/upload-raw-source.mjs --page ` | `gbrain files upload-raw --page ` | +| `node scripts/save-report.mjs --cron ` | `gbrain report --type --content "..."` | +| `node scripts/validate-brain-links.mjs` | `gbrain files verify` | +| `node scripts/migrate-brain-files.mjs` | `gbrain files mirror + redirect` | +| Manual back-link checking | `gbrain backlinks check --dir ` | +| Manual page quality review | `gbrain lint --fix` | + +**For each replacement:** +1. Find references to the old script in your AGENTS.md, skills, or cron jobs +2. Replace with the gbrain command +3. The gbrain command uses your configured storage backend automatically + +**Example: update AGENTS.md or skill files:** +``` +# Old: +node /data/.openclaw/workspace/scripts/upload-raw-source.mjs \ + --page --type transcript + +# New: +gbrain files upload-raw --page --type transcript +``` + +``` +# Old: +node /data/.openclaw/workspace/scripts/brain-publish.mjs --password + +# New: +gbrain publish --password +``` + +``` +# Old: +node /data/.openclaw/workspace/scripts/save-report.mjs \ + --cron "enrichment-sweep" --title "Enrichment Sweep" --content "..." + +# New: +gbrain report --type enrichment-sweep --title "Enrichment Sweep" --content "..." +``` + +### Step 3: Run the new tools on your brain + +```bash +# Check back-link health +gbrain backlinks check --dir /path/to/brain + +# If gaps found, fix them: +gbrain backlinks fix --dir /path/to/brain --dry-run # preview +gbrain backlinks fix --dir /path/to/brain # apply + +# Lint for quality issues +gbrain lint /path/to/brain + +# If fixable issues found: +gbrain lint /path/to/brain --fix --dry-run # preview +gbrain lint /path/to/brain --fix # apply + +# Save a report of what you fixed +gbrain report --type migration \ + --title "v0.9.0 Migration" \ + --content "Ran backlinks fix (N gaps fixed) and lint --fix (M issues fixed)." \ + --dir /path/to/brain +``` + +### Step 4: Configure storage backend (if not already done) If your brain repo has binary files (images, PDFs, audio, video), configure cloud storage to keep git lean: ```bash -# Option A: Supabase Storage (recommended if you already use Supabase) -# Get your service role key from: Supabase Dashboard > Project Settings > API +# Supabase Storage (recommended if you already use Supabase) gbrain config set storage.backend supabase gbrain config set storage.bucket brain-files gbrain config set storage.projectUrl https://YOUR-PROJECT.supabase.co gbrain config set storage.serviceRoleKey YOUR_SERVICE_ROLE_KEY -# Option B: S3-compatible (AWS, Cloudflare R2, MinIO) +# Or S3-compatible (AWS, Cloudflare R2, MinIO) gbrain config set storage.backend s3 gbrain config set storage.bucket brain-files gbrain config set storage.region us-east-1 gbrain config set storage.accessKeyId YOUR_KEY gbrain config set storage.secretAccessKey YOUR_SECRET -# For R2/MinIO: also set storage.endpoint ``` -Verify: `gbrain files upload test.txt --page test && gbrain files list test` +Then migrate existing binaries: +```bash +gbrain files status +gbrain files mirror +gbrain files redirect +``` -### Step 3: Migrate existing binary files (if storage configured) +### Step 5: Update cron jobs + +If you have cron jobs that call custom scripts, update them: ```bash -# See what you have -gbrain files status +# Old cron entry: +*/30 * * * * node /path/to/scripts/validate-brain-links.mjs -# If local binary files exist: -gbrain files mirror --dry-run # Preview what would upload -gbrain files mirror # Upload to cloud (keeps local) -gbrain files redirect --dry-run # Preview what would redirect -gbrain files redirect # Replace local with .redirect.yaml - -# Verify -gbrain files status -gbrain files signed-url # Test URL generation +# New: +*/30 * * * * gbrain files verify ``` -### Step 4: Upgrade legacy .redirect files (if any exist) - -If you have `.redirect` files from v0.8.x, they continue to work (backward -compatible). To upgrade them to the richer `.redirect.yaml` format: +### Step 6: Add new maintenance crons (recommended) ```bash -# Find legacy redirects -find -name "*.redirect" -not -name "*.redirect.yaml" +# Daily backlink check + auto-fix +0 3 * * * gbrain backlinks fix --dir /path/to/brain -# For each: restore then re-redirect (picks up new format) -gbrain files restore -gbrain files mirror -gbrain files redirect +# Weekly lint + auto-fix +0 4 * * 0 gbrain lint /path/to/brain --fix ``` -### Step 5: Review skills (same as v0.8.1) +### Step 7: Review skill updates -- Check 5 recent pages for back-link gaps -- Check sources/ for misfiled pages -- If voice recipe is set up: apply unicode/PII fixes -- If x-to-brain is set up: add image OCR, stagger crons +The skill files have been updated to reference gbrain commands: +- `skills/ingest/SKILL.md` -- uses `gbrain files upload-raw` for raw sources +- `skills/maintain/SKILL.md` -- uses `gbrain backlinks`, `gbrain lint`, `gbrain report` +- `skills/publish/SKILL.md` -- NEW skill for `gbrain publish` +- `skills/enrich/SKILL.md` -- references `gbrain files upload-raw` for raw API data +- `skills/_brain-filing-rules.md` -- documents `.redirect.yaml` format and commands -### Step 6: Done +Read the updated skills to pick up the new patterns. + +### Step 8: Done ```bash mkdir -p ~/.gbrain/migrations -echo '{"version":"0.9.0","ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","status":"complete","storage_configured":'$(gbrain config get storage.backend 2>/dev/null && echo true || echo false)'}' >> ~/.gbrain/migrations/completed.jsonl +echo '{"version":"0.9.0","ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","status":"complete","tools_available":["publish","backlinks","lint","report","upload-raw","signed-url"],"scripts_replaced":true}' >> ~/.gbrain/migrations/completed.jsonl ```