triton6564685/gbrain
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: v0.9.0 migration tells agents to swap scripts for built-in commands

Browse Source 
Migration file now:
- Lists all 5 new deterministic commands with usage examples
- Includes a script-to-command replacement table (old -> new)
- Tells the agent to find custom script references in AGENTS.md,
  skills, and cron jobs and replace with gbrain commands
- Adds recommended cron jobs for daily backlink fix + weekly lint
- References the Thin Harness, Fat Skills thread
This commit is contained in:
Garry Tan
2026-04-11 21:26:30 -10:00
parent 13fca3768b
commit d798d81462
1 changed files with 171 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files

244
skills/migrations/v0.9.0.md
View File

@@ -1,133 +1,231 @@
---
version: 0.9.0
feature_pitch:
headline: "Large files, smart uploads, production-grade skills"
description: "Files over 100 MB auto-upload to cloud storage via TUS resumable upload. .redirect.yaml pointers keep your git repo lean. All skills upgraded with battle-tested patterns from a production deployment."
headline: "5 new deterministic tools, smart file uploads, production-grade skills"
description: "gbrain publish, backlinks, lint, report, and upload-raw. Code+skill pairs -- deterministic TypeScript does the work, skills tell the agent when to use it. Plus TUS resumable uploads, .redirect.yaml pointers, and battle-tested skill patterns."
tiers:
- name: "Core (no storage)"
description: "Upgraded skills only. Back-linking, filing rules, enrichment protocol, media ingest, citations."
- name: "Core tools (everyone)"
description: "publish, backlinks, lint, report -- zero external deps, work immediately."
setup: "gbrain check-update && gbrain upgrade"
- name: "With Supabase Storage"
description: "Full file management. Large files in cloud, .redirect.yaml pointers in git, signed URLs, TUS resumable upload."
setup: "Configure storage backend, then gbrain files mirror + redirect for existing binaries."
description: "upload-raw, signed-url, file migration lifecycle. Large files in cloud, git stays lean."
setup: "Configure storage backend, then gbrain files mirror + redirect."
---
# v0.9.0 Migration: Smart File Storage + Production Skills
# v0.9.0 Migration: Deterministic Tools + Smart File Storage
This migration covers both the file storage infrastructure (code changes) and
the skill patterns (latent space changes). No database schema changes required.
This is a major upgrade. GBrain now ships deterministic tools alongside skills --
code for data, LLMs for judgment. No database schema changes required.
## What's New
## What's New: 5 Deterministic Commands
### File Storage Infrastructure (Code)
These commands run without LLM calls. They are the "code" half of the
[Thin Harness, Fat Skills](https://x.com/garrytan/status/2042925773300908103) pattern.
**Smart upload routing:**
- `gbrain files upload-raw <file> --page <slug> --type <type>` auto-routes by size
- < 100 MB text/PDF: stays in git (returns `{storage: "git"}`)
- >= 100 MB or media: uploads to cloud storage, creates `.redirect.yaml` pointer
- Files >= 100 MB use TUS resumable upload (6 MB chunks with retry/backoff)
### 1. `gbrain publish` -- shareable HTML from brain pages
**New commands:**
- `gbrain files upload-raw` -- smart upload with size routing and pointer creation
- `gbrain files signed-url <path>` -- generate 1-hour signed URL for private files
- `gbrain files status` -- show migration state of directories
```bash
gbrain publish brain/people/jane-doe.md # local HTML
gbrain publish brain/people/jane-doe.md --password # auto-generated pw
gbrain publish brain/people/jane-doe.md --password "pw" # custom pw
gbrain publish brain/people/jane-doe.md --out share.html # custom output
```
**Upgraded redirect format:**
- Old: `.redirect` (5 fields: moved_to, bucket, path, moved_at, original_hash)
- New: `.redirect.yaml` (10 fields: target, bucket, storage_path, size, size_human,
hash, mime, uploaded, source_url, type)
- File resolver supports BOTH formats for backward compatibility
Strips private data (frontmatter, citations, confirmations, brain links, timeline).
Optional AES-256-GCM encryption with client-side decryption. Dark/light mode,
mobile-optimized. Self-contained HTML, no server needed.
**File migration lifecycle:**
1. `gbrain files mirror <dir>` -- upload to cloud, keep local copies
2. `gbrain files redirect <dir>` -- replace local with `.redirect.yaml` (verifies remote first)
3. `gbrain files restore <dir>` -- download back from cloud
4. `gbrain files clean <dir> --yes` -- remove pointers (cloud is sole source)
**Skill:** `skills/publish/SKILL.md` tells the agent when to publish, defaults
(always encrypt), and sharing workflows (local file, cloud upload + signed URL,
static hosting).
### Skill Patterns (Latent Space)
### 2. `gbrain backlinks check/fix` -- enforce the Iron Law
All patterns from v0.8.1 are included plus:
- Skills reference actual `gbrain files` commands (not theoretical patterns)
- Maintain skill includes file storage health checks
- Setup skill includes storage configuration phase with migration guidance
- Voice recipe uses `gbrain files upload-raw` for call audio
- Filing rules document the `.redirect.yaml` format and upload commands
```bash
gbrain backlinks check --dir /path/to/brain # report missing back-links
gbrain backlinks fix --dir /path/to/brain # create them
gbrain backlinks fix --dir /path/to/brain --dry-run # preview
```
Scans all pages for entity mentions (links to people/ and companies/), checks
if those entity pages link back. Creates timeline entries for missing back-links.
### 3. `gbrain lint [--fix]` -- catch LLM slop
```bash
gbrain lint /path/to/brain # report issues
gbrain lint /path/to/brain --fix # auto-fix what's fixable
gbrain lint brain/people/jane.md # lint single file
```
Catches: LLM preambles ("Of course! Here is..."), wrapping code fences,
placeholder dates (YYYY-MM-DD, XX-XX), missing frontmatter, broken citations,
empty sections. `--fix` strips the fixable ones automatically.
### 4. `gbrain report` -- audit trail
```bash
gbrain report --type enrichment-sweep --title "Weekly Sweep" --content "..."
echo "report body" | gbrain report --type meeting-sync --title "Daily Sync"
```
Creates: `brain/reports/{type}/{YYYY-MM-DD-HHMM}.md` with frontmatter.
Used by the maintain skill for enrichment sweeps, backlink checks, lint runs.
### 5. `gbrain files upload-raw` -- smart file uploads
```bash
gbrain files upload-raw <file> --page <slug> --type <type>
gbrain files signed-url <storage-path>
```
Auto-routes by size: < 100 MB text stays in git, >= 100 MB or media goes to
cloud storage via TUS resumable upload (6 MB chunks with retry). Creates
`.redirect.yaml` pointers in the brain repo.
## Agent Instructions
### Step 1: Verify upgrade
```bash
cat VERSION # should show 0.9.0
gbrain --version # should show 0.9.0
gbrain doctor --json
```
### Step 2: Configure storage backend (optional but recommended)
### Step 2: Replace custom scripts with built-in commands
**This is the key step.** If your agent uses custom scripts for any of these
operations, swap them for the built-in gbrain commands:
| Old pattern (custom script) | New pattern (built-in) |
|----------------------------|----------------------|
| `node scripts/brain-publish.mjs <page>` | `gbrain publish <page> --password` |
| `node scripts/upload-raw-source.mjs <file> --page <slug>` | `gbrain files upload-raw <file> --page <slug>` |
| `node scripts/save-report.mjs --cron <type>` | `gbrain report --type <type> --content "..."` |
| `node scripts/validate-brain-links.mjs` | `gbrain files verify` |
| `node scripts/migrate-brain-files.mjs` | `gbrain files mirror + redirect` |
| Manual back-link checking | `gbrain backlinks check --dir <brain>` |
| Manual page quality review | `gbrain lint <brain> --fix` |
**For each replacement:**
1. Find references to the old script in your AGENTS.md, skills, or cron jobs
2. Replace with the gbrain command
3. The gbrain command uses your configured storage backend automatically
**Example: update AGENTS.md or skill files:**
```
# Old:
node /data/.openclaw/workspace/scripts/upload-raw-source.mjs <file> \
--page <slug> --type transcript
# New:
gbrain files upload-raw <file> --page <slug> --type transcript
```
```
# Old:
node /data/.openclaw/workspace/scripts/brain-publish.mjs <page> --password
# New:
gbrain publish <page> --password
```
```
# Old:
node /data/.openclaw/workspace/scripts/save-report.mjs \
--cron "enrichment-sweep" --title "Enrichment Sweep" --content "..."
# New:
gbrain report --type enrichment-sweep --title "Enrichment Sweep" --content "..."
```
### Step 3: Run the new tools on your brain
```bash
# Check back-link health
gbrain backlinks check --dir /path/to/brain
# If gaps found, fix them:
gbrain backlinks fix --dir /path/to/brain --dry-run # preview
gbrain backlinks fix --dir /path/to/brain # apply
# Lint for quality issues
gbrain lint /path/to/brain
# If fixable issues found:
gbrain lint /path/to/brain --fix --dry-run # preview
gbrain lint /path/to/brain --fix # apply
# Save a report of what you fixed
gbrain report --type migration \
--title "v0.9.0 Migration" \
--content "Ran backlinks fix (N gaps fixed) and lint --fix (M issues fixed)." \
--dir /path/to/brain
```
### Step 4: Configure storage backend (if not already done)
If your brain repo has binary files (images, PDFs, audio, video), configure
cloud storage to keep git lean:
```bash
# Option A: Supabase Storage (recommended if you already use Supabase)
# Get your service role key from: Supabase Dashboard > Project Settings > API
# Supabase Storage (recommended if you already use Supabase)
gbrain config set storage.backend supabase
gbrain config set storage.bucket brain-files
gbrain config set storage.projectUrl https://YOUR-PROJECT.supabase.co
gbrain config set storage.serviceRoleKey YOUR_SERVICE_ROLE_KEY
# Option B: S3-compatible (AWS, Cloudflare R2, MinIO)
# Or S3-compatible (AWS, Cloudflare R2, MinIO)
gbrain config set storage.backend s3
gbrain config set storage.bucket brain-files
gbrain config set storage.region us-east-1
gbrain config set storage.accessKeyId YOUR_KEY
gbrain config set storage.secretAccessKey YOUR_SECRET
# For R2/MinIO: also set storage.endpoint
```
Verify: `gbrain files upload test.txt --page test && gbrain files list test`
Then migrate existing binaries:
```bash
gbrain files status
gbrain files mirror <brain-dir>
gbrain files redirect <brain-dir>
```
### Step 3: Migrate existing binary files (if storage configured)
### Step 5: Update cron jobs
If you have cron jobs that call custom scripts, update them:
```bash
# See what you have
gbrain files status
# Old cron entry:
*/30 * * * * node /path/to/scripts/validate-brain-links.mjs
# If local binary files exist:
gbrain files mirror <brain-dir> --dry-run # Preview what would upload
gbrain files mirror <brain-dir> # Upload to cloud (keeps local)
gbrain files redirect <brain-dir> --dry-run # Preview what would redirect
gbrain files redirect <brain-dir> # Replace local with .redirect.yaml
# Verify
gbrain files status
gbrain files signed-url <any-storage-path> # Test URL generation
# New:
*/30 * * * * gbrain files verify
```
### Step 4: Upgrade legacy .redirect files (if any exist)
If you have `.redirect` files from v0.8.x, they continue to work (backward
compatible). To upgrade them to the richer `.redirect.yaml` format:
### Step 6: Add new maintenance crons (recommended)
```bash
# Find legacy redirects
find <brain-dir> -name "*.redirect" -not -name "*.redirect.yaml"
# Daily backlink check + auto-fix
0 3 * * * gbrain backlinks fix --dir /path/to/brain
# For each: restore then re-redirect (picks up new format)
gbrain files restore <dir>
gbrain files mirror <dir>
gbrain files redirect <dir>
# Weekly lint + auto-fix
0 4 * * 0 gbrain lint /path/to/brain --fix
```
### Step 5: Review skills (same as v0.8.1)
### Step 7: Review skill updates
- Check 5 recent pages for back-link gaps
- Check sources/ for misfiled pages
- If voice recipe is set up: apply unicode/PII fixes
- If x-to-brain is set up: add image OCR, stagger crons
The skill files have been updated to reference gbrain commands:
- `skills/ingest/SKILL.md` -- uses `gbrain files upload-raw` for raw sources
- `skills/maintain/SKILL.md` -- uses `gbrain backlinks`, `gbrain lint`, `gbrain report`
- `skills/publish/SKILL.md` -- NEW skill for `gbrain publish`
- `skills/enrich/SKILL.md` -- references `gbrain files upload-raw` for raw API data
- `skills/_brain-filing-rules.md` -- documents `.redirect.yaml` format and commands
### Step 6: Done
Read the updated skills to pick up the new patterns.
### Step 8: Done
```bash
mkdir -p ~/.gbrain/migrations
echo '{"version":"0.9.0","ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","status":"complete","storage_configured":'$(gbrain config get storage.backend 2>/dev/null && echo true || echo false)'}' >> ~/.gbrain/migrations/completed.jsonl
echo '{"version":"0.9.0","ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","status":"complete","tools_available":["publish","backlinks","lint","report","upload-raw","signed-url"],"scripts_replaced":true}' >> ~/.gbrain/migrations/completed.jsonl
```