* feat: battle-tested skill patterns from production deployment Backport production-learned brain-operations patterns: - Iron Law of Back-Linking (mandatory bidirectional linking) - Brain filing rules (file by primary subject, not format) - Enrichment protocol (7-step pipeline, 3-tier system, person/company templates) - Media ingest workflows (articles, videos, podcasts, PDFs, screenshots) - Citation requirements (mandatory [Source: ...] on every fact) - Test Before Bulk operating principle - Voice recipe: unicode crash fix, PII scrub, identity-first prompt, DIY STT+LLM+TTS - X-to-Brain recipe: image OCR, Filtered Stream, tweet rating rubric, cron stagger * chore: bump version and changelog (v0.8.1) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add _brain-filing-rules.md to CLAUDE.md key files * feat: smart file upload with TUS resumable and .redirect.yaml pointers - Supabase Storage auto-selects upload method by file size: < 100 MB standard POST, >= 100 MB TUS resumable (6 MB chunks + retry) - Signed URL generation for private bucket access (1-hour expiry) - New `upload-raw` command with size routing: small text stays in git, large/media files go to cloud with .redirect.yaml pointer - New `signed-url` command for generating access links - File resolver supports both .redirect.yaml (v0.9+) and .redirect (legacy) - Redirect format upgraded: 10 fields with full metadata - All migration commands (mirror, redirect, restore, clean) handle both formats * feat: skills reference actual gbrain file commands - Filing rules document upload-raw, signed-url, and .redirect.yaml format - Ingest skill uses gbrain files upload-raw for raw source preservation - Maintain skill adds file storage health checks - Setup skill adds storage configuration phase with migration guidance - Voice recipe uses upload-raw for call audio storage - Migration v0.9.0 with complete storage setup instructions * chore: bump version and changelog (v0.9.0) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: gbrain publish -- shareable HTML with password protection First code+skill pair: deterministic code does the work (strip private data, encrypt with AES-256-GCM, generate self-contained HTML), the skill tells the agent when and how to use it. 34 new tests. See: https://x.com/garrytan/status/2042925773300908103 * feat: backlinks check/fix, page lint, and report commands Three new deterministic tools (zero LLM calls): - gbrain backlinks check/fix -- scans brain for entity mentions without back-links, creates them. Enforces the Iron Law from the skills. - gbrain lint [--fix] -- catches LLM preambles, code fence wrapping, placeholder dates, missing frontmatter, broken citations, empty sections. --fix auto-strips fixable artifacts. - gbrain report --type <name> -- saves timestamped reports to brain/reports/{type}/YYYY-MM-DD-HHMM.md for audit trails. 33 new tests (409 total, 0 fail). * feat: v0.9.0 migration tells agents to swap scripts for built-in commands Migration file now: - Lists all 5 new deterministic commands with usage examples - Includes a script-to-command replacement table (old -> new) - Tells the agent to find custom script references in AGENTS.md, skills, and cron jobs and replace with gbrain commands - Adds recommended cron jobs for daily backlink fix + weekly lint - References the Thin Harness, Fat Skills thread * fix: CLI routing bugs found during DX review - Fixed subArgs reference error in handleCliOnly (used wrong variable name) - Renamed gbrain backlinks check/fix to gbrain check-backlinks to avoid conflict with existing backlinks operation (per-page incoming links) - Added TOOLS section to --help output showing publish, check-backlinks, lint, report - Added upload-raw and signed-url to FILES section in --help - Updated all docs/migration references to use check-backlinks * fix: security hardening from adversarial review - XSS: sanitize marked.parse() output (strip script/iframe/on* attrs) - Path traversal: validate report --type against [a-z0-9-] pattern - TUS: HEAD request before retry to get server's actual offset (TUS spec) - Pointer: upload-raw now includes pointer content in JSON output - Symlinks: use lstatSync in all walkers to prevent directory escape --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
232 lines
8.1 KiB
Markdown
232 lines
8.1 KiB
Markdown
---
|
|
version: 0.9.0
|
|
feature_pitch:
|
|
headline: "5 new deterministic tools, smart file uploads, production-grade skills"
|
|
description: "gbrain publish, backlinks, lint, report, and upload-raw. Code+skill pairs -- deterministic TypeScript does the work, skills tell the agent when to use it. Plus TUS resumable uploads, .redirect.yaml pointers, and battle-tested skill patterns."
|
|
tiers:
|
|
- name: "Core tools (everyone)"
|
|
description: "publish, backlinks, lint, report -- zero external deps, work immediately."
|
|
setup: "gbrain check-update && gbrain upgrade"
|
|
- name: "With Supabase Storage"
|
|
description: "upload-raw, signed-url, file migration lifecycle. Large files in cloud, git stays lean."
|
|
setup: "Configure storage backend, then gbrain files mirror + redirect."
|
|
---
|
|
|
|
# v0.9.0 Migration: Deterministic Tools + Smart File Storage
|
|
|
|
This is a major upgrade. GBrain now ships deterministic tools alongside skills --
|
|
code for data, LLMs for judgment. No database schema changes required.
|
|
|
|
## What's New: 5 Deterministic Commands
|
|
|
|
These commands run without LLM calls. They are the "code" half of the
|
|
[Thin Harness, Fat Skills](https://x.com/garrytan/status/2042925773300908103) pattern.
|
|
|
|
### 1. `gbrain publish` -- shareable HTML from brain pages
|
|
|
|
```bash
|
|
gbrain publish brain/people/jane-doe.md # local HTML
|
|
gbrain publish brain/people/jane-doe.md --password # auto-generated pw
|
|
gbrain publish brain/people/jane-doe.md --password "pw" # custom pw
|
|
gbrain publish brain/people/jane-doe.md --out share.html # custom output
|
|
```
|
|
|
|
Strips private data (frontmatter, citations, confirmations, brain links, timeline).
|
|
Optional AES-256-GCM encryption with client-side decryption. Dark/light mode,
|
|
mobile-optimized. Self-contained HTML, no server needed.
|
|
|
|
**Skill:** `skills/publish/SKILL.md` tells the agent when to publish, defaults
|
|
(always encrypt), and sharing workflows (local file, cloud upload + signed URL,
|
|
static hosting).
|
|
|
|
### 2. `gbrain check-backlinks check/fix` -- enforce the Iron Law
|
|
|
|
```bash
|
|
gbrain check-backlinks check --dir /path/to/brain # report missing back-links
|
|
gbrain check-backlinks fix --dir /path/to/brain # create them
|
|
gbrain check-backlinks fix --dir /path/to/brain --dry-run # preview
|
|
```
|
|
|
|
Scans all pages for entity mentions (links to people/ and companies/), checks
|
|
if those entity pages link back. Creates timeline entries for missing back-links.
|
|
|
|
### 3. `gbrain lint [--fix]` -- catch LLM slop
|
|
|
|
```bash
|
|
gbrain lint /path/to/brain # report issues
|
|
gbrain lint /path/to/brain --fix # auto-fix what's fixable
|
|
gbrain lint brain/people/jane.md # lint single file
|
|
```
|
|
|
|
Catches: LLM preambles ("Of course! Here is..."), wrapping code fences,
|
|
placeholder dates (YYYY-MM-DD, XX-XX), missing frontmatter, broken citations,
|
|
empty sections. `--fix` strips the fixable ones automatically.
|
|
|
|
### 4. `gbrain report` -- audit trail
|
|
|
|
```bash
|
|
gbrain report --type enrichment-sweep --title "Weekly Sweep" --content "..."
|
|
echo "report body" | gbrain report --type meeting-sync --title "Daily Sync"
|
|
```
|
|
|
|
Creates: `brain/reports/{type}/{YYYY-MM-DD-HHMM}.md` with frontmatter.
|
|
Used by the maintain skill for enrichment sweeps, backlink checks, lint runs.
|
|
|
|
### 5. `gbrain files upload-raw` -- smart file uploads
|
|
|
|
```bash
|
|
gbrain files upload-raw <file> --page <slug> --type <type>
|
|
gbrain files signed-url <storage-path>
|
|
```
|
|
|
|
Auto-routes by size: < 100 MB text stays in git, >= 100 MB or media goes to
|
|
cloud storage via TUS resumable upload (6 MB chunks with retry). Creates
|
|
`.redirect.yaml` pointers in the brain repo.
|
|
|
|
## Agent Instructions
|
|
|
|
### Step 1: Verify upgrade
|
|
|
|
```bash
|
|
gbrain --version # should show 0.9.0
|
|
gbrain doctor --json
|
|
```
|
|
|
|
### Step 2: Replace custom scripts with built-in commands
|
|
|
|
**This is the key step.** If your agent uses custom scripts for any of these
|
|
operations, swap them for the built-in gbrain commands:
|
|
|
|
| Old pattern (custom script) | New pattern (built-in) |
|
|
|----------------------------|----------------------|
|
|
| `node scripts/brain-publish.mjs <page>` | `gbrain publish <page> --password` |
|
|
| `node scripts/upload-raw-source.mjs <file> --page <slug>` | `gbrain files upload-raw <file> --page <slug>` |
|
|
| `node scripts/save-report.mjs --cron <type>` | `gbrain report --type <type> --content "..."` |
|
|
| `node scripts/validate-brain-links.mjs` | `gbrain files verify` |
|
|
| `node scripts/migrate-brain-files.mjs` | `gbrain files mirror + redirect` |
|
|
| Manual back-link checking | `gbrain check-backlinks check --dir <brain>` |
|
|
| Manual page quality review | `gbrain lint <brain> --fix` |
|
|
|
|
**For each replacement:**
|
|
1. Find references to the old script in your AGENTS.md, skills, or cron jobs
|
|
2. Replace with the gbrain command
|
|
3. The gbrain command uses your configured storage backend automatically
|
|
|
|
**Example: update AGENTS.md or skill files:**
|
|
```
|
|
# Old:
|
|
node /data/.openclaw/workspace/scripts/upload-raw-source.mjs <file> \
|
|
--page <slug> --type transcript
|
|
|
|
# New:
|
|
gbrain files upload-raw <file> --page <slug> --type transcript
|
|
```
|
|
|
|
```
|
|
# Old:
|
|
node /data/.openclaw/workspace/scripts/brain-publish.mjs <page> --password
|
|
|
|
# New:
|
|
gbrain publish <page> --password
|
|
```
|
|
|
|
```
|
|
# Old:
|
|
node /data/.openclaw/workspace/scripts/save-report.mjs \
|
|
--cron "enrichment-sweep" --title "Enrichment Sweep" --content "..."
|
|
|
|
# New:
|
|
gbrain report --type enrichment-sweep --title "Enrichment Sweep" --content "..."
|
|
```
|
|
|
|
### Step 3: Run the new tools on your brain
|
|
|
|
```bash
|
|
# Check back-link health
|
|
gbrain check-backlinks check --dir /path/to/brain
|
|
|
|
# If gaps found, fix them:
|
|
gbrain check-backlinks fix --dir /path/to/brain --dry-run # preview
|
|
gbrain check-backlinks fix --dir /path/to/brain # apply
|
|
|
|
# Lint for quality issues
|
|
gbrain lint /path/to/brain
|
|
|
|
# If fixable issues found:
|
|
gbrain lint /path/to/brain --fix --dry-run # preview
|
|
gbrain lint /path/to/brain --fix # apply
|
|
|
|
# Save a report of what you fixed
|
|
gbrain report --type migration \
|
|
--title "v0.9.0 Migration" \
|
|
--content "Ran backlinks fix (N gaps fixed) and lint --fix (M issues fixed)." \
|
|
--dir /path/to/brain
|
|
```
|
|
|
|
### Step 4: Configure storage backend (if not already done)
|
|
|
|
If your brain repo has binary files (images, PDFs, audio, video), configure
|
|
cloud storage to keep git lean:
|
|
|
|
```bash
|
|
# Supabase Storage (recommended if you already use Supabase)
|
|
gbrain config set storage.backend supabase
|
|
gbrain config set storage.bucket brain-files
|
|
gbrain config set storage.projectUrl https://YOUR-PROJECT.supabase.co
|
|
gbrain config set storage.serviceRoleKey YOUR_SERVICE_ROLE_KEY
|
|
|
|
# Or S3-compatible (AWS, Cloudflare R2, MinIO)
|
|
gbrain config set storage.backend s3
|
|
gbrain config set storage.bucket brain-files
|
|
gbrain config set storage.region us-east-1
|
|
gbrain config set storage.accessKeyId YOUR_KEY
|
|
gbrain config set storage.secretAccessKey YOUR_SECRET
|
|
```
|
|
|
|
Then migrate existing binaries:
|
|
```bash
|
|
gbrain files status
|
|
gbrain files mirror <brain-dir>
|
|
gbrain files redirect <brain-dir>
|
|
```
|
|
|
|
### Step 5: Update cron jobs
|
|
|
|
If you have cron jobs that call custom scripts, update them:
|
|
|
|
```bash
|
|
# Old cron entry:
|
|
*/30 * * * * node /path/to/scripts/validate-brain-links.mjs
|
|
|
|
# New:
|
|
*/30 * * * * gbrain files verify
|
|
```
|
|
|
|
### Step 6: Add new maintenance crons (recommended)
|
|
|
|
```bash
|
|
# Daily backlink check + auto-fix
|
|
0 3 * * * gbrain check-backlinks fix --dir /path/to/brain
|
|
|
|
# Weekly lint + auto-fix
|
|
0 4 * * 0 gbrain lint /path/to/brain --fix
|
|
```
|
|
|
|
### Step 7: Review skill updates
|
|
|
|
The skill files have been updated to reference gbrain commands:
|
|
- `skills/ingest/SKILL.md` -- uses `gbrain files upload-raw` for raw sources
|
|
- `skills/maintain/SKILL.md` -- uses `gbrain check-backlinks`, `gbrain lint`, `gbrain report`
|
|
- `skills/publish/SKILL.md` -- NEW skill for `gbrain publish`
|
|
- `skills/enrich/SKILL.md` -- references `gbrain files upload-raw` for raw API data
|
|
- `skills/_brain-filing-rules.md` -- documents `.redirect.yaml` format and commands
|
|
|
|
Read the updated skills to pick up the new patterns.
|
|
|
|
### Step 8: Done
|
|
|
|
```bash
|
|
mkdir -p ~/.gbrain/migrations
|
|
echo '{"version":"0.9.0","ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","status":"complete","tools_available":["publish","backlinks","lint","report","upload-raw","signed-url"],"scripts_replaced":true}' >> ~/.gbrain/migrations/completed.jsonl
|
|
```
|