feat: v0.9.0 -- smart file storage, publish, production-grade skills (#62)

* feat: battle-tested skill patterns from production deployment

Backport production-learned brain-operations patterns:
- Iron Law of Back-Linking (mandatory bidirectional linking)
- Brain filing rules (file by primary subject, not format)
- Enrichment protocol (7-step pipeline, 3-tier system, person/company templates)
- Media ingest workflows (articles, videos, podcasts, PDFs, screenshots)
- Citation requirements (mandatory [Source: ...] on every fact)
- Test Before Bulk operating principle
- Voice recipe: unicode crash fix, PII scrub, identity-first prompt, DIY STT+LLM+TTS
- X-to-Brain recipe: image OCR, Filtered Stream, tweet rating rubric, cron stagger

* chore: bump version and changelog (v0.8.1)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add _brain-filing-rules.md to CLAUDE.md key files

* feat: smart file upload with TUS resumable and .redirect.yaml pointers

- Supabase Storage auto-selects upload method by file size:
  < 100 MB standard POST, >= 100 MB TUS resumable (6 MB chunks + retry)
- Signed URL generation for private bucket access (1-hour expiry)
- New `upload-raw` command with size routing: small text stays in git,
  large/media files go to cloud with .redirect.yaml pointer
- New `signed-url` command for generating access links
- File resolver supports both .redirect.yaml (v0.9+) and .redirect (legacy)
- Redirect format upgraded: 10 fields with full metadata
- All migration commands (mirror, redirect, restore, clean) handle both formats

* feat: skills reference actual gbrain file commands

- Filing rules document upload-raw, signed-url, and .redirect.yaml format
- Ingest skill uses gbrain files upload-raw for raw source preservation
- Maintain skill adds file storage health checks
- Setup skill adds storage configuration phase with migration guidance
- Voice recipe uses upload-raw for call audio storage
- Migration v0.9.0 with complete storage setup instructions

* chore: bump version and changelog (v0.9.0)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: gbrain publish -- shareable HTML with password protection

First code+skill pair: deterministic code does the work (strip private data,
encrypt with AES-256-GCM, generate self-contained HTML), the skill tells the
agent when and how to use it. 34 new tests.

See: https://x.com/garrytan/status/2042925773300908103

* feat: backlinks check/fix, page lint, and report commands

Three new deterministic tools (zero LLM calls):

- gbrain backlinks check/fix -- scans brain for entity mentions without
  back-links, creates them. Enforces the Iron Law from the skills.
- gbrain lint [--fix] -- catches LLM preambles, code fence wrapping,
  placeholder dates, missing frontmatter, broken citations, empty sections.
  --fix auto-strips fixable artifacts.
- gbrain report --type <name> -- saves timestamped reports to
  brain/reports/{type}/YYYY-MM-DD-HHMM.md for audit trails.

33 new tests (409 total, 0 fail).

* feat: v0.9.0 migration tells agents to swap scripts for built-in commands

Migration file now:
- Lists all 5 new deterministic commands with usage examples
- Includes a script-to-command replacement table (old -> new)
- Tells the agent to find custom script references in AGENTS.md,
  skills, and cron jobs and replace with gbrain commands
- Adds recommended cron jobs for daily backlink fix + weekly lint
- References the Thin Harness, Fat Skills thread

* fix: CLI routing bugs found during DX review

- Fixed subArgs reference error in handleCliOnly (used wrong variable name)
- Renamed gbrain backlinks check/fix to gbrain check-backlinks to avoid
  conflict with existing backlinks operation (per-page incoming links)
- Added TOOLS section to --help output showing publish, check-backlinks,
  lint, report
- Added upload-raw and signed-url to FILES section in --help
- Updated all docs/migration references to use check-backlinks

* fix: security hardening from adversarial review

- XSS: sanitize marked.parse() output (strip script/iframe/on* attrs)
- Path traversal: validate report --type against [a-z0-9-] pattern
- TUS: HEAD request before retry to get server's actual offset (TUS spec)
- Pointer: upload-raw now includes pointer content in JSON output
- Symlinks: use lstatSync in all walkers to prevent directory escape

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-04-11 21:46:07 -10:00
committed by GitHub
parent 91ced664b6
commit baf3517868
30 changed files with 3239 additions and 92 deletions

View File

@@ -2,6 +2,91 @@
All notable changes to GBrain will be documented in this file.
## [0.9.0] - 2026-04-11
### Added
- **Large files don't bloat your git repo anymore.** `gbrain files upload-raw`
auto-routes by size: text and PDFs under 100 MB stay in git, everything larger
(or any media file) goes to Supabase Storage with a `.redirect.yaml` pointer
left in the repo. Files over 100 MB use TUS resumable upload (6 MB chunks with
retry and backoff) so a flaky connection doesn't lose a 2 GB video upload.
`gbrain files signed-url` generates 1-hour access links for private buckets.
- **The full file migration lifecycle works end to end.** `mirror` uploads to
cloud and keeps local copies. `redirect` replaces local files with
`.redirect.yaml` pointers (verifies remote exists first, won't delete data).
`restore` downloads back from cloud. `clean` removes pointers when you're sure.
`status` shows where you are. Three states, zero data loss risk.
- **Your brain now enforces its own graph integrity.** The Iron Law of Back-Linking
is mandatory across all skills. Every mention of a person or company creates
a bidirectional link. This transforms your brain from a flat file store into a
traversable knowledge graph.
- **Filing rules prevent the #1 brain mistake.** New `skills/_brain-filing-rules.md`
stops the most common error: dumping everything into `sources/`. File by primary
subject, not format. Includes notability gate and citation requirements.
- **Enrichment protocol that actually works.** Rewritten from a 46-line API list to
a 7-step pipeline with 3-tier system, person/company page templates, pluggable
data sources, validation rules, and bulk enrichment safety.
- **Ingest handles everything.** Articles, videos, podcasts, PDFs, screenshots,
meeting transcripts, social media. Each with a workflow that uses real gbrain
commands (`upload-raw`, `signed-url`) instead of theoretical patterns.
- **Citation requirements across all skills.** Every fact needs inline
`[Source: ...]` citations. Three formats, source precedence hierarchy.
- **Maintain skill catches what you missed.** Back-link enforcement, citation audit,
filing violations, file storage health checks, benchmark testing.
- **Voice calls don't crash on em dashes anymore.** Unicode sanitization for Twilio
WebSocket, PII scrub, identity-first prompt, DIY STT+LLM+TTS pipeline option,
Smart VAD default, auto-upload call audio via `gbrain files upload-raw`.
- **X-to-Brain gets eyes.** Image OCR, Filtered Stream real-time monitoring,
6-dimension tweet rating rubric, outbound tweet monitoring, cron staggering.
- **Share brain pages without exposing the brain.** `gbrain publish` generates
beautiful, self-contained HTML from any brain page. Strips private data
(frontmatter, citations, confirmations, brain links, timeline) automatically.
Optional AES-256-GCM password gate with client-side decryption, no server
needed. Dark/light mode, mobile-optimized typography. This is the first
code+skill pair: deterministic code does the work, the skill tells the agent
when and how. See the [Thin Harness, Fat Skills](https://x.com/garrytan/status/2042925773300908103)
thread for the architecture philosophy.
### Changed
- **Supabase Storage** now auto-selects upload method by file size: standard POST
for < 100 MB, TUS resumable for >= 100 MB. Signed URL generation for private
bucket access (1-hour expiry).
- **File resolver** supports both `.redirect.yaml` (v0.9+) and legacy `.redirect`
(v0.8) formats for backward compatibility.
- **Redirect format** upgraded from `.redirect` (5 fields) to `.redirect.yaml`
(10 fields: target, bucket, storage_path, size, size_human, hash, mime,
uploaded, source_url, type).
- **All skills** updated to reference actual `gbrain files` commands instead of
theoretical patterns.
- **Back-link enforcer closes the loop.** `gbrain check-backlinks check` scans your
brain for entity mentions without back-links. `gbrain check-backlinks fix` creates
them. The Iron Law of Back-Linking is in every skill, now the code enforces it.
- **Page linter catches LLM slop.** `gbrain lint` flags "Of course! Here is..."
preambles, wrapping code fences, placeholder dates, missing frontmatter, broken
citations, and empty sections. `gbrain lint --fix` auto-strips the fixable ones.
Every brain that uses AI for ingestion accumulates this. Now it's one command.
- **Audit trail for everything.** `gbrain report --type enrichment-sweep` saves
timestamped reports to `brain/reports/{type}/YYYY-MM-DD-HHMM.md`. The maintain
skill references this for enrichment sweeps, meeting syncs, and maintenance runs.
- **Publish skill** added to manifest (8th skill). First code+skill pair.
- Skills version bumped to 0.9.0.
- 67 new unit tests across publish, backlinks, lint, and report. Total: 409 pass.
## [0.8.0] - 2026-04-11
### Added