Post-ship requirements section: document-release is NOT optional. Lists every
file that must be checked on every ship. A ship without updated docs is incomplete.
Also adds remote MCP server files to Key files section.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- fix(file_upload): call storage.upload() in all 3 paths (operation, CLI upload, CLI sync) with rollback semantics (#22 Bug #9)
- fix(import): use atomic index counter for parallel queue instead of array.shift() race, preserve checkpoint on errors (#22 Bug #3)
- fix(s3): replace unsigned fetch with @aws-sdk/client-s3 for proper SigV4 auth, supports R2/MinIO via forcePathStyle (#22 Bug #10)
- fix(redirect): verify remote file exists before deleting local copy, skip files not found in storage (#22 Bug #5)
- deps: add @aws-sdk/client-s3
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
initSchema() previously read schema.sql from disk at runtime via readFileSync,
which broke in compiled Bun binaries and Deno Edge Functions. Now uses a
generated schema-embedded.ts constant (run `bun run build:schema` to regenerate).
- Removes fs and path imports from postgres-engine.ts and db.ts
- Adds scripts/build-schema.sh for one-source-of-truth generation
- Adds build:schema npm script
Fixes Issue #22 Bug #6.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- fix(mcp): use ListToolsRequestSchema/CallToolRequestSchema instead of string literals (Issue #9, PR #25)
- fix(mcp): handleToolCall reads dry_run from params instead of hardcoding false (#22 Bug #11)
- fix(search): keyword search returns best chunk per page via DISTINCT ON, not all chunks (#22 Bug #8)
- fix(search): dedup layer 1 keeps top 3 chunks per page instead of collapsing to 1 (#22 Bug #12)
- fix(engine): transaction uses scoped engine via Object.create, no shared state mutation (#22 Bug #2)
- fix(engine): upsertChunks uses UPSERT instead of DELETE+INSERT, preserves existing embeddings (#22 Bug #1)
- fix(slugs): validateSlug normalizes to lowercase, pathToSlug lowercases consistently (#22 Bug #4)
- schema: add unique index on content_chunks(page_id, chunk_index) for UPSERT support
- schema: add access_tokens and mcp_request_log tables via migration
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add SKILLPACK Section 18 — Live Sync (MUST ADD)
Contract-first guide for keeping the vector DB in sync with the brain
repo. Documents the pooler prerequisite (Session mode required for
transactions), sync + embed primitives, four example approaches (cron,
--watch, webhook, git hook), isSyncable exclusions, silent skip warning,
and OpenClaw/Hermes cron registration examples.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add GBRAIN_VERIFY.md installation verification runbook
Six-check runbook: schema (doctor), skillpack loaded, auto-update,
live sync (coverage check + embed check + end-to-end push-and-search
test), embedding coverage, brain-first lookup protocol. Emphasizes
"sync ran" != "sync worked" — the real test is searching for corrected
text after a push.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add setup Phases H (Live Sync) and I (Verification)
Phase H: MUST ADD live sync setup — pooler prerequisite check, automatic
sync configuration (agent picks approach), sync+embed chaining, coverage
verification. Phase I: run GBRAIN_VERIFY.md end-to-end before declaring
setup complete.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add install steps 8-9 (live sync + verification)
Step 8: set up automatic sync with SKILLPACK Section 18 reference.
Step 9: run GBRAIN_VERIFY.md runbook. Add GBRAIN_VERIFY.md to docs
section.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add API key loading instructions to CLAUDE.md
Source ~/.zshrc before running Tier 2 tests so OPENAI_API_KEY and
ANTHROPIC_API_KEY are available. Without this, embedding and skills
tests skip silently.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: bump version to v0.5.0
Live sync, verification runbook, API key loading instructions.
Version markers updated in SKILLPACK and RECOMMENDED_SCHEMA.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add anti-hand-roll rule to skill routing in CLAUDE.md
Explicitly prohibit manually running git commit + push + gh pr create
when /ship is available. /ship handles VERSION, CHANGELOG,
document-release, reviews, and coverage audit. Hand-rolling skips
all of these. Added "commit and ship" / "push and ship" variants
to the ship routing rule.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: changelog voice rule + rewrite 0.5.0 changelog to sell the upgrade
CLAUDE.md: add changelog voice guidance — lead with benefits, not
implementation details. Make users want to upgrade.
CHANGELOG: rewrite 0.5.0 entries from dry feature descriptions to
capability-focused bullets ("your brain never falls behind" not
"SKILLPACK Section 18 added").
SKILLPACK Section 17: update the auto-update message template to
instruct agents to sell the upgrade, not just summarize the diff.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add v0.5.0 migration directive for live sync + verification
Agents upgrading from v0.4.x will automatically: check their pooler
connection string, set up automatic sync, and run the verification
runbook. Without this migration file, upgrading agents would learn
about live sync (by re-reading Section 18) but wouldn't set it up.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: sharpen migration file guidance in CLAUDE.md
Replace vague "requires agent action" with concrete trigger list:
new setup steps existing users don't have, MUST ADD skillpack sections,
schema changes, deprecated commands, new verification steps, new crons.
Add the key test: "if an existing user upgrades and does nothing else,
will their brain work worse?"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: make Section 17 upgrade flow work for direct user requests
Section 17 was structured as a cron-initiated flow only. An agent
handling "upgrade gbrain" might just run the command and stop, missing
the post-upgrade steps where the value is (re-read skills, run
migrations, schema sync). Added explicit entry point for direct
upgrade requests. Made Steps 2-4 more concrete about where to find
files and why migrations can't be skipped.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add E2E sync tests — git-to-DB pipeline (11 tests)
Tests the full sync lifecycle against real Postgres+pgvector:
- First sync imports all pages from a git repo
- Second sync with no changes returns up_to_date
- Incremental sync picks up new files (add → commit → sync → verify)
- Incremental sync picks up modifications — THE CRITICAL TEST:
corrected text appears in DB and keyword search after sync
- Incremental sync handles deletes
- Non-syncable files are excluded (README, .raw/, ops/)
- Sync state (last_commit, last_run) persisted to config
- Sync logged to ingest_log
- --full reimports everything
- --dry-run shows changes without applying
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: strengthen CLAUDE.md to always run ALL test tiers
Replace passive "source zshrc" suggestion with ALWAYS directive.
Explicitly state that "run all tests" means ALL tiers including
Tier 2 with API keys. Do not skip Tier 2 just because keys need
loading.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: Tier 2 E2E tests — correct openclaw CLI invocation
The tests used `openclaw -p` which doesn't exist. The correct command
is `openclaw agent --local --agent <id> --message <prompt>`. Also fixed
JSON output parsing (structured JSON goes to stderr, not stdout — use
non-JSON mode instead). Fixed ingest test to assert on agent response
text rather than test DB state (the agent writes to its own configured
DB, not the ephemeral test DB).
82 tests pass, 0 fail, 0 skip across all 5 E2E files.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* security: pin GitHub Actions to commit SHAs, add gitleaks CI
- Pin all 5 actions (checkout, setup-bun, upload-artifact, download-artifact,
action-gh-release) to commit SHAs across 3 workflow files
- Add permissions: contents: read to test.yml and e2e.yml
- Add gitleaks secret scanning job to test.yml
- Pin openclaw install to v2026.4.9 in e2e.yml
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* security: add .gitleaks.toml config
Allowlists test fixtures, example env files, and skill documentation
to prevent false positives from the gitleaks CI step.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add GitHub Actions SHA maintenance rule to CLAUDE.md
Instructs /ship and /review to check for stale SHA pins and update
them, keeping action versions fresh without manual effort.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add S3 Sig V4 TODO from CSO audit
Deferred from security audit. S3 storage backend accepts credentials
but sends unsigned requests. Implement when S3 becomes a real
deployment path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.4.2)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OpenClaw ships DREAMS.md by default. Hermes users get a cron job
recipe with session_search + gbrain + memory consolidation, plus
Honcho for dialectic reasoning.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GBrain install instructions and skills work with both OpenClaw and
Hermes Agent. First mention in each file says OpenClaw/Hermes,
subsequent references say OpenClaw.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New flow: Get Connected > Direct Connection String > Session Pooler >
copy Shared Pooler. The old gear icon > Project Settings path is stale.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
First person origin story, Postgres is optional, dream cycle mention,
and condensed check-update install step.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Documents DREAMS.md, the nightly cron that scans conversations,
enriches thin entities, fixes broken citations, and consolidates
memory.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add `gbrain check-update` command for auto-update notifications
Deterministic collector that checks GitHub Releases for new versions,
compares semver (minor+ only, skips patches), and fetches changelog diffs.
Exports `detectInstallMethod()` from upgrade.ts for reuse. Includes 15
unit tests covering version comparison, CLI wiring, and error handling.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add E2E upgrade tests against real GitHub API
Exercises check-update CLI end-to-end: valid JSON output, human-readable
mode, help text, graceful no-releases handling, and version comparison
wiring. Skips gracefully when network is unavailable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add SKILLPACK Section 17 — auto-update notifications
Full agent playbook for the update lifecycle: check, notify, consent,
upgrade, skills refresh, schema sync, report. Includes standalone
self-update for skillpack-only users via version markers and raw
GitHub URL fetching. Adds version markers to both SKILLPACK and
RECOMMENDED_SCHEMA headers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add auto-update step 7 to install paste, setup Phase G, migrations dir
Adds step 7 to the OpenClaw install paste (default-on update checks).
Setup skill gets Phase G (conditional offer for manual installs) and
schema state tracking via ~/.gbrain/update-state.json. Creates
skills/migrations/ directory for version-specific upgrade directives.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: update CLAUDE.md with E2E test DB lifecycle, migration conventions
Adds E2E test DB lifecycle instructions (spin up, run, tear down).
Documents version migration convention (skills/migrations/v[version].md)
and schema state tracking (~/.gbrain/update-state.json). Updates test
file counts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: broken semver comparison in extractChangelogBetween
The version range check compared minor versions without guarding on
major being equal, causing incorrect changelog entries to be captured
(e.g., v0.5.0 would match when upgrading from v1.2.0). Extracted
semverGt/semverLte helpers for correct comparisons. Added 5 tests
for extractChangelogBetween covering cross-major, same-version, and
malformed input cases.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.4.1)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pattern for when LLMs keep failing at mechanical tasks despite prompt fixes.
Real example: email Gmail links dropped 5x, fixed by moving URL generation to
a deterministic Node.js collector script that feeds pre-formatted data to the LLM.
Architecture: deterministic pipeline → structured data → LLM analysis layer.
Same pattern as x-collector (Twitter data) — generalized to email, calendar,
Slack, GitHub, and any recurring data pull.
Co-authored-by: root <root@localhost>
OpenClaw setup now instructs agents to read the SKILLPACK and update
all skills with production agent patterns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bug report template (includes gbrain doctor --json field) and
feature request template.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New "GBrain without OpenClaw" section: standalone CLI, MCP server
config (Claude Code + Cursor), TypeScript library with examples,
and skill file loading table
- New "File storage and migration" section: three-stage lifecycle
(mirror/redirect/clean), all 10 file subcommands, storage backends
- SKILLPACK promoted throughout: bold callout in "Production Agent"
section, bold link in Docs section
- Removed duplicate "Using as a library" and "MCP server" sections
(now covered in the unified non-OpenClaw guide)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- gbrain init success message now prints the skillpack path
- Setup skill adds Phase E: load the production agent guide
- Agents are instructed to read and inject key SKILLPACK patterns
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
gbrain get with no args now shows "Usage: gbrain get <slug>" instead of
leaking a raw Postgres driver error (UNDEFINED_VALUE).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: widen validateSlug to accept any filename characters
Git is the system of record. Slugs are lowercased repo-relative paths.
The restrictive regex rejected spaces, parens, and special chars, blocking
5,861 Apple Notes files from importing. Now only rejects empty slugs,
path traversal (..), and leading slash.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: enable RLS on all tables with BYPASSRLS safety check
Without RLS, the Supabase anon key gives full read access to the DB.
Enable RLS on all 10 tables with no policies — the postgres role
(used by gbrain via pooler) has BYPASSRLS and is unaffected. Only
enables if the current role actually has BYPASSRLS privilege to
avoid locking ourselves out on non-Supabase setups.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: import resilience — 5MB limit, error suppression, structured progress
Raise MAX_FILE_SIZE from 1MB to 5MB for Apple Notes with attachments.
Track error patterns and suppress after 5 identical errors to prevent
5,861 identical warnings from killing the agent process. Replace \r
progress bar with structured log lines (rate, ETA) for agent parsing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: init detects IPv6-only Supabase URLs, adds pgvector check
Detect db.*.supabase.co direct URLs and warn about IPv6 failure.
On ECONNREFUSED/ETIMEDOUT to Supabase, suggest the Session pooler
connection string with exact dashboard click path. Check for pgvector
extension after connecting and fail with clear instructions if missing.
Update wizard hints to show pooler URL format.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add pre-ship requirement for E2E tests
E2E tests against real Postgres+pgvector must pass before /ship or
/review. Adds the requirement to CLAUDE.md so all agents enforce it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: parallel import with per-worker engine instances
Refactor PostgresEngine to support instance-level DB connections instead
of only the module-global singleton. Each worker gets its own connection
with poolSize:2 (vs 10 for the main engine), so 8 workers = 16 connections.
Add --workers N flag to gbrain import. Workers pull from a shared queue
and use independent engine instances — no transaction context corruption.
The bottleneck is network round-trips to Supabase (one per page upsert).
Parallel workers cut import time proportionally.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: automatic schema migration runner
Migrations are embedded as string constants in migrate.ts (survives
Bun --compile). Each migration runs in a transaction for clean rollback
on failure. Runs automatically on initSchema() — no manual step needed
when a user updates the gbrain binary against an older DB.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: pluggable storage backend (S3 + Supabase Storage + local)
Add StorageBackend interface with three implementations:
- S3Storage: works with AWS S3, Cloudflare R2, MinIO (any S3-compatible)
- SupabaseStorage: uses Supabase Storage REST API with service role key
- LocalStorage: filesystem-based, for testing
Add file-resolver.ts with fallback chain: local file → .redirect
breadcrumb → .supabase marker → storage backend. Supports the
three-stage migration (mirror → redirect → clean).
Add yaml-lite.ts for parsing marker and breadcrumb files without
adding a YAML dependency.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: gbrain doctor command — health checks with --json output
Checks: connection, pgvector extension, RLS on all tables, schema
version, embedding coverage. Outputs structured JSON with --json flag
for agent parsing. Exit code 0 if healthy, 1 if issues found.
Agents should run gbrain doctor --json when any command fails.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: rewrite setup skill + README for agent-first DX
Setup skill: add Why Supabase, step-by-step project creation, explicit
agent instructions (nohup for large imports, doctor on failure, don't
ask for anon key), available init flags, file migration offer after
first import. Remove ClawHub references.
README: simplify to single OpenClaw install path, remove ClawHub, fix
squatted npm name to github:garrytan/gbrain, add Supabase settings
note about Session pooler.
Add Apple Notes test fixtures with spaces and parens in filenames for
E2E testing of the slug fix.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add RLS verification, schema health, and nohup hints to maintain skill
Maintenance skill now checks RLS status and schema version as part of
periodic health checks. Adds nohup pattern for large embedding refreshes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: import resume checkpoint + Supabase smart URL parsing
Import resume: saves checkpoint every 100 files to ~/.gbrain/import-checkpoint.json.
On restart with same directory and file count, skips already-processed files.
Use --fresh to ignore checkpoint and start over. Cleared on successful completion.
Supabase admin: extractProjectRef() parses any Supabase URL format (dashboard,
direct, pooler, project URL) to extract the project ref. discoverPoolerUrl()
uses the Management API to find the correct pooler connection string (including
the exact region prefix). checkRls() verifies RLS status via the API.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add 56 unit tests for all new code
8 new test files covering every feature added in this branch:
- slug-validation.test.ts: spaces, parens, unicode, path traversal (10 tests)
- yaml-lite.test.ts: parse + stringify, marker/redirect formats (9 tests)
- supabase-admin.test.ts: extractProjectRef for 4 URL formats (7 tests)
- migrate.test.ts: version export, runMigrations callable (2 tests)
- storage.test.ts: LocalStorage CRUD + createStorage factory (14 tests)
- file-resolver.test.ts: fallback chain, redirect, marker parsing (6 tests)
- import-resume.test.ts: checkpoint save/load/resume/fresh (6 tests)
- doctor.test.ts: module export, CLI registration (3 tests)
Total: 184 pass, 0 fail (up from 128).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: bulk chunk INSERT + E2E tests for all new features
Bulk INSERT: upsertChunks now builds a multi-row VALUES query instead
of inserting chunks one-by-one. Reduces DB round-trips by ~50x per page.
E2E tests added to mechanical.test.ts:
- Slug with special chars: import Apple Notes fixtures with spaces/parens,
verify search finds them, verify idempotency
- RLS verification: check pg_tables.rowsecurity on all tables, verify
current user has BYPASSRLS
- Doctor command: verify exit 0 on healthy DB, --json produces valid JSON
with check structure
- Parallel import: --workers 2 produces same page count as sequential
Unit tests added:
- setup-branching.test.ts: IPv6 detection, defaultWorkers auto-tuning,
smart URL parsing across all Supabase URL formats
Fixtures added:
- large/big-file.md (2.1MB) for testing raised file size limit
- apple-notes/ fixtures already existed
Total: 200 pass, 0 fail (up from 184).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: --json on init/import, file migration CLI, lifecycle tests
--json flag: init and import now support --json for structured output.
Agents get parseable JSON instead of human-readable text.
File migration CLI: implement mirror, unmirror, redirect, restore,
clean, and status subcommands for the three-stage file migration
lifecycle (local → mirrored → redirected → cloud-only).
File migration tests: full lifecycle test covering every transition
in the state machine (LOCAL → MIRROR → UNMIRROR → REDIRECT → RESTORE
→ CLEAN), including edge cases and file resolver at each stage.
Bulk chunk INSERT: upsertChunks now builds multi-row parameterized
VALUES query, reducing round-trips per page from ~50 to 1.
Total: 207 pass, 0 fail.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: thorough E2E tests for parallel import concurrency
Replace the weak single-comparison parallel import test with 7 tests:
- Sequential baseline: capture page count, chunk count, and all slugs
- --workers 2: verify page count matches sequential
- Chunk count matches (no duplicates from concurrent writes)
- Page slugs match exactly
- No duplicate pages (SQL GROUP BY HAVING count > 1)
- No duplicate chunks (SQL GROUP BY page_id, chunk_index)
- --workers 4: also works correctly
- Re-import with workers is idempotent
These tests catch the exact bug Codex found (db.ts singleton causing
concurrent transaction corruption) by verifying data integrity after
parallel writes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add batch embedding queue as P1 TODO
Deferred during eng review (per-worker embedding is good enough for now).
Revisit after profiling real imports to confirm embedding is the bottleneck.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: E2E test failures — fixture counts, arg parsing, doctor exit code
Fix fixture count assertions: 13 → 16 pages (added apple-notes + large file),
companies 2 → 3 (ohmygreen), concepts 3 → 5 (notes, big-file).
Fix --workers arg parsing: the worker count value (e.g. "2") was being
picked up as the directory arg. Skip flag values when finding the dir.
Fix doctor exit code: warnings (like missing embeddings) should exit 0,
only actual failures exit 1. E2E tests import with --no-embed, so
embeddings are always WARN.
Fix E2E CLI tests: add initCli() before doctor and parallel import
tests so ~/.gbrain/config.json exists for the subprocess.
All E2E tests pass: 63 pass, 0 fail.
All unit tests pass: 207 pass, 0 fail.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: update project documentation for v0.4.0
New CHANGELOG entry for all post-0.3.0 features (doctor, storage backends,
parallel import, resume checkpoints, RLS, schema migrations, --json output).
Version bumped 0.3.0 → 0.4.0 across all manifests.
CLAUDE.md: test count 9→19, skill count 8→7, added key files.
CONTRIBUTING.md: fixture count 13→16, added missing source files.
README.md: added gbrain doctor to commands, fixed stale welcome PRs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add GBRAIN_SKILLPACK.md reference architecture
Production agent patterns from a real deployment with 14,700+ brain files.
Covers: entity detection on every message, brain-first lookup protocol,
7-step enrichment pipeline with tiered API spend, compiled truth + timeline,
source attribution with mandatory citations, meeting ingestion with entity
propagation, cron schedule with quiet hours and travel-aware timezone,
YouTube/media ingestion via Diarize.io, integration guides for ClawVisor,
Circleback webhooks, and Quo/OpenPhone SMS. Opens with the Vannevar Bush
memex framing and the originals folder for capturing intellectual capital.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: rewrite README opener with memex pitch and production architecture
Replace code-first opener with mimetic-desire pitch: Vannevar Bush memex
tagline, production brain numbers (10K+ files, 3K+ people, 13 years of
calendar), "ask it anything" examples, compounding thesis.
New sections: The Compounding Thesis (read-write loop), Architecture
(three-column diagram), What a Production Agent Looks Like (SKILLPACK
reference), How gbrain fits with OpenClaw (three-layer complement).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: update skills with brain-first lookup, entity detection, heartbeat
setup: Phase D rewritten with brain-first lookup protocol (gbrain search
→ query → get → grep fallback), sync-after-write rule, memory_search
complement table.
query: token-budget awareness (chunks not full pages), source precedence
hierarchy (user > compiled truth > timeline > external).
ingest: entity detection on every message (scan, check brain, create or
enrich, commit and sync).
maintain: heartbeat integration (doctor, embed --stale, sync verification,
stale compiled truth detection).
briefing: gbrain-native context loading (search attendees before meetings,
search sender before email, daily deal/meeting/commitment queries).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add OpenClaw positioning to README opener
Make it clear up top that GBrain is built for OpenClaw agents and
works with any OpenClaw deployment.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: credit Karpathy's Knowledge LLM vision, add origin story
GBrain started as Karpathy's LLM wiki idea built for real. Worked great
until the brain hit thousands of files and grep fell apart. GBrain is the
search layer that had to exist once the brain outgrew grep.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: contract-first operations.ts with OperationError, dry_run, importFromContent
30 shared operations as single source of truth for CLI and MCP.
- OperationError with typed error codes (page_not_found, invalid_params, etc.)
- dry_run support on all mutating operations
- importFromContent split from importFile with transaction wrapping
- Idempotency hash now includes ALL fields (title, type, frontmatter, tags)
- Config env var fallback: GBRAIN_DATABASE_URL > DATABASE_URL > config file
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: rewrite MCP server + CLI + tools-json from operations
server.ts: 233 -> ~80 lines. Tool definitions and dispatch generated from operations[].
cli.ts: shared operations auto-registered, CLI-only commands kept as manual dispatch.
tools-json: generated FROM operations[], eliminating the third contract surface.
Parity test verifies structural contract between operations, CLI, and MCP.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: delete 12 command files migrated to operations.ts
Handler logic for get, put, delete, list, search, query, health, stats,
tags, link, timeline, and version now lives in operations.ts.
Kept: init, upgrade, import, export, files, embed, sync, serve, call, config.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: init --non-interactive, upgrade verification, schema migration
- gbrain init --non-interactive --url <url> for plugin mode (no TTY required)
- Post-upgrade version verification in gbrain upgrade
- Drop storage_url from files table (storage_path is the only identifier)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: tool-agnostic skills + new setup skill
All 7 skills rewritten with intent-based language instead of CLI commands.
Works with both CLI and MCP plugin contexts.
New setup skill replaces install: auto-provision Supabase via CLI,
AGENTS.md injection, target TTHW < 2 min.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: ClawHub bundle plugin, CI workflows, v0.3.0
- openclaw.plugin.json with configSchema, MCP server config, skill listing
- GitHub Actions: test on push/PR, multi-platform release (macOS arm64 + Linux x64)
- Version bump 0.3.0, CHANGELOG, README ClawHub section, CLAUDE.md updated
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: idempotency hash mismatch + MCP dry_run passthrough
importFromContent now passes its all-fields hash through putPage via
content_hash on PageInput, so the stored hash matches the computed hash.
Previously the skip-if-unchanged check never fired because the hash
formulas differed.
MCP server now passes dry_run from tool params to OperationContext.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.3.0.0)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: schema loader handles PL/pgSQL $$ blocks
Delete the semicolon-based SQL splitter in db.ts which broke on
PL/pgSQL trigger functions containing semicolons inside $$ delimiter
blocks. Use single conn.unsafe(schemaSql) call instead — the postgres
driver handles multi-statement SQL natively. schema.sql already uses
IF NOT EXISTS / CREATE OR REPLACE for idempotency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: E2E test infrastructure + realistic brain fixtures
Add test infrastructure for running E2E tests against real
Postgres+pgvector. Includes:
- test/e2e/helpers.ts: DB lifecycle, fixture import, timing, diagnostics
- 13 fixture files as a miniature realistic brain (people, companies,
deals, meetings, concepts, projects, sources) following the
compiled truth + timeline format from GBRAIN_RECOMMENDED_SCHEMA.md
- docker-compose.test.yml: local pgvector convenience (port 5433)
- .env.testing.example: template for test credentials
- package.json: add test:e2e script
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: E2E test suites + CI workflow
Tier 1 (mechanical.test.ts): 14 test suites covering all operations
against real Postgres — page CRUD, search with quality scoring, links,
tags, timeline, versions, admin, chunks, resolution, ingest log, raw
data, files, idempotency stress, setup journey (full CLI flow), init
edge cases, schema idempotency, schema diff guard, performance baselines.
Tier 1 (mcp.test.ts): MCP protocol test — spawns server, sends JSON-RPC,
verifies tools/list matches operations count.
Tier 2 (skills.test.ts): OpenClaw skill tests — ingest, query, health.
Skips gracefully when dependencies missing.
CI (.github/workflows/e2e.yml): Tier 1 on every PR (pgvector service),
Tier 2 nightly/manual with API key secrets.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: E2E test fixes + traverseGraph jsonb cast
- Fix traverseGraph query: cast json_agg to jsonb_agg so SELECT DISTINCT works
- Fix put_page tests to use importFromContent with noEmbed (no OpenAI key in Tier 1)
- Fix get_health assertion (page_count not total_pages)
- Fix raw_data test to handle JSONB string/object return
- Simplify MCP test to verify tool generation directly
- Add timeouts to CLI subprocess tests
- Use port 5434 for docker-compose (5433 often in use)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: update all project docs for E2E test suite
- CLAUDE.md: updated test count (9 unit + 3 E2E), added E2E test
instructions, fixed skill count to 8
- CONTRIBUTING.md: updated project structure with test/e2e/, added E2E
test instructions, rewrote "Adding a new command" to reflect
contract-first architecture (add to operations.ts, done)
- README.md: fixed table count (10 not 9), added recommended schema doc
to Docs section, added E2E instructions to Contributing section
- CHANGELOG.md: added E2E test suite, docker-compose, schema loader fix,
and traverseGraph jsonb fix to v0.3.0 entry
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: expand brain schema — database architecture, dedup, enrichment sources, worked examples
Rewrite the recommended schema doc: present the database layer (entity registry,
event ledger, fact store, relationship graph) as the core architecture rather than
a future upgrade. Add entity identity/deduplication, enrichment source ordering,
epistemic discipline, three worked examples, concurrency guidance, and browser
budget. Smooth language for open-source readability.
* chore: bump version and changelog (v0.2.0.2)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add recommended brain schema
Full LLM-maintained knowledge base architecture: MECE directory structure,
compiled truth + timeline pages, enrichment pipeline, resolver decision
tree, skill architecture, and cron job recommendations.
* docs: fix first-time experience — remove fictional kindling, add GitHub URL
- Remove all references to data/kindling/ (never existed)
- OpenClaw paste now references https://github.com/garrytan/gbrain
- "Try it" section rewritten as three-act story with user's own data
- Agent picks dynamic query based on imported content
- Step 5 links to recommended schema doc for brain restructuring
- Includes bun install fallback in paste step 1
* chore: bump version and changelog (v0.2.0.1)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract importFile from import.ts + add tag reconciliation
Shared single-file import function used by both import and sync.
Adds tag reconciliation (removes stale tags on reimport), >1MB file
skip, and import->sync checkpoint continuity (writes git HEAD to
config table after import so sync picks up seamlessly).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add sync pure functions, updateSlug engine method, and sync tests
- buildSyncManifest: parses git diff --name-status -M output
- isSyncable: filters to .md pages, excludes hidden/ops/.raw/skip-list
- pathToSlug: converts file paths to page slugs with optional prefix
- updateSlug: renames page slug in-place (preserves page_id, chunks, embeddings)
- rewriteLinks: stub for v0.2 (FKs use page_id, already correct)
- 20 new tests, all passing (39 total across 3 files)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add gbrain sync command with CLI, MCP, and watch mode
18-step sync protocol: read config, git pull, ancestry validation,
git diff --name-status -M for net changes, isSyncable filter, process
deletes/renames/adds/modifies via importFile, batch optimization,
sync state checkpoint in Postgres config table. Watch mode with
polling and consecutive error counter. MCP sync_brain tool returns
structured SyncResult. Stale page deletion for un-syncable files.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add files table, gbrain files commands, and config show redaction
- files table: page_slug FK with ON DELETE SET NULL + ON UPDATE CASCADE,
storage_path, storage_url, mime_type, content_hash for dedup
- gbrain files list/upload/sync/verify commands for Supabase Storage
- gbrain config show redacts postgresql:// passwords and secret keys
- CLI help updated with FILES section
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add install skill for GBrain onboarding
6-phase install workflow: environment discovery, Supabase setup (magic
path via CLI OAuth or fallback 2-copy-paste), init + import, ongoing
sync cron, optional file migration with mandatory verification, and
agent teaching (AGENTS.md rules). Every error gets what + why + fix.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: update project documentation for v0.2.0
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add v0.2 features to README (sync, files, install skill)
README.md: added sync command to IMPORT/EXPORT section, added FILES
section with 4 commands, added files table to schema diagram, added
install skill to skills table, updated MCP tools count from 20 to 21
(sync_brain added).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: OpenClaw DX improvements (skill count, upgrade docs, config show help)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: consolidate version to single source of truth
Create src/version.ts that reads from package.json via static import
(safe for bun compiled binaries). Update mcp/server.ts from hardcoded
'0.1.0' to use shared VERSION. Bump skills/manifest.json to 0.2.0.
* fix: upgrade detection order, npm→bun naming, clawhub false positives
Reorder detection: node_modules first, binary second, clawhub last.
Rename 'npm' install method to 'bun'. Use 'clawhub --version' instead
of 'which clawhub' to avoid false positives from dangling symlinks.
Add 120s timeout to execSync calls to prevent hanging. Add --help flag.
* feat: per-command --help, unknown command check before DB connection
Add COMMAND_HELP map covering all 28 commands. Check --help before
init/upgrade dispatch and before connectEngine() so help works without
a database. Use COMMAND_HELP keys as known-command set to catch unknown
commands before wasting a DB round-trip.
* docs: standardize npm references to bun, add Upgrade section to README
Fix init.ts: npx→bunx, npm→bun for supabase CLI guidance.
Fix README: npm install→bun add for standalone CLI install.
Add ## Upgrade section to README with all three install methods.
Update install skill Upgrading section to list bun, ClawHub, and binary.
* test: full coverage audit — CLI dispatch, upgrade detection, config, edge cases
New test files:
- test/cli.test.ts: COMMAND_HELP ↔ switch consistency, version from
package.json, per-command --help, unknown command handling, global help
- test/upgrade.test.ts: detection order verification, npm→bun naming,
clawhub --version (not which), timeout presence
- test/config.test.ts: redactUrl for postgresql URLs, edge cases
Extended existing tests:
- test/sync.test.ts: empty string pathToSlug, uppercase .MD rejection,
deeply nested files, multiple renames, unknown status codes
- test/markdown.test.ts: multiple --- separators, missing frontmatter,
no frontmatter at all, empty string, type inference from paths
Tests: 39 → 83 (+44 new). All pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: 100% coverage — import-file mock engine, files utils, chunker edge cases
New test files:
- test/import-file.test.ts (9 tests): mock BrainEngine to test importFile
without DB — MAX_FILE_SIZE skip, content_hash dedup, tag reconciliation
(remove stale + add new), compiled_truth/timeline chunking, noEmbed flag,
sequential chunk_index
- test/files.test.ts (22 tests): getMimeType for all extensions + uppercase
+ unknown + no-extension, fileHash consistency + different content + empty,
collectFiles pattern (skip .md, skip hidden dirs, recurse, sorted output)
Extended:
- test/chunkers/recursive.test.ts (+6 tests): single newline splits,
word-only text, clause delimiters, lossless preservation, default options,
mixed delimiter hierarchy
Tests: 83 → 118 (+35 new). All pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: add CLAUDE.md with project context and gstack skill routing rules
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: initialize project with Bun + TypeScript
package.json with dependencies (postgres, pgvector, openai, anthropic,
MCP SDK, gray-matter). TypeScript config targeting ESNext with bundler
module resolution.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add foundation layer — engine interface, Postgres engine, schema
BrainEngine pluggable interface with full PostgresEngine: CRUD, search
(keyword + vector), links, tags, timeline, versions, stats, health,
ingest log, config. Trigger-based tsvector spanning pages +
timeline_entries. Markdown parser with frontmatter, compiled_truth /
timeline splitting, and round-trip serialization. 19 tests passing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add 3-tier chunking and embedding service
Recursive delimiter-aware chunker (5-level hierarchy, 300-word chunks,
50-word overlap). Semantic chunker with Savitzky-Golay boundary detection
and recursive fallback. LLM-guided chunker via Claude Haiku with sliding
window topic detection. OpenAI embedding service with batch support,
exponential backoff, and rate limit handling.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add hybrid search with RRF fusion, expansion, and 4-layer dedup
Hybrid search merges vector (pgvector HNSW) + keyword (tsvector) via
Reciprocal Rank Fusion. Multi-query expansion via Claude Haiku generates
2 alternative phrasings. 4-layer dedup pipeline: by source, cosine
similarity, type diversity (60% cap), per-page cap.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add GBRAIN_V0 spec, pluggable engine architecture, SQLite engine plan
GBRAIN_V0.md: full product spec with architecture decisions, CLI commands,
schema, search architecture, chunking strategies, first-time experience,
and future plans. ENGINES.md: pluggable engine interface, capability matrix,
how to add new backends. SQLITE_ENGINE.md: complete SQLite implementation
plan with schema, FTS5 setup, vector search options, and contributor guide.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add CLI with all commands
Full CLI dispatcher with 25+ commands: init (Supabase wizard), get, put,
delete, list, search, query (hybrid RRF), import (bulk with progress bar),
export (round-trip), embed, stats, health, tag/untag/tags, link/unlink/
backlinks/graph, timeline/timeline-add, history/revert, config, upgrade,
serve, call. Smart slug resolution on reads. Version snapshots on updates.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add MCP stdio server with all brain tools
20 MCP tools mirroring CLI operations: get/put/delete/list pages,
search (keyword), query (hybrid RRF + expansion), tags, links with
graph traversal, timeline, stats, health, version history, and revert.
Auto-chunks and embeds on put_page. CLI and MCP share the same engine.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add 6 skill files and ClawHub manifest
Fat markdown skills for AI agents: ingest (meetings/docs/articles with
timeline merge), query (3-layer search + synthesis + citations), maintain
(health checks, stale detection, orphan audit), enrich (external API
enrichment), briefing (daily briefing compilation), migrate (universal
migration from Obsidian/Notion/Logseq/markdown/CSV/JSON/Roam).
ClawHub manifest for skill distribution.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add README, CONTRIBUTING, update CLAUDE.md test references
README with quickstart, commands, architecture, library usage, MCP setup,
and links to design docs. CONTRIBUTING with setup, project structure,
and guides for adding commands and engines. CLAUDE.md updated to reference
actual test files instead of planned-but-unwritten import test.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address adversarial review findings — 5 critical/high fixes
- revertToVersion: add page_id check to prevent cross-page data corruption
- traverseGraph: use UNION instead of UNION ALL for cycle safety
- embedAll: preserve all chunks when embedding stale subset only
- embedding: throw on retry exhaustion instead of returning zero vectors
- putPage: validate slugs to prevent path traversal on export
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.1.0)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: expand README with schema, install, search architecture, and motivation
Why it exists, how search works (with ASCII diagram), full database schema
with all 9 tables and index details, chunking strategies explained, storage
estimates, setup wizard walkthrough, knowledge model with example page,
library usage with more examples, expanded skills table.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: add MIT license (Copyright 2026 Garry Tan)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add OpenClaw install flow as primary option in README
OpenClaw users just say "install gbrain" and the orchestrator handles
everything: package install, Supabase setup wizard, skill registration.
Shows the conversational interface for querying, ingesting, and briefings.
ClawHub and standalone CLI paths follow as alternatives.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add prerequisites and explicit OpenClaw install instructions
Prerequisites table listing Supabase, OpenAI, and Anthropic dependencies
with links. Environment variable setup. Explicit step-by-step prompt for
OpenClaw users showing exactly what to tell the orchestrator. Note that
search degrades gracefully without API keys (keyword-only without OpenAI,
no expansion without Anthropic).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: scrub named references, add PG essay demo section to README
Replace all Pedro/Brex/Jensen Huang/River AI examples with Paul Graham
essay examples using the kindling corpus. Add "Try it" section to README
showing the power of hybrid search on PG essays in 90 seconds. Update
test fixtures to use concept pages instead of person pages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
