Files
gbrain/test/sync.test.ts
Garry Tan ecebd5552a feat: GBrain v0.2.0 — incremental sync, file storage, install skill (#2)
* refactor: extract importFile from import.ts + add tag reconciliation

Shared single-file import function used by both import and sync.
Adds tag reconciliation (removes stale tags on reimport), >1MB file
skip, and import->sync checkpoint continuity (writes git HEAD to
config table after import so sync picks up seamlessly).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add sync pure functions, updateSlug engine method, and sync tests

- buildSyncManifest: parses git diff --name-status -M output
- isSyncable: filters to .md pages, excludes hidden/ops/.raw/skip-list
- pathToSlug: converts file paths to page slugs with optional prefix
- updateSlug: renames page slug in-place (preserves page_id, chunks, embeddings)
- rewriteLinks: stub for v0.2 (FKs use page_id, already correct)
- 20 new tests, all passing (39 total across 3 files)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add gbrain sync command with CLI, MCP, and watch mode

18-step sync protocol: read config, git pull, ancestry validation,
git diff --name-status -M for net changes, isSyncable filter, process
deletes/renames/adds/modifies via importFile, batch optimization,
sync state checkpoint in Postgres config table. Watch mode with
polling and consecutive error counter. MCP sync_brain tool returns
structured SyncResult. Stale page deletion for un-syncable files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add files table, gbrain files commands, and config show redaction

- files table: page_slug FK with ON DELETE SET NULL + ON UPDATE CASCADE,
  storage_path, storage_url, mime_type, content_hash for dedup
- gbrain files list/upload/sync/verify commands for Supabase Storage
- gbrain config show redacts postgresql:// passwords and secret keys
- CLI help updated with FILES section

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add install skill for GBrain onboarding

6-phase install workflow: environment discovery, Supabase setup (magic
path via CLI OAuth or fallback 2-copy-paste), init + import, ongoing
sync cron, optional file migration with mandatory verification, and
agent teaching (AGENTS.md rules). Every error gets what + why + fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update project documentation for v0.2.0

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add v0.2 features to README (sync, files, install skill)

README.md: added sync command to IMPORT/EXPORT section, added FILES
section with 4 commands, added files table to schema diagram, added
install skill to skills table, updated MCP tools count from 20 to 21
(sync_brain added).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: OpenClaw DX improvements (skill count, upgrade docs, config show help)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: consolidate version to single source of truth

Create src/version.ts that reads from package.json via static import
(safe for bun compiled binaries). Update mcp/server.ts from hardcoded
'0.1.0' to use shared VERSION. Bump skills/manifest.json to 0.2.0.

* fix: upgrade detection order, npm→bun naming, clawhub false positives

Reorder detection: node_modules first, binary second, clawhub last.
Rename 'npm' install method to 'bun'. Use 'clawhub --version' instead
of 'which clawhub' to avoid false positives from dangling symlinks.
Add 120s timeout to execSync calls to prevent hanging. Add --help flag.

* feat: per-command --help, unknown command check before DB connection

Add COMMAND_HELP map covering all 28 commands. Check --help before
init/upgrade dispatch and before connectEngine() so help works without
a database. Use COMMAND_HELP keys as known-command set to catch unknown
commands before wasting a DB round-trip.

* docs: standardize npm references to bun, add Upgrade section to README

Fix init.ts: npx→bunx, npm→bun for supabase CLI guidance.
Fix README: npm install→bun add for standalone CLI install.
Add ## Upgrade section to README with all three install methods.
Update install skill Upgrading section to list bun, ClawHub, and binary.

* test: full coverage audit — CLI dispatch, upgrade detection, config, edge cases

New test files:
- test/cli.test.ts: COMMAND_HELP ↔ switch consistency, version from
  package.json, per-command --help, unknown command handling, global help
- test/upgrade.test.ts: detection order verification, npm→bun naming,
  clawhub --version (not which), timeout presence
- test/config.test.ts: redactUrl for postgresql URLs, edge cases

Extended existing tests:
- test/sync.test.ts: empty string pathToSlug, uppercase .MD rejection,
  deeply nested files, multiple renames, unknown status codes
- test/markdown.test.ts: multiple --- separators, missing frontmatter,
  no frontmatter at all, empty string, type inference from paths

Tests: 39 → 83 (+44 new). All pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: 100% coverage — import-file mock engine, files utils, chunker edge cases

New test files:
- test/import-file.test.ts (9 tests): mock BrainEngine to test importFile
  without DB — MAX_FILE_SIZE skip, content_hash dedup, tag reconciliation
  (remove stale + add new), compiled_truth/timeline chunking, noEmbed flag,
  sequential chunk_index
- test/files.test.ts (22 tests): getMimeType for all extensions + uppercase
  + unknown + no-extension, fileHash consistency + different content + empty,
  collectFiles pattern (skip .md, skip hidden dirs, recurse, sorted output)

Extended:
- test/chunkers/recursive.test.ts (+6 tests): single newline splits,
  word-only text, clause delimiters, lossless preservation, default options,
  mixed delimiter hierarchy

Tests: 83 → 118 (+35 new). All pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:50:15 -07:00

180 lines
6.1 KiB
TypeScript

import { describe, test, expect } from 'bun:test';
import { buildSyncManifest, isSyncable, pathToSlug } from '../src/core/sync.ts';
describe('buildSyncManifest', () => {
test('parses A/M/D entries from single commit', () => {
const output = `A\tpeople/new-person.md\nM\tpeople/existing-person.md\nD\tpeople/deleted-person.md`;
const manifest = buildSyncManifest(output);
expect(manifest.added).toEqual(['people/new-person.md']);
expect(manifest.modified).toEqual(['people/existing-person.md']);
expect(manifest.deleted).toEqual(['people/deleted-person.md']);
expect(manifest.renamed).toEqual([]);
});
test('parses R100 rename entries', () => {
const output = `R100\tpeople/old-name.md\tpeople/new-name.md`;
const manifest = buildSyncManifest(output);
expect(manifest.renamed).toEqual([{ from: 'people/old-name.md', to: 'people/new-name.md' }]);
expect(manifest.added).toEqual([]);
expect(manifest.modified).toEqual([]);
expect(manifest.deleted).toEqual([]);
});
test('parses partial rename (R075)', () => {
const output = `R075\tpeople/old.md\tpeople/new.md`;
const manifest = buildSyncManifest(output);
expect(manifest.renamed).toEqual([{ from: 'people/old.md', to: 'people/new.md' }]);
});
test('handles empty diff', () => {
const manifest = buildSyncManifest('');
expect(manifest.added).toEqual([]);
expect(manifest.modified).toEqual([]);
expect(manifest.deleted).toEqual([]);
expect(manifest.renamed).toEqual([]);
});
test('handles mixed entries with blank lines', () => {
const output = `A\tpeople/a.md\n\nM\tpeople/b.md\n\nD\tpeople/c.md`;
const manifest = buildSyncManifest(output);
expect(manifest.added).toEqual(['people/a.md']);
expect(manifest.modified).toEqual(['people/b.md']);
expect(manifest.deleted).toEqual(['people/c.md']);
});
test('skips malformed lines', () => {
const output = `A\tpeople/a.md\ngarbage line\nM\tpeople/b.md`;
const manifest = buildSyncManifest(output);
expect(manifest.added).toEqual(['people/a.md']);
expect(manifest.modified).toEqual(['people/b.md']);
});
});
describe('isSyncable', () => {
test('accepts normal .md files', () => {
expect(isSyncable('people/pedro-franceschi.md')).toBe(true);
expect(isSyncable('meetings/2026-04-03-lunch.md')).toBe(true);
expect(isSyncable('daily/2026-04-05.md')).toBe(true);
expect(isSyncable('notes.md')).toBe(true);
});
test('rejects non-.md files', () => {
expect(isSyncable('people/photo.jpg')).toBe(false);
expect(isSyncable('config.json')).toBe(false);
expect(isSyncable('src/cli.ts')).toBe(false);
});
test('rejects files in hidden directories', () => {
expect(isSyncable('.git/config')).toBe(false);
expect(isSyncable('.obsidian/plugins.md')).toBe(false);
expect(isSyncable('people/.hidden/secret.md')).toBe(false);
});
test('rejects .raw/ sidecar directories', () => {
expect(isSyncable('people/pedro.raw/source.md')).toBe(false);
expect(isSyncable('dir/.raw/notes.md')).toBe(false);
});
test('rejects skip-list basenames', () => {
expect(isSyncable('schema.md')).toBe(false);
expect(isSyncable('index.md')).toBe(false);
expect(isSyncable('log.md')).toBe(false);
expect(isSyncable('README.md')).toBe(false);
expect(isSyncable('people/README.md')).toBe(false);
});
test('rejects ops/ directory', () => {
expect(isSyncable('ops/deploy-log.md')).toBe(false);
expect(isSyncable('ops/config.md')).toBe(false);
});
});
describe('pathToSlug', () => {
test('strips .md extension', () => {
expect(pathToSlug('people/pedro-franceschi.md')).toBe('people/pedro-franceschi');
});
test('preserves case', () => {
expect(pathToSlug('People/Pedro-Franceschi.md')).toBe('People/Pedro-Franceschi');
});
test('strips leading slash', () => {
expect(pathToSlug('/people/pedro.md')).toBe('people/pedro');
});
test('normalizes backslash separators', () => {
expect(pathToSlug('people\\pedro.md')).toBe('people/pedro');
});
test('handles flat files', () => {
expect(pathToSlug('notes.md')).toBe('notes');
});
test('handles nested paths', () => {
expect(pathToSlug('projects/gbrain/spec.md')).toBe('projects/gbrain/spec');
});
test('adds repo prefix when provided', () => {
expect(pathToSlug('people/pedro.md', 'brain')).toBe('brain/people/pedro');
});
test('no prefix when not provided', () => {
expect(pathToSlug('people/pedro.md')).toBe('people/pedro');
});
test('handles empty string', () => {
expect(pathToSlug('')).toBe('');
});
test('handles file with only extension', () => {
expect(pathToSlug('.md')).toBe('');
});
});
describe('isSyncable edge cases', () => {
test('rejects uppercase .MD extension', () => {
// isSyncable checks path.endsWith('.md'), so .MD should fail
expect(isSyncable('people/someone.MD')).toBe(false);
});
test('rejects files with no extension', () => {
expect(isSyncable('README')).toBe(false);
});
test('accepts deeply nested .md files', () => {
expect(isSyncable('a/b/c/d/e/f/deep.md')).toBe(true);
});
test('rejects .md files inside nested hidden dirs', () => {
expect(isSyncable('docs/.internal/secret.md')).toBe(false);
});
});
describe('buildSyncManifest edge cases', () => {
test('handles tab-separated fields correctly', () => {
const output = "A\tpath/to/file.md";
const manifest = buildSyncManifest(output);
expect(manifest.added).toEqual(['path/to/file.md']);
});
test('handles multiple renames', () => {
const output = [
'R100\told/a.md\tnew/a.md',
'R095\told/b.md\tnew/b.md',
].join('\n');
const manifest = buildSyncManifest(output);
expect(manifest.renamed).toHaveLength(2);
expect(manifest.renamed[0].from).toBe('old/a.md');
expect(manifest.renamed[1].from).toBe('old/b.md');
});
test('ignores unknown status codes', () => {
const output = "X\tunknown/file.md";
const manifest = buildSyncManifest(output);
expect(manifest.added).toEqual([]);
expect(manifest.modified).toEqual([]);
expect(manifest.deleted).toEqual([]);
expect(manifest.renamed).toEqual([]);
});
});