feat: GBrain v0.3.0 — contract-first architecture + ClawHub plugin (#7)

* feat: contract-first operations.ts with OperationError, dry_run, importFromContent

30 shared operations as single source of truth for CLI and MCP.
- OperationError with typed error codes (page_not_found, invalid_params, etc.)
- dry_run support on all mutating operations
- importFromContent split from importFile with transaction wrapping
- Idempotency hash now includes ALL fields (title, type, frontmatter, tags)
- Config env var fallback: GBRAIN_DATABASE_URL > DATABASE_URL > config file

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: rewrite MCP server + CLI + tools-json from operations

server.ts: 233 -> ~80 lines. Tool definitions and dispatch generated from operations[].
cli.ts: shared operations auto-registered, CLI-only commands kept as manual dispatch.
tools-json: generated FROM operations[], eliminating the third contract surface.
Parity test verifies structural contract between operations, CLI, and MCP.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: delete 12 command files migrated to operations.ts

Handler logic for get, put, delete, list, search, query, health, stats,
tags, link, timeline, and version now lives in operations.ts.
Kept: init, upgrade, import, export, files, embed, sync, serve, call, config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: init --non-interactive, upgrade verification, schema migration

- gbrain init --non-interactive --url <url> for plugin mode (no TTY required)
- Post-upgrade version verification in gbrain upgrade
- Drop storage_url from files table (storage_path is the only identifier)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: tool-agnostic skills + new setup skill

All 7 skills rewritten with intent-based language instead of CLI commands.
Works with both CLI and MCP plugin contexts.
New setup skill replaces install: auto-provision Supabase via CLI,
AGENTS.md injection, target TTHW < 2 min.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: ClawHub bundle plugin, CI workflows, v0.3.0

- openclaw.plugin.json with configSchema, MCP server config, skill listing
- GitHub Actions: test on push/PR, multi-platform release (macOS arm64 + Linux x64)
- Version bump 0.3.0, CHANGELOG, README ClawHub section, CLAUDE.md updated

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: idempotency hash mismatch + MCP dry_run passthrough

importFromContent now passes its all-fields hash through putPage via
content_hash on PageInput, so the stored hash matches the computed hash.
Previously the skip-if-unchanged check never fired because the hash
formulas differed.

MCP server now passes dry_run from tool params to OperationContext.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.3.0.0)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: schema loader handles PL/pgSQL $$ blocks

Delete the semicolon-based SQL splitter in db.ts which broke on
PL/pgSQL trigger functions containing semicolons inside $$ delimiter
blocks. Use single conn.unsafe(schemaSql) call instead — the postgres
driver handles multi-statement SQL natively. schema.sql already uses
IF NOT EXISTS / CREATE OR REPLACE for idempotency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: E2E test infrastructure + realistic brain fixtures

Add test infrastructure for running E2E tests against real
Postgres+pgvector. Includes:
- test/e2e/helpers.ts: DB lifecycle, fixture import, timing, diagnostics
- 13 fixture files as a miniature realistic brain (people, companies,
  deals, meetings, concepts, projects, sources) following the
  compiled truth + timeline format from GBRAIN_RECOMMENDED_SCHEMA.md
- docker-compose.test.yml: local pgvector convenience (port 5433)
- .env.testing.example: template for test credentials
- package.json: add test:e2e script

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: E2E test suites + CI workflow

Tier 1 (mechanical.test.ts): 14 test suites covering all operations
against real Postgres — page CRUD, search with quality scoring, links,
tags, timeline, versions, admin, chunks, resolution, ingest log, raw
data, files, idempotency stress, setup journey (full CLI flow), init
edge cases, schema idempotency, schema diff guard, performance baselines.

Tier 1 (mcp.test.ts): MCP protocol test — spawns server, sends JSON-RPC,
verifies tools/list matches operations count.

Tier 2 (skills.test.ts): OpenClaw skill tests — ingest, query, health.
Skips gracefully when dependencies missing.

CI (.github/workflows/e2e.yml): Tier 1 on every PR (pgvector service),
Tier 2 nightly/manual with API key secrets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: E2E test fixes + traverseGraph jsonb cast

- Fix traverseGraph query: cast json_agg to jsonb_agg so SELECT DISTINCT works
- Fix put_page tests to use importFromContent with noEmbed (no OpenAI key in Tier 1)
- Fix get_health assertion (page_count not total_pages)
- Fix raw_data test to handle JSONB string/object return
- Simplify MCP test to verify tool generation directly
- Add timeouts to CLI subprocess tests
- Use port 5434 for docker-compose (5433 often in use)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update all project docs for E2E test suite

- CLAUDE.md: updated test count (9 unit + 3 E2E), added E2E test
  instructions, fixed skill count to 8
- CONTRIBUTING.md: updated project structure with test/e2e/, added E2E
  test instructions, rewrote "Adding a new command" to reflect
  contract-first architecture (add to operations.ts, done)
- README.md: fixed table count (10 not 9), added recommended schema doc
  to Docs section, added E2E instructions to Contributing section
- CHANGELOG.md: added E2E test suite, docker-compose, schema loader fix,
  and traverseGraph jsonb fix to v0.3.0 entry

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-04-08 23:26:11 -10:00
committed by GitHub
parent ee9e6689ad
commit a86f995883
67 changed files with 3827 additions and 1394 deletions

View File

@@ -1,66 +1,29 @@
import { describe, test, expect } from 'bun:test';
import { readFileSync } from 'fs';
// Read cli.ts source to extract COMMAND_HELP keys and switch cases
// Read cli.ts source for structural checks
const cliSource = readFileSync(new URL('../src/cli.ts', import.meta.url), 'utf-8');
// Extract COMMAND_HELP keys from the map
function extractCommandHelpKeys(source: string): string[] {
const mapMatch = source.match(/const COMMAND_HELP:\s*Record<string,\s*string>\s*=\s*\{([\s\S]*?)\};/);
if (!mapMatch) return [];
const keys: string[] = [];
for (const m of mapMatch[1].matchAll(/^\s*['"]?([a-z-]+)['"]?\s*:/gm)) {
keys.push(m[1]);
}
return keys.sort();
}
// Extract switch case labels from the switch(command) block
function extractSwitchCases(source: string): string[] {
const cases: string[] = [];
for (const m of source.matchAll(/case\s+'([^']+)':\s*\{/g)) {
cases.push(m[1]);
}
return [...new Set(cases)].sort();
}
// Extract commands handled before the switch (init, upgrade)
function extractEarlyCommands(source: string): string[] {
const cmds: string[] = [];
for (const m of source.matchAll(/if\s*\(command\s*===\s*'([^']+)'\)/g)) {
if (!['--help', '-h', '--version', '--tools-json'].includes(m[1])) {
cmds.push(m[1]);
}
}
return [...new Set(cmds)].sort();
}
describe('CLI COMMAND_HELP consistency', () => {
const helpKeys = extractCommandHelpKeys(cliSource);
const switchCases = extractSwitchCases(cliSource);
const earlyCmds = extractEarlyCommands(cliSource);
const allHandled = [...switchCases, ...earlyCmds].sort();
test('COMMAND_HELP has entries for all switch cases', () => {
for (const cmd of switchCases) {
expect(helpKeys).toContain(cmd);
}
describe('CLI structure', () => {
test('imports operations from operations.ts', () => {
expect(cliSource).toContain("from './core/operations.ts'");
});
test('COMMAND_HELP has entries for early-dispatch commands (init, upgrade)', () => {
for (const cmd of earlyCmds) {
expect(helpKeys).toContain(cmd);
}
test('builds cliOps map from operations', () => {
expect(cliSource).toContain('cliOps');
});
test('every COMMAND_HELP key maps to a handled command', () => {
for (const key of helpKeys) {
expect(allHandled).toContain(key);
}
test('CLI_ONLY set contains expected commands', () => {
expect(cliSource).toContain("'init'");
expect(cliSource).toContain("'upgrade'");
expect(cliSource).toContain("'import'");
expect(cliSource).toContain("'export'");
expect(cliSource).toContain("'embed'");
expect(cliSource).toContain("'files'");
});
test('COMMAND_HELP has at least 25 entries', () => {
expect(helpKeys.length).toBeGreaterThanOrEqual(25);
test('has formatResult function for CLI output', () => {
expect(cliSource).toContain('function formatResult');
});
});
@@ -77,24 +40,6 @@ describe('CLI version', () => {
});
});
describe('CLI help text', () => {
test('every COMMAND_HELP entry starts with Usage:', () => {
const mapMatch = cliSource.match(/const COMMAND_HELP:\s*Record<string,\s*string>\s*=\s*\{([\s\S]*?)\};/);
expect(mapMatch).not.toBeNull();
// Verify by importing and checking
const keys = extractCommandHelpKeys(cliSource);
expect(keys.length).toBeGreaterThan(0);
// Each help string in the source should contain 'Usage:'
for (const key of keys) {
const pattern = new RegExp(`['"]?${key.replace('-', '\\-')}['"]?:\\s*['"\`]([^'"\`]*)`);
const match = cliSource.match(pattern);
if (match) {
expect(match[1]).toContain('Usage:');
}
}
});
});
describe('CLI dispatch integration', () => {
test('--version outputs version', async () => {
const proc = Bun.spawn(['bun', 'run', 'src/cli.ts', '--version'], {
@@ -143,18 +88,6 @@ describe('CLI dispatch integration', () => {
expect(exitCode).toBe(0);
});
test('init --help prints usage without running wizard', async () => {
const proc = Bun.spawn(['bun', 'run', 'src/cli.ts', 'init', '--help'], {
cwd: new URL('..', import.meta.url).pathname,
stdout: 'pipe',
stderr: 'pipe',
});
const stdout = await new Response(proc.stdout).text();
const exitCode = await proc.exited;
expect(stdout).toContain('Usage: gbrain init');
expect(exitCode).toBe(0);
});
test('--help prints global help', async () => {
const proc = Bun.spawn(['bun', 'run', 'src/cli.ts', '--help'], {
cwd: new URL('..', import.meta.url).pathname,
@@ -168,16 +101,19 @@ describe('CLI dispatch integration', () => {
expect(exitCode).toBe(0);
});
test('files --help prints subcommand help', async () => {
const proc = Bun.spawn(['bun', 'run', 'src/cli.ts', 'files', '--help'], {
test('--tools-json outputs valid JSON with operations', async () => {
const proc = Bun.spawn(['bun', 'run', 'src/cli.ts', '--tools-json'], {
cwd: new URL('..', import.meta.url).pathname,
stdout: 'pipe',
stderr: 'pipe',
});
const stdout = await new Response(proc.stdout).text();
const exitCode = await proc.exited;
expect(stdout).toContain('files list');
expect(stdout).toContain('files upload');
expect(exitCode).toBe(0);
await proc.exited;
const tools = JSON.parse(stdout);
expect(Array.isArray(tools)).toBe(true);
expect(tools.length).toBeGreaterThanOrEqual(30);
expect(tools[0]).toHaveProperty('name');
expect(tools[0]).toHaveProperty('description');
expect(tools[0]).toHaveProperty('parameters');
});
});