* fix: splitBody and inferType for wiki-style markdown content - splitBody now requires explicit timeline sentinel (<!-- timeline -->, --- timeline ---, or --- directly before ## Timeline / ## History). A bare --- in body text is a markdown horizontal rule, not a separator. This fixes the 83% content truncation @knee5 reported on a 1,991-article wiki where 4,856 of 6,680 wikilinks were lost. - serializeMarkdown emits <!-- timeline --> sentinel for round-trip stability. - inferType extended with /writing/, /wiki/analysis/, /wiki/guides/, /wiki/hardware/, /wiki/architecture/, /wiki/concepts/. Path order is most-specific-first so projects/blog/writing/essay.md → writing, not project. - PageType union extended: writing, analysis, guide, hardware, architecture. Updates test/import-file.test.ts to use the new sentinel. Co-Authored-By: @knee5 (PR #187) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: JSONB double-encode bug on Postgres + parseEmbedding NaN scores Two related Postgres-string-typed-data bugs that PGLite hid: 1. JSONB double-encode (postgres-engine.ts:107,668,846 + files.ts:254): ${JSON.stringify(value)}::jsonb in postgres.js v3 stringified again on the wire, storing JSONB columns as quoted string literals. Every frontmatter->>'key' returned NULL on Postgres-backed brains; GIN indexes were inert. Switched to sql.json(value), which is the postgres.js-native JSONB encoder (Parameter with OID 3802). Affected columns: pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata. page_versions.frontmatter is downstream via INSERT...SELECT and propagates the fix. 2. pgvector embeddings returning as strings (utils.ts): getEmbeddingsByChunkIds returned "[0.1,0.2,...]" instead of Float32Array on Supabase, producing [NaN] cosine scores. Adds parseEmbedding() helper handling Float32Array, numeric arrays, and pgvector string format. Throws loud on malformed vectors (per Codex's no-silent-NaN requirement); returns null for non-vector strings (treated as "no embedding here"). rowToChunk delegates to parseEmbedding. E2E regression test at test/e2e/postgres-jsonb.test.ts asserts jsonb_typeof = 'object' AND col->>'k' returns expected scalar across all 5 affected columns — the test that should have caught the original bug. Runs in CI via the existing pgvector service. Co-Authored-By: @knee5 (PR #187 — JSONB triple-fix) Co-Authored-By: @leonardsellem (PR #175 — parseEmbedding) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: extract wikilink syntax with ancestor-search slug resolution extractMarkdownLinks now handles [[page]] and [[page|Display Text]] alongside standard [text](page.md). For wiki KBs where authors omit leading ../ (thinking in wiki-root-relative terms), resolveSlug walks ancestor directories until it finds a matching slug. Without this, wikilinks under tech/wiki/analysis/ targeting [[../../finance/wiki/concepts/foo]] silently dangled when the correct relative depth was 3 × ../ instead of 2. Co-Authored-By: @knee5 (PR #187) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: gbrain repair-jsonb + v0.12.1 migration + CI grep guard - New gbrain repair-jsonb command. Detects rows where jsonb_typeof(col) = 'string' and rewrites them via (col #>> '{}')::jsonb across 5 affected columns: pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter. Idempotent — re-running is a no-op. PGLite engines short-circuit cleanly (the bug never affected the parameterized encode path PGLite uses). --dry-run shows what would be repaired; --json for scripting. - New v0_12_1.ts migration orchestrator. Phases: schema → repair → verify. Modeled on v0_12_0 pattern, registered in migrations/index.ts. Runs automatically via gbrain upgrade / apply-migrations. - CI grep guard at scripts/check-jsonb-pattern.sh fails the build if anyone reintroduces the ${JSON.stringify(x)}::jsonb interpolation pattern. Wired into bun test via package.json. Best-effort static analysis (multi-line and helper-wrapped variants are caught by the E2E round-trip test instead). - Updates apply-migrations.test.ts expectations to account for the new v0.12.1 entry in the registry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.12.1) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update project documentation for v0.12.1 - CLAUDE.md: document repair-jsonb command, v0_12_1 migration, splitBody sentinel contract, inferType wiki subtypes, CI grep guard, new test files (repair-jsonb, migrations-v0_12_1, markdown) - README.md: add gbrain repair-jsonb to ADMIN command reference - INSTALL_FOR_AGENTS.md: fix verification count (6 -> 7), add v0.12.1 upgrade guidance for Postgres brains - docs/GBRAIN_VERIFY.md: add check #8 for JSONB integrity on Postgres-backed brains - docs/UPGRADING_DOWNSTREAM_AGENTS.md: add v0.12.1 section with migration steps, splitBody contract, wiki subtype inference - skills/migrate/SKILL.md: document native wikilink extraction via gbrain extract links (v0.12.1+) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
294 lines
7.6 KiB
Markdown
294 lines
7.6 KiB
Markdown
# GBrain Installation Verification Runbook
|
|
|
|
Run these checks after install to confirm every part of GBrain is working.
|
|
Each check includes the command, expected output, and what to do if it fails.
|
|
|
|
The most important check is #4 (live sync). "Sync ran" is not the same as
|
|
"sync worked." A sync that silently skips pages because of a pooler bug is
|
|
worse than no sync at all, because you think it's working.
|
|
|
|
---
|
|
|
|
## 1. Schema Verification
|
|
|
|
**Command:**
|
|
|
|
```bash
|
|
gbrain doctor --json
|
|
```
|
|
|
|
**Expected:** All checks return `"ok"`:
|
|
- `connection`: connected, N pages
|
|
- `pgvector`: extension installed
|
|
- `rls`: enabled on all tables
|
|
- `schema_version`: current
|
|
- `embeddings`: coverage percentage
|
|
|
|
**If it fails:** The doctor output includes specific fix instructions for each
|
|
check. See `skills/setup/SKILL.md` Error Recovery table.
|
|
|
|
---
|
|
|
|
## 2. Skillpack Loaded
|
|
|
|
**Check:** Ask the agent: "What is the brain-agent loop?"
|
|
|
|
**Expected:** The agent references GBRAIN_SKILLPACK.md Section 2 and describes
|
|
the read-write cycle: detect entities, read brain, respond with context, write
|
|
brain, sync.
|
|
|
|
**If it fails:** The agent hasn't loaded the skillpack. Run step 6 from the
|
|
install paste (read `docs/GBRAIN_SKILLPACK.md`).
|
|
|
|
---
|
|
|
|
## 3. Auto-Update Configured
|
|
|
|
**Command:**
|
|
|
|
```bash
|
|
gbrain check-update --json
|
|
```
|
|
|
|
**Expected:** Returns JSON with `current_version`, `latest_version`,
|
|
`update_available` (boolean). The cron `gbrain-update-check` is registered.
|
|
|
|
**If it fails:** Run step 7 from the install paste. See GBRAIN_SKILLPACK.md
|
|
Section 17.
|
|
|
|
---
|
|
|
|
## 4. Live Sync Actually Works
|
|
|
|
This is the most important check. Three parts.
|
|
|
|
### 4a. Coverage Check
|
|
|
|
Compare page count in the DB against syncable file count in the repo:
|
|
|
|
```bash
|
|
gbrain stats
|
|
```
|
|
|
|
Then count syncable files:
|
|
|
|
```bash
|
|
find /data/brain -name '*.md' \
|
|
-not -path '*/.*' \
|
|
-not -path '*/.raw/*' \
|
|
-not -path '*/ops/*' \
|
|
-not -name 'README.md' \
|
|
-not -name 'index.md' \
|
|
-not -name 'schema.md' \
|
|
-not -name 'log.md' \
|
|
| wc -l
|
|
```
|
|
|
|
**Expected:** Page count in `gbrain stats` should be close to the file count.
|
|
Some difference is normal (files added since last sync), but if page count is
|
|
less than half the file count, sync is silently skipping pages.
|
|
|
|
**If page count is way too low:** The #1 cause is the connection pooler bug.
|
|
Check your `DATABASE_URL`:
|
|
- If it contains `pooler.supabase.com:6543`, verify it's using **Session mode**,
|
|
not Transaction mode.
|
|
- Transaction mode breaks `engine.transaction()` and causes `.begin() is not a
|
|
function` errors.
|
|
- Fix: switch to Session mode pooler string, then run `gbrain sync --full`
|
|
to reimport everything.
|
|
|
|
### 4b. Embed Check
|
|
|
|
```bash
|
|
gbrain stats
|
|
```
|
|
|
|
**Expected:** Embedded chunk count should be close to total chunk count.
|
|
|
|
**If embedded is much lower than total:**
|
|
|
|
```bash
|
|
gbrain embed --stale
|
|
```
|
|
|
|
If `OPENAI_API_KEY` is not set, embeddings can't be generated. Keyword search
|
|
still works without embeddings, but hybrid/semantic search won't.
|
|
|
|
### 4c. End-to-End Test
|
|
|
|
This is the real test. Edit a brain page, push, wait, search.
|
|
|
|
1. Edit a page in the brain repo (e.g., correct a fact on a person's page):
|
|
|
|
```bash
|
|
# Example: fix a line in Gustaf's page
|
|
cd /data/brain
|
|
# Make a small edit to any .md file
|
|
git add -A && git commit -m "test: verify live sync" && git push
|
|
```
|
|
|
|
2. Wait for the next sync cycle (cron interval or `--watch` poll).
|
|
|
|
3. Search for the corrected text:
|
|
|
|
```bash
|
|
gbrain search "<text from the correction>"
|
|
```
|
|
|
|
**Expected:** The search returns the **corrected** text, not the old version.
|
|
|
|
**If it returns old text:** Sync failed silently. Check:
|
|
- Is the sync cron registered and running?
|
|
- Is `gbrain sync --watch` still alive (if using watch mode)?
|
|
- Run `gbrain config get sync.last_run` to see when sync last ran.
|
|
- Run `gbrain sync --repo /data/brain` manually and check for errors.
|
|
- If you see `.begin() is not a function`, fix the pooler (see 4a above).
|
|
|
|
---
|
|
|
|
## 5. Embedding Coverage
|
|
|
|
**Command:**
|
|
|
|
```bash
|
|
gbrain stats
|
|
```
|
|
|
|
**Expected:** Embedded chunk count matches (or is close to) total chunk count.
|
|
|
|
**If zero or very low:** `OPENAI_API_KEY` may be missing or invalid. Check:
|
|
|
|
```bash
|
|
echo $OPENAI_API_KEY | head -c 10
|
|
```
|
|
|
|
If blank, set the key. Then:
|
|
|
|
```bash
|
|
gbrain embed --stale
|
|
```
|
|
|
|
---
|
|
|
|
## 6. Brain-First Lookup Protocol
|
|
|
|
**Check:** Ask the agent about a person or concept that exists in the brain.
|
|
|
|
**Expected:** The agent uses `gbrain search` or `gbrain query` FIRST, not grep
|
|
or external APIs. The response includes brain-sourced context with source
|
|
attribution.
|
|
|
|
**If it fails:** The brain-first lookup protocol isn't injected into the agent's
|
|
system context. See `skills/setup/SKILL.md` Phase D.
|
|
|
|
---
|
|
|
|
## 7. Knowledge Graph Wired
|
|
|
|
The v0.12.0 graph layer needs to be populated for existing brains. New writes are
|
|
auto-linked, but historical pages need a one-time backfill.
|
|
|
|
**Command:**
|
|
|
|
```bash
|
|
gbrain stats | grep -E 'links|timeline'
|
|
```
|
|
|
|
**Expected:** Both `links` and `timeline_entries` are non-zero (assuming the brain
|
|
has content with entity references and dated markdown).
|
|
|
|
**If it's zero on a brain with imported content:** Run the backfill.
|
|
|
|
```bash
|
|
gbrain extract links --source db --dry-run | head -5 # preview
|
|
gbrain extract links --source db # commit
|
|
gbrain extract timeline --source db
|
|
gbrain stats # confirm > 0
|
|
```
|
|
|
|
**Bonus check** — graph traversal works:
|
|
|
|
```bash
|
|
# Pick any well-connected slug from your brain
|
|
gbrain graph-query people/<some-person-slug> --depth 2
|
|
```
|
|
|
|
**Expected:** Indented tree of typed edges (`--attended-->`, `--works_at-->`, etc.).
|
|
If the slug has no inbound or outbound links, try a different one or run extract
|
|
again.
|
|
|
|
**If extract finds nothing:** Your pages may not use entity-reference syntax. The
|
|
extractor matches `[Name](people/slug)`, `[Name](../people/slug.md)`, and bare
|
|
`people/slug` references. If your brain uses a different format, the auto-link
|
|
heuristics won't find them — file an issue with a sample page.
|
|
|
|
---
|
|
|
|
## 8. JSONB Frontmatter Integrity (v0.12.2)
|
|
|
|
Postgres-backed brains created before v0.12.2 had double-encoded JSONB columns
|
|
(`frontmatter->>'key'` returned NULL, GIN indexes were inert). `gbrain upgrade`
|
|
runs `gbrain repair-jsonb` automatically via the `v0_12_2` orchestrator.
|
|
Verify the repair succeeded.
|
|
|
|
**Command:**
|
|
|
|
```bash
|
|
gbrain repair-jsonb --dry-run --json
|
|
```
|
|
|
|
**Expected:** `totalRepaired: 0` across all 5 columns (`pages.frontmatter`,
|
|
`raw_data.data`, `ingest_log.pages_updated`, `files.metadata`,
|
|
`page_versions.frontmatter`). A zero count means every row is properly-typed
|
|
JSON objects, not string-encoded JSON.
|
|
|
|
**If the count is > 0:** The repair didn't run or was interrupted. Re-run
|
|
without `--dry-run`:
|
|
|
|
```bash
|
|
gbrain repair-jsonb
|
|
```
|
|
|
|
Idempotent. PGLite brains always report 0 (unaffected by the original bug).
|
|
|
|
**Bonus check** — frontmatter-keyed queries actually resolve:
|
|
|
|
```bash
|
|
gbrain call list_pages '{"frontmatterKey": "type", "frontmatterValue": "person"}'
|
|
```
|
|
|
|
If this returns rows on a brain with person pages, the JSONB path is healthy.
|
|
|
|
---
|
|
|
|
## Quick Verification (all checks in one pass)
|
|
|
|
```bash
|
|
# 1. Schema
|
|
gbrain doctor --json
|
|
|
|
# 2. Sync recency
|
|
gbrain config get sync.last_run
|
|
|
|
# 3. Page count + embed coverage
|
|
gbrain stats
|
|
|
|
# 4. Search works
|
|
gbrain search "test query from your brain content"
|
|
|
|
# 5. Catch any unembedded chunks
|
|
gbrain embed --stale
|
|
|
|
# 6. Auto-update
|
|
gbrain check-update --json
|
|
|
|
# 7. Knowledge graph populated (links + timeline > 0)
|
|
gbrain stats | grep -E 'links|timeline'
|
|
|
|
# 8. JSONB integrity (v0.12.2 — Postgres only, PGLite always 0)
|
|
gbrain repair-jsonb --dry-run --json
|
|
```
|
|
|
|
If all eight return successfully, the installation is healthy. For the full
|
|
end-to-end sync test (4c), push a real change and verify it appears in search.
|