v0.35.5.0 fix wave: bootstrap + orphans + think MCP + worktree + walker (#1111)
* fix(bootstrap): extend probes for files/oauth_clients/sources.archived* + add MIGRATIONS introspection guard Adds 7 new forward-reference probes to applyForwardReferenceBootstrap on both engines, closes the column-only forward-ref class via a new MIGRATIONS-source introspection contract test. New probes: - files.source_id + files.page_id (v18 forward refs) - oauth_clients.source_id + oauth_clients.federated_read (v60+v61+v65) - sources.archived + archived_at + archive_expires_at (v34 promoted from JSONB) The sources.archived* columns are the codex-flagged class: they're added inline in v34's CREATE TABLE definition but `CREATE TABLE IF NOT EXISTS sources` is a no-op on pre-v34 brains, so downstream visibility filters (search/list_pages) trip on old brains. needsPagesBootstrap now folds archive columns into its CREATE TABLE so pre-v0.18 brains get a v34-shape sources in one go; needsSourcesArchive then only fires on the pre-v34 case (sources exists, archive cols don't). Closes the structural bug class via test/helpers/extract-added-columns.ts: reads src/core/migrate.ts as text and extracts every ALTER TABLE ADD COLUMN. The new contract test asserts every (table, column) pair is covered by EITHER the bootstrap's ALTER TABLE statements, the bootstrap's CREATE TABLE definitions, OR the schema blob's CREATE TABLE bodies. The column-only class (no index, no FK; just an inline CREATE TABLE column the schema blob can't add to existing tables) is now caught at PR time. Source-text introspection catches all three migration shapes uniformly: - top-level `sql:` field - `sqlFor.postgres` / `sqlFor.pglite` overrides - handler-body `engine.runMigration(N, \`ALTER TABLE ...\`)` (v34 shape) Pre-existing parseBaseTableColumns parser bug fixed: now strips `--` line comments and `/* ... */` blocks before identifying column names. Without this, a column preceded by a comment was silently dropped. Catches pages.page_kind and others that were silently uncovered. 13 columns added by migrations but not in PGLITE_SCHEMA_SQL are exempted with a unified rationale: they have no schema-blob forward reference; migration handles all upgrade paths cleanly. Refreshing the schema blob is a separate concern. Issues closed: #1018 (v60 oauth_clients), #974 (files.source_id/page_id), #820 (v0.13.0 migration files.page_id cascade); pre-empts the sources.archived class before any pre-v34 brain trips on it. Tests: - 9 cases in test/schema-bootstrap-coverage.test.ts (5 existing + 4 new) - helper-level unit tests cover SQL shape variants (IF NOT EXISTS, quoted identifiers, ALTER TABLE IF EXISTS ONLY, multi-statement) - planted-bug regression verifies the gate actually catches new uncovered columns Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(orphans): filter soft-deleted pages on both candidate and link-source sides Closes #1021. The v0.26.5 soft-delete invariant requires that findOrphanPages exclude both: 1. Candidate pages that are themselves soft-deleted 2. Inbound links from soft-deleted source pages Pre-fix, findOrphanPages had no deleted_at filter at all. Soft-deleted pages with no inbound links were counted as orphans (inflating counts). Pre-codex-tension-D11, only the candidate-side filter was planned. Codex C11 caught the second case: a live page that has ONE inbound link from a soft-deleted source page was hidden from orphan results — the link still existed in the links table, the EXISTS subquery saw it, the page looked "linked." Now the inner JOIN on pages enforces src.deleted_at IS NULL. Three regression tests pin the contract: - soft-deleted page with no inbound → NOT orphan - live page with ONLY inbound link from soft-deleted source → IS orphan - live page with live inbound → NOT orphan (smoke check that the new filters don't break unchanged behavior) Engine parity: same SQL shape on both Postgres and PGLite engines. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(think): route runThink through gateway.chat adapter (closes #952) Pre-fix, runThink instantiated `new Anthropic()` directly and read ANTHROPIC_API_KEY from process.env. Claude Desktop's stdio MCP launch doesn't inherit shell env, so `gbrain config set anthropic_api_key sk-...` (writes to ~/.gbrain/config.json) never reached the SDK and every MCP think call degraded to "no LLM available." The adapter routes through gateway.chat() — the canonical seam per CLAUDE.md. Gateway reads the API key from gbrain config OR env, picks up prompt caching, rate-leases, retry, and the test seam (__setChatTransportForTests) that v0.31.12 established. Per plan-eng-review D10 (cross-model tension with codex C7+C8+C9+C10), four spec points landed: 1. Drop `new Anthropic()` direct path entirely. Every non-stub LLM call from runThink routes through gateway. 2. Real availability check (NOT a false-positive `getChatModel()` truthy). `tryBuildGatewayClient` probes both the recipe (resolveRecipe throws AIConfigError on unknown providers) AND the API key (reads process.env + loadConfig at the gbrain config layer for parity with gateway's own auth resolution). Returns null on miss; runThink takes the graceful "no LLM available" early-return preserving the legacy NO_ANTHROPIC_API_KEY warning signal. 3. Model-id normalization. resolveModel returns bare anthropic ids (claude-opus-4-7); gateway.chat needs provider:model. Adapter auto-prefixes anthropic: when the id is bare. Provider:model strings pass through unchanged. 4. Response-shape conversion. ChatResult → Anthropic.Message via chatResultToMessage. mapStopReason translates gateway's provider-neutral stop reasons (end / length / tool_calls / refusal / content_filter / other) to Anthropic's stop_reason ('end_turn' / 'max_tokens' / 'tool_use'); refusal/content_filter/other fall through to end_turn (no Anthropic equivalent). Usage tokens pass through. `opts.client` injection preserved (test seam — see ThinkLLMClient). `opts.stubResponse` preserved (pure-test escape). Tests: - test/think-gateway-adapter.test.ts (9 cases): response shape, stop reason mapping, model-id normalization (bare + prefixed), provider unknown returns null, ANTHROPIC_API_KEY absent returns null (regression for legacy graceful degradation), hasAnthropicKey reads process.env correctly. Uses withEnv per the test-isolation contract. - test/think-pipeline.serial.test.ts (17 existing cases): unchanged; the graceful-degradation case at line 213 still produces the NO_ANTHROPIC_API_KEY warning because tryBuildGatewayClient returns null when no key is configured, taking the legacy early-return path. Closes #952. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sync): distinguish git worktree from submodule via path-segment match (closes #889) Pre-fix, `manageGitignore` treated every `.git`-as-file as a submodule and skipped gitignore management. Both submodules AND worktrees use `.git` as a file (not a directory), so the legacy `statSync.isFile()` check couldn't discriminate. Worktrees got misclassified as submodules and their .gitignore wasn't managed. Per plan-eng-review D4 (chose path-segment match over absolute-vs- relative path heuristic): the gitdir path contains: - `/modules/<name>` for submodules (skip — managed by parent repo) - `/worktrees/<name>` for worktrees (MANAGE — first-class repo) Both are documented Git internal layouts, stable across all 4 {relative, absolute} × {modules, worktrees} combinations including the absorbed-submodule edge case from `git submodule absorbgitdirs` (where the submodule's gitdir flips to an absolute path). Malformed `.git` file (no `gitdir:` prefix, IO error) → MANAGE, preserving the pre-#889 catch{} fail-closed-toward-managing semantics. Tests (5 new + 1 regression renamed): - REGRESSION: submodule relative gitdir/modules/ → skip (D49 contract) - absorbed submodule absolute gitdir/modules/ → skip (edge case) - CRITICAL: worktree absolute gitdir/worktrees/ → MANAGE (closes #889) - worktree relative gitdir/worktrees/ → MANAGE - malformed .git file → MANAGE (preserves catch behavior) - regular .git directory → MANAGE (existing smoke) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(walkers): pruneDir helper + descent-time exclusion + transcript predicate (closes #923, #202) Per plan-eng-review D12 (cross-model tension with codex C12+C13), three structural changes: 1. Extract `pruneDir(name)` helper in src/core/sync.ts. Returns false for directory names walkers must NEVER descend into: `node_modules` (latent bug — no leading dot), dot-prefix dirs (`.git`, `.obsidian`, `.raw`, `.cache`, etc.), `ops`, and `*.raw` sidecar dirs (gbrain convention — `people/pedro.raw/` holds raw source for pedro.md). Walkers consult it at descent time BEFORE recursion, saving the IO cost of walking entire vendor / hidden / sidecar subtrees only to filter them at file-emit time. 2. `isSyncable` itself gains the same exclusion set (via pruneDir on each path segment). Closes the latent bug where node_modules markdown files slipped through: `node_modules/some-pkg/README.md` returned true pre-fix because the legacy dot-prefix check only blocked `.node_modules` (with a leading dot), not the actual `node_modules`. CRITICAL regression test in test/sync.test.ts pins the contract per IRON RULE. 3. Two walkers rewritten to use pruneDir at descent + per-walker file predicate at emit: - `walkMarkdownFiles` (src/commands/extract.ts): pruneDir + isSyncable ({strategy:'markdown'}). Pre-fix this walker had ONLY an ad-hoc dot-prefix exclusion and didn't call isSyncable at all — descended into node_modules, emitted markdown files from there, ignored README/ ops/.raw filters. - `listTextFiles` (src/core/cycle/transcript-discovery.ts): pruneDir + own .txt/.md predicate. DOES NOT use isSyncable({strategy:'markdown'}) because transcripts accept .txt and don't share markdown sync's README/ops exclusions (codex C12). Also made RECURSIVE — pre-fix it walked only the top dir, so transcripts in `corpus/2026/` were invisible (codex C14 — descent-time pruning is the right shape but the test would have passed vacuously on a non-recursive walker). Verified blast radius before adding node_modules: every existing isSyncable caller (sync.ts:558-561 sync filter, frontmatter.ts:264 validate, brain-writer.ts:305 reverse-write, import.ts:454 import filter) wants node_modules excluded — this is a latent-bug fix, not a behavior change for any legitimate caller. Tests: - 7 new isSyncable cases including the node_modules CRITICAL regression - 6 new pruneDir cases (node_modules, dot-prefix, ops, *.raw, content dirs that should pass, empty-string default) - Existing extract.test.ts + extract-fs.test.ts unchanged and passing Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(todos): file v0.36.x follow-ups for runThink rewrite + Supabase bootstrap parity Two follow-up TODOs filed during the v0.36 dreamy-thompson wave: 1. runThink full rewrite (D5+D7 from plan-eng-review): drop the ThinkLLMClient indirection now that v0.36 routes through gateway.chat. 12+ tests need migration to __setChatTransportForTests. Blocked by this wave landing. 2. Supabase parity test for applyForwardReferenceBootstrap (codex C6 residual): real Docker Postgres E2E catches schema correctness but not Supabase pooler/direct-pool routing. The probe uses this.sql but PostgresEngine.initSchema chooses a DDL connection; the divergence has caused multiple historical wedges (#699, #820 lineage). Both entries include full context per the CLAUDE.md TODOS-format spec (what, why, pros, cons, blocked-by, plan reference). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(bootstrap): thread DDL connection through applyForwardReferenceBootstrap Codex adversarial review during /ship caught a P1: initSchema selected a DDL connection, took pg_advisory_lock(42) on it, but applyForwardReferenceBootstrap used `this.sql` (the instance pool) inside. Bootstrap probes ran outside the lock scope on a different connection. Failure mode: two concurrent gbrain instances could BOTH enter the bootstrap block on Supabase transaction-pooler setups because the advisory lock was held on a different connection than the one running ALTER TABLE. The pooler's statement_timeout could also kill the probes mid-flight without affecting the lock-holder, leaving an inconsistent schema state. Fix: applyForwardReferenceBootstrap now accepts an optional connection parameter. initSchema passes the DDL conn (the one holding the lock). this.sql remains the fallback for any unit-test path that calls bootstrap directly. PGLite engine doesn't need this change — single connection, no pooler. This was pre-existing (every prior probe used this.sql), but the v0.36 wave is explicitly about fixing the Supabase upgrade-wedge class. Codex's position was correct: don't ship the wave with the underlying connection mismatch still there. The Supabase parity TEST FIXTURE follow-up remains on TODOS.md (test infra needed to PROVE the fix works under real pooler topology), but the bug itself is closed. 15/15 bootstrap tests pass. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.35.5.0) Six-correctness-fix wave: bootstrap forward-ref class (4 issues + 1 pre-empt), orphans soft-delete leak (both sides), runThink → gateway.chat adapter, git worktree vs submodule discriminator, walker pruneDir + descent-time exclusion, plus a Codex-P1 catch during /ship that threaded the DDL connection through applyForwardReferenceBootstrap. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update CLAUDE.md for v0.35.5.0 backend correctness wave Fold v0.35.5.0 file-level annotations into CLAUDE.md: - postgres-engine.ts + pglite-engine.ts: 7 new applyForwardReferenceBootstrap probes (files.source_id/page_id, oauth_clients.source_id/federated_read, sources.archived/archived_at/archive_expires_at) + DDL connection threading - test/schema-bootstrap-coverage.test.ts: new MIGRATIONS-source introspection guard + parseBaseTableColumns comment-stripping fix - src/core/sync.ts: new pruneDir helper + manageGitignore worktree discriminator - src/core/think/index.ts (new entry): runThink gateway adapter for MCP stdio key resolution - src/core/operations.ts (new entry): findOrphanPages soft-delete filter Regenerate llms-full.txt via bun run build:llms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
@@ -232,7 +232,12 @@ export class PostgresEngine implements BrainEngine {
|
||||
// Pre-schema bootstrap: add forward-referenced state the embedded schema
|
||||
// blob requires but that older brains don't have yet (issues #366/#375/
|
||||
// #378/#396 + #266/#357). Idempotent on fresh installs and modern brains.
|
||||
await this.applyForwardReferenceBootstrap();
|
||||
// Threads the DDL connection (same one holding the advisory lock above)
|
||||
// so bootstrap probes run on the locked connection — without this, the
|
||||
// probes ran through `this.sql` (the pooler/instance pool) outside the
|
||||
// lock, opening a concurrent-bootstrap race for Supabase users on the
|
||||
// transaction pooler. Codex P1 finding from v0.36 dreamy-thompson wave.
|
||||
await this.applyForwardReferenceBootstrap(conn);
|
||||
|
||||
await conn.unsafe(sqlText);
|
||||
|
||||
@@ -294,8 +299,13 @@ export class PostgresEngine implements BrainEngine {
|
||||
* `test/schema-bootstrap-coverage.test.ts` (PGLite side) and
|
||||
* `test/e2e/postgres-bootstrap.test.ts` (Postgres side).
|
||||
*/
|
||||
private async applyForwardReferenceBootstrap(): Promise<void> {
|
||||
const conn = this.sql;
|
||||
private async applyForwardReferenceBootstrap(injectedConn?: postgres.Sql): Promise<void> {
|
||||
// Use the caller-provided connection (DDL pool, holding the advisory lock
|
||||
// from initSchema) when available — falls back to this.sql for backward
|
||||
// compatibility with any unit-test path that still calls bootstrap directly.
|
||||
// Production path always passes the DDL conn so bootstrap probes run inside
|
||||
// the same lock scope as SCHEMA_SQL replay.
|
||||
const conn = injectedConn ?? this.sql;
|
||||
|
||||
// Single round-trip probe for every forward-reference target.
|
||||
// current_schema() resolves to whatever search_path the connection uses,
|
||||
@@ -319,6 +329,16 @@ export class PostgresEngine implements BrainEngine {
|
||||
subagent_provider_id_exists: boolean;
|
||||
ingest_log_exists: boolean;
|
||||
ingest_log_source_id_exists: boolean;
|
||||
files_exists: boolean;
|
||||
files_source_id_exists: boolean;
|
||||
files_page_id_exists: boolean;
|
||||
oauth_clients_exists: boolean;
|
||||
oauth_clients_source_id_exists: boolean;
|
||||
oauth_clients_federated_read_exists: boolean;
|
||||
sources_exists: boolean;
|
||||
sources_archived_exists: boolean;
|
||||
sources_archived_at_exists: boolean;
|
||||
sources_archive_expires_at_exists: boolean;
|
||||
}[]>`
|
||||
SELECT
|
||||
EXISTS (SELECT 1 FROM information_schema.tables
|
||||
@@ -356,7 +376,27 @@ export class PostgresEngine implements BrainEngine {
|
||||
EXISTS (SELECT 1 FROM information_schema.tables
|
||||
WHERE table_schema = current_schema() AND table_name = 'ingest_log') AS ingest_log_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'ingest_log' AND column_name = 'source_id') AS ingest_log_source_id_exists
|
||||
WHERE table_schema = current_schema() AND table_name = 'ingest_log' AND column_name = 'source_id') AS ingest_log_source_id_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.tables
|
||||
WHERE table_schema = current_schema() AND table_name = 'files') AS files_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'files' AND column_name = 'source_id') AS files_source_id_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'files' AND column_name = 'page_id') AS files_page_id_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.tables
|
||||
WHERE table_schema = current_schema() AND table_name = 'oauth_clients') AS oauth_clients_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'oauth_clients' AND column_name = 'source_id') AS oauth_clients_source_id_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'oauth_clients' AND column_name = 'federated_read') AS oauth_clients_federated_read_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.tables
|
||||
WHERE table_schema = current_schema() AND table_name = 'sources') AS sources_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'sources' AND column_name = 'archived') AS sources_archived_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'sources' AND column_name = 'archived_at') AS sources_archived_at_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'sources' AND column_name = 'archive_expires_at') AS sources_archive_expires_at_exists
|
||||
`;
|
||||
const probe = probeRows[0]!;
|
||||
|
||||
@@ -386,26 +426,52 @@ export class PostgresEngine implements BrainEngine {
|
||||
// source_id. Old brains have ingest_log without source_id; bootstrap adds
|
||||
// the column before SCHEMA_SQL replay creates the index.
|
||||
const needsIngestLogSourceId = probe.ingest_log_exists && !probe.ingest_log_source_id_exists;
|
||||
// v0.18 (v18): files.source_id + files.page_id added; idx_files_source_id
|
||||
// and idx_files_page_id in SCHEMA_SQL crash without them.
|
||||
const needsFilesBootstrap = probe.files_exists
|
||||
&& (!probe.files_source_id_exists || !probe.files_page_id_exists);
|
||||
// v0.34.1 (v60+v61+v65): oauth_clients.source_id + federated_read added;
|
||||
// FK to sources(id) + GIN index idx_oauth_clients_federated_read in
|
||||
// SCHEMA_SQL crash without them.
|
||||
const needsOauthClientsBootstrap = probe.oauth_clients_exists
|
||||
&& (!probe.oauth_clients_source_id_exists || !probe.oauth_clients_federated_read_exists);
|
||||
// v0.26.5 (v34): sources.archived + archived_at + archive_expires_at added
|
||||
// for soft-delete lifecycle. SCHEMA_SQL's `CREATE TABLE IF NOT EXISTS sources`
|
||||
// is a no-op on pre-existing sources tables (won't add columns), so the
|
||||
// visibility filters in search/list_pages trip on old brains. Bootstrap
|
||||
// closes the gap before any visibility-filter SQL runs.
|
||||
const needsSourcesArchive = probe.sources_exists
|
||||
&& (!probe.sources_archived_exists
|
||||
|| !probe.sources_archived_at_exists
|
||||
|| !probe.sources_archive_expires_at_exists);
|
||||
|
||||
if (!needsPagesBootstrap && !needsLinksBootstrap && !needsChunksBootstrap
|
||||
&& !needsPagesDeletedAt && !needsMcpLogBootstrap && !needsSubagentProviderId
|
||||
&& !needsChunksEmbeddingImage && !needsPagesRecency
|
||||
&& !needsIngestLogSourceId) return;
|
||||
&& !needsIngestLogSourceId && !needsFilesBootstrap
|
||||
&& !needsOauthClientsBootstrap && !needsSourcesArchive) return;
|
||||
|
||||
console.log(' Pre-v0.21 brain detected, applying forward-reference bootstrap');
|
||||
|
||||
if (needsPagesBootstrap) {
|
||||
// Mirror schema-embedded.ts's `sources` shape so the subsequent
|
||||
// SCHEMA_SQL CREATE TABLE IF NOT EXISTS is a true no-op.
|
||||
// Archive columns (v34) are folded in here so a pre-v18 brain doesn't
|
||||
// need needsSourcesArchive to also fire — bootstrap creates a complete
|
||||
// v34-shape sources in one go. needsSourcesArchive then only fires on
|
||||
// the pre-v34 case (sources exists, archive cols don't).
|
||||
await conn.unsafe(`
|
||||
CREATE TABLE IF NOT EXISTS sources (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
local_path TEXT,
|
||||
last_commit TEXT,
|
||||
last_sync_at TIMESTAMPTZ,
|
||||
config JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
local_path TEXT,
|
||||
last_commit TEXT,
|
||||
last_sync_at TIMESTAMPTZ,
|
||||
config JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
archived BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
archived_at TIMESTAMPTZ,
|
||||
archive_expires_at TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
INSERT INTO sources (id, name, config)
|
||||
VALUES ('default', 'default', '{"federated": true}'::jsonb)
|
||||
@@ -518,6 +584,50 @@ export class PostgresEngine implements BrainEngine {
|
||||
ALTER TABLE ingest_log ADD COLUMN IF NOT EXISTS source_id TEXT NOT NULL DEFAULT 'default';
|
||||
`);
|
||||
}
|
||||
|
||||
if (needsFilesBootstrap) {
|
||||
// v18 (files_provenance_columns) adds source_id + page_id to files plus
|
||||
// idx_files_source_id and idx_files_page_id in SCHEMA_SQL. Pre-v18 brains
|
||||
// crash on the CREATE INDEX. Bootstrap adds both columns; v18 runs later
|
||||
// via runMigrations and is idempotent.
|
||||
await conn.unsafe(`
|
||||
ALTER TABLE files ADD COLUMN IF NOT EXISTS source_id TEXT
|
||||
NOT NULL DEFAULT 'default' REFERENCES sources(id) ON DELETE CASCADE;
|
||||
ALTER TABLE files ADD COLUMN IF NOT EXISTS page_id INTEGER
|
||||
REFERENCES pages(id) ON DELETE SET NULL;
|
||||
`);
|
||||
}
|
||||
|
||||
if (needsOauthClientsBootstrap) {
|
||||
// v60+v61+v65 (oauth_clients_source_id_fk, oauth_clients_federated_read_column,
|
||||
// oauth_clients_federated_read_gin_index) add source_id + federated_read
|
||||
// and the GIN index idx_oauth_clients_federated_read. SCHEMA_SQL's
|
||||
// FK + index references crash on pre-v60 brains. Bootstrap mirrors the
|
||||
// v60+v61 column shape; v60-v65 run later via runMigrations and are
|
||||
// idempotent (and handle backfill + the v64 RESTRICT-flip).
|
||||
await conn.unsafe(`
|
||||
ALTER TABLE oauth_clients ADD COLUMN IF NOT EXISTS source_id TEXT
|
||||
DEFAULT 'default' REFERENCES sources(id) ON DELETE SET NULL;
|
||||
ALTER TABLE oauth_clients ADD COLUMN IF NOT EXISTS federated_read TEXT[]
|
||||
NOT NULL DEFAULT '{}';
|
||||
`);
|
||||
}
|
||||
|
||||
if (needsSourcesArchive) {
|
||||
// v34 (destructive_guard_columns) promotes archive lifecycle from JSONB
|
||||
// config to real columns on sources. SCHEMA_SQL's `CREATE TABLE IF NOT EXISTS
|
||||
// sources` is a no-op against an existing pre-v34 sources table, so the
|
||||
// column-add never lands until the v34 migration runs. v34's UPDATE
|
||||
// statements + downstream visibility filters (search/query/list_pages)
|
||||
// need the columns to exist on the table schema. Bootstrap adds the
|
||||
// three columns; v34 runs later via runMigrations and is idempotent
|
||||
// (and handles JSONB → column backfill).
|
||||
await conn.unsafe(`
|
||||
ALTER TABLE sources ADD COLUMN IF NOT EXISTS archived BOOLEAN NOT NULL DEFAULT FALSE;
|
||||
ALTER TABLE sources ADD COLUMN IF NOT EXISTS archived_at TIMESTAMPTZ;
|
||||
ALTER TABLE sources ADD COLUMN IF NOT EXISTS archive_expires_at TIMESTAMPTZ;
|
||||
`);
|
||||
}
|
||||
}
|
||||
|
||||
async transaction<T>(fn: (engine: BrainEngine) => Promise<T>): Promise<T> {
|
||||
@@ -1901,15 +2011,25 @@ export class PostgresEngine implements BrainEngine {
|
||||
|
||||
async findOrphanPages(): Promise<Array<{ slug: string; title: string; domain: string | null }>> {
|
||||
const sql = this.sql;
|
||||
// Soft-delete filter on BOTH sides:
|
||||
// - candidate: p.deleted_at IS NULL — soft-deleted pages aren't orphan candidates
|
||||
// - link source: src.deleted_at IS NULL — links FROM soft-deleted pages don't count as inbound
|
||||
// Without the link-source filter, a live page can hide from orphan results purely
|
||||
// because a soft-deleted page links to it. v0.26.5 invariant; codex C11.
|
||||
const rows = await sql`
|
||||
SELECT
|
||||
p.slug,
|
||||
COALESCE(p.title, p.slug) AS title,
|
||||
p.frontmatter->>'domain' AS domain
|
||||
FROM pages p
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM links l WHERE l.to_page_id = p.id
|
||||
)
|
||||
WHERE p.deleted_at IS NULL
|
||||
AND NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM links l
|
||||
JOIN pages src ON src.id = l.from_page_id
|
||||
WHERE l.to_page_id = p.id
|
||||
AND src.deleted_at IS NULL
|
||||
)
|
||||
ORDER BY p.slug
|
||||
`;
|
||||
return rows as unknown as Array<{ slug: string; title: string; domain: string | null }>;
|
||||
|
||||
Reference in New Issue
Block a user