From 488e4824e84d749a075bd3761b36c13e526bfbf8 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Thu, 14 May 2026 20:15:29 -0700 Subject: [PATCH] =?UTF-8?q?v0.34.1.0=20fix(mcp):=20MCP=20fix=20wave=20?= =?UTF-8?q?=E2=80=94=20source-isolation=20P0=20+=20PKCE=20DCR=20+=20federa?= =?UTF-8?q?ted=5Fread=20+=203=20more=20(#996)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(mcp): skip stdin EOF handlers when MCP_STDIO=1 OpenClaw's bundle-mcp gateway and similar wrappers pipe the JSON-RPC handshake on stdin then close their stdin half. Pre-fix, both stdin 'end' and 'close' listeners (server.ts:65-66 and serve.ts:204-206) treated this as a permanent disconnect and shut the server down before the first tool call arrived. Guard both sites with `process.env.MCP_STDIO !== '1'`. Signal handlers (SIGTERM/SIGINT/SIGHUP), transport.onclose, and the parent-process watchdog still cover legitimate shutdown paths. The serve.ts site threads the env read through an injectable `mcpStdio?: boolean` on ServeOptions so tests stay isolated (no process.env mutation per scripts/check-test-isolation.sh R1). Tests: 3 new cases in test/serve-stdio-lifecycle.test.ts pin the guard's invariants — mcpStdio=true must NOT trigger shutdown on stdin EOF, signals must still drive shutdown with mcpStdio=true, and mcpStdio=false (default) preserves existing CLI behavior. 25/25 pass. Origin: PR #870. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(oauth): honor token_endpoint_auth_method=none for PKCE public clients RFC 7591 §3.2.1: when a DCR client declares token_endpoint_auth_method="none" (PKCE-only public clients like Claude Code, Cursor), the authorization server MUST NOT issue a client_secret. Pre-fix, registerClient unconditionally minted a secret, and the MCP SDK's clientAuth middleware then rejected valid public-client flows on /token because it expected client.client_secret to match. Three changes to src/core/oauth-provider.ts:registerClient: - Gate clientSecret generation on isPublicClient = (auth_method === 'none'). Public clients store client_secret_hash = NULL. - Omit client_secret from the response payload for public clients. Confidential clients (default client_secret_post and explicit client_secret_basic) keep their existing one-time-reveal shape. - Normalize NULL secret_hash to JS undefined in getClient so SDK middleware (which checks client.client_secret === undefined, not === null) correctly identifies public clients and skips the secret-comparison branch on /token. Schema is already permissive (client_secret_hash TEXT, no NOT NULL on both src/schema.sql and src/core/pglite-schema.ts) — no migration needed. Tests: 5 new cases in test/oauth.test.ts pin: - public client → no client_secret in response (#11 from plan) - default auth_method → secret unchanged (regression guard) - explicit client_secret_post → secret unchanged - getClient NULL→undefined normalization - PKCE full /authorize → /token end-to-end with no secret (#15 from plan) 69/69 oauth.test.ts cases pass. typecheck clean. Origin: PR #909. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(serve-http): --bind HOST, default to loopback (127.0.0.1) Adds `gbrain serve --http --bind ` to control which network interface the HTTP MCP server listens on. Default flipped from `0.0.0.0` (pre-v0.34) to `127.0.0.1` (v0.34.0+). Why the flip: gbrain's primary use case is a personal-knowledge brain on a laptop. The previous default exposed brains on every interface — one accidental `--http` invocation away from publishing the brain to a LAN. Server operators who need remote access pass `--bind 0.0.0.0` (or a specific interface). Codex's outside-voice on the original PR #864 correctly flagged that the additive flag wasn't actually the fix; the default needed to change for the safety claim to hold. If `--public-url` is set but `--bind` is unset, runServeHttp prints a loud stderr WARN at startup recommending `--bind 0.0.0.0`. Declaring a public URL while quietly binding loopback is almost always a misconfiguration; we want the operator to see it on first start, not silently fail remote requests. Startup banner now includes a `Bind:` row so the listening interface is visible alongside Port / Engine / Issuer. Origin: PR #864, extended with D11 (default flip) per /plan-eng-review codex outside-voice review. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(mcp): seal source-isolation leak on read path (P0) Pre-fix, an authenticated OAuth MCP client scoped to source-A could enumerate source-B pages via six read-side ops: search, query (text AND image paths), list_pages, traverse_graph, and find_experts. The v0.31.8 source-scoping pattern shipped through dispatch.ts but the op handlers never threaded ctx.sourceId into their engine calls, and hybridSearch.ts:223's explicit SearchOpts rebuild dropped sourceId even when callers passed it. Sealing the leak: - src/core/operations.ts adds sourceScopeOpts(ctx), the canonical precedence ladder: ctx.auth.allowedSources (federated) wins over ctx.sourceId (scalar) wins over nothing. Threaded into all 5 read-side op handlers + the query-image-path searchVector call (the 6th leak surface codex caught in plan review). - src/core/search/hybrid.ts:223 now threads sourceId + sourceIds fields through the inner SearchOpts rebuild. The explicit pick shape is preserved (HNSW inner-CTE ordering depends on it) but extended. - src/core/types.ts adds sourceIds?: string[] to SearchOpts + PageFilters (D9: federated read needs array-shaped engine filter or fan-out; array wins for hot retrieval). - src/core/operations.ts AuthInfo gains sourceId + allowedSources (D2: identity surface symmetric with the federated_read column #876 will add). - Both engines now apply WHERE source_id = $N (scalar) or = ANY($N::text[]) (array) at the SQL layer for searchKeyword, searchKeywordChunks, searchVector, listPages, traverseGraph, traversePaths. Array form wins when both are set. The searchVector filter pushes into the inner HNSW CTE (codex flagged this placement during plan review). - traverseGraph + traversePaths signatures gain opts.sourceId + opts.sourceIds; engine.ts interface updated. - findExperts (the whoknows op, D3 5th leak surface) accepts sourceId + sourceIds and threads them into its internal hybridSearch call. PR #861 was authored before v0.33 shipped so this op wasn't covered in the original PR. Auth wiring: - GBrainOAuthProvider.verifyAccessToken populates AuthInfo.sourceId from oauth_clients.source_id. JOIN guarded by isUndefinedColumnError so pre-v55 brains degrade to legacy projection rather than refusing every token verification. - GBrainOAuthProvider.registerClientManual gains a sourceId parameter (defaults to 'default'). DCR registerClient also sets source_id='default' on the inserted row. - serve-http.ts:929 cleanup: AuthInfo.sourceId is now a real typed field. The cast + GBRAIN_SOURCE env fallback chain is gone (D13). Legacy bearer tokens default to 'default' source in verifyAccessToken. - http-transport.ts (legacy access_tokens path) threads sourceId='default' through DispatchOpts so v0.22.7 callers stay source-scoped. - auth.ts CLI adds --source flag to gbrain auth register-client. Migration v55 (D10 + D13): - ALTER TABLE oauth_clients ADD COLUMN source_id TEXT (nullable). - Backfill UPDATE source_id = 'default' WHERE source_id IS NULL — preserves v0.33 effective behavior verbatim for legacy clients. - ADD CONSTRAINT FK ... REFERENCES sources(id) ON DELETE SET NULL, wrapped in DO block so re-runs against fresh-install brains (where the FK already lives inline in SCHEMA_SQL) no-op cleanly. - CREATE INDEX idx_oauth_clients_source_id WHERE source_id IS NOT NULL for the verifyAccessToken JOIN. - GBRAIN_ACCEPT_SILENT_WIDEN env-flag wired through the runner via SET LOCAL gbrain.accept_silent_widen — reserved for future migrations that hit the silent-widen footgun codex flagged. This migration doesn't need it (column is brand new; no pre-existing stale values possible by definition). - src/core/pglite-schema.ts + src/schema.sql include the column + FK + index inline for fresh installs. Tests: new test/e2e/source-isolation-pglite.test.ts with 13 regression cases — one per leak surface (search/list_pages/traverse/etc.) plus explicit AuthInfo.sourceId and AuthInfo.allowedSources op-handler threading checks. Full unit suite: 6034 pass / 0 fail. PGLite initSchema time dropped from 2.4s to 850ms after consolidating v55's DO blocks (multiple DO blocks were slow on PGLite; one DO block for the FK install only is fine). Origin: PR #861 + plan-eng-review decisions D2/D3/D4/D9/D10/D13 + F2. Co-Authored-By: Claude Opus 4.7 (1M context) * feat(gateway): multimodal embedding for openai-compatible providers Pre-fix, embedMultimodal hardcoded a recipe.id === 'voyage' branch and threw AIConfigError for every other recipe. Multimodal-capable providers fronted by LiteLLM (or any openai-compatible proxy) were unreachable even when the operator had wired up the model. The fix: - src/core/ai/gateway.ts adds embedMultimodalOpenAICompat() that POSTs to the standard /embeddings endpoint with content arrays carrying image_url entries. Routing comes from the existing recipe.implementation switch — Voyage stays on its own /multimodalembeddings path; every other openai-compatible recipe flows through the new helper. - src/core/ai/recipes/litellm-proxy.ts declares supports_multimodal: true so embedMultimodal accepts the recipe. No multimodal_models allow-list: LiteLLM is a passthrough proxy and the user owns model-id selection; provider rejection (400 from upstream) is the right enforcement layer there. Voyage's static allow-list shape stays unchanged (its 12 models share supports_multimodal but only one is multimodal-capable). - D12 runtime dimension validation: the new helper checks the returned vector length against the recipe's declared default_dims (preferred) or the brain's embedding_dimensions config. Mismatch throws AIConfigError with model id + observed + expected so the operator can swap models or rebuild the column. Pre-fix, a wrong-dim response would surface as a cryptic pgvector "vector dimension mismatch" at INSERT time. - Auth resolution routes through the existing defaultResolveAuth helper so optional-auth recipes (LiteLLM proxy with no LITELLM_API_KEY) and required-auth recipes both share one code path. Optional-auth sends "Authorization: Bearer unauthenticated" which servers like Ollama / llama-server ignore but the SDK contract requires. Tests: 11 new cases in test/openai-compat-multimodal.test.ts cover happy-path, multi-input batching, unauthenticated proxy, D12 dim mismatch + default-dim fallback, 401 / 400 / malformed-JSON / non-array error paths, and an explicit Voyage-regression test pinning that the new openai-compat route doesn't accidentally hijack the Voyage path. All 41 multimodal-related tests pass (existing voyage suite + new). typecheck clean. Origin: PR #875 + plan-eng-review D12 (runtime dim validation). Co-Authored-By: Claude Opus 4.7 (1M context) * feat(oauth): federated_read read scope (#876) Pre-fix, OAuth clients had a single source-scope axis (source_id, added in v55). A client could either write+read one source OR be a super-reader across all sources (via NULL source_id). There was no middle ground — WeCare-style L3 dept clients that need to write to dept-x but read dept-x + parent canon + shared canon had no expression. #876 adds federated_read TEXT[] as an orthogonal read-scope axis. source_id is the WRITE authority; federated_read is the READ authority. They default to matching values (read scope == write scope, the pre-v0.34 default) when a client is registered without an explicit federated read list. Migrations v56-v60 (six new migrations on top of v55): - v56: ALTER TABLE ... ADD COLUMN federated_read TEXT[] NOT NULL DEFAULT '{}'. - v57 (F5): explicit CASE backfill so source_id IS NULL → '{}' (not an array containing NULL — codex caught this ambiguity during plan review). - v58: post-backfill validation. Fails loud if any row's source_id isn't in its federated_read array, pointing at a logic bug in v57 if fired. - v59: flip the source_id FK from ON DELETE SET NULL to ON DELETE RESTRICT now that federated_read provides the alternative scope-loss path. Pre-flip, deleting a source could silently widen any oauth_client to super-reader; post-flip, source delete is refused if any client references it (operator must revoke/re-scope first). - v60: GIN index on federated_read for array-containment queries. Auth wiring: - GBrainOAuthProvider.verifyAccessToken JOINs c.federated_read and populates AuthInfo.allowedSources. Pre-v56 / pre-v55 brains degrade via the existing isUndefinedColumnError fallback chain. - registerClientManual gains a federatedRead?: string[] parameter (defaults to [sourceId]). - DCR registerClient sets source_id='default' + federated_read=['default'] on the inserted row. - auth.ts CLI adds --federated-read SRC1,SRC2,... flag. The register-client output now prints "Federated reads:" so operators confirm the scope they set. Engines consume the federated array through the SearchOpts.sourceIds / PageFilters.sourceIds field that #861 added (no engine changes here — the plumbing was D9). sourceScopeOpts in operations.ts already prefers the auth.allowedSources array over scalar ctx.sourceId when set. Test seam: - test/book-mirror.test.ts now spawns the CLI with GBRAIN_HOME pointed at a tempdir so the test isn't sensitive to the developer's local ~/.gbrain/config.json. Pre-fix the test could silently inherit a real Postgres connection and hang past the default 5s test timeout. Fresh GBRAIN_HOME → "No brain configured" → exit 1 in <1s. - test/e2e/source-isolation-pglite.test.ts gains one more regression case: AuthInfo.allowedSources = [] (explicit empty) MUST NOT widen scope to "all sources" — the silent-widen footgun precedence ladder. - test/openai-compat-multimodal.test.ts is part of the wave's commits via the migrate.ts changes that bump the schema chain. typecheck-only fix on a captured-auth type was already in #875's tree. 6045 unit tests pass / 0 fail. typecheck clean. PGLite initSchema runs v55-v60 in ~786ms total (within the test-harness budget for tests using the canonical beforeAll engine pattern). Origin: PR #876 + plan-eng-review F5 (CASE backfill). Co-Authored-By: Claude Opus 4.7 (1M context) * v0.34.0.0: MCP fix wave (#870 #909 #864 #861 #875 #876) VERSION + package.json + CHANGELOG bump for the six-PR MCP fix wave. Schema chain extends from v54 → v60; oauth_clients gains source_id + federated_read columns; auth'd MCP clients now stay inside their scope across all read-side ops; PKCE-only DCR works; --bind defaults to loopback; LiteLLM multimodal embedding ships. Contributed by @Hansen1018 (#870), @ding-modding (#909), @DukeDawg (#864), @toilalesondev (#861 + #876), @yoelgal (#875). Co-Authored-By: Claude Opus 4.7 (1M context) * docs: update project documentation for v0.34.0.0 Sync README, CLAUDE.md, SECURITY.md, docs/architecture/topologies.md, and docs/mcp/DEPLOY.md to reflect the v0.34.0.0 MCP fix wave: - README: document --bind HOST default (loopback), --source + --federated-read register-client flags, PKCE public-client gate - SECURITY.md: note loopback-by-default for serve --http, update the trust-proxy contract to point at the new default - CLAUDE.md: annotate operations.ts (sourceScopeOpts helper), oauth-provider.ts (verifyAccessToken JOIN + PKCE public clients), serve-http.ts (--bind flag), gateway.ts (openai-compat multimodal + dim validation), mcp/server.ts (MCP_STDIO guard), auth.ts (--source + --federated-read), migrate.ts (v58-v63 chain), engine.ts (sourceIds field). Add 4 new test-file entries for source-isolation-pglite, openai-compat-multimodal, serve-stdio-lifecycle, oauth.test.ts PKCE cases - docs/architecture/topologies.md: source-scoped register-client example, --bind 0.0.0.0 for thin-client host setup - docs/mcp/DEPLOY.md: --bind explanation in the ngrok section, source-scoped client recipe - llms-full.txt: regenerated per the CLAUDE.md-edit chaser rule Co-Authored-By: Claude Opus 4.7 (1M context) * chore: bump v0.34.0.0 → v0.34.1.0 Renumbering the MCP fix wave from v0.34.0.0 to v0.34.1.0 so the release slot lands between master's v0.33.2.1 and the next minor. Touches every release-artifact mention: - VERSION: 0.34.0.0 → 0.34.1.0 - package.json: same - CHANGELOG.md header + "To take advantage" block - CLAUDE.md key-files annotations (8 entries that document this wave) - llms-full.txt (regen from CLAUDE.md) - README.md / SECURITY.md / docs/architecture/topologies.md / docs/mcp/DEPLOY.md - Wave code-comment markers ("// v0.34.0 (#NNN):" → "// v0.34.1 (#NNN):") Test files renamed alongside since they were committed with the wave. Commit subjects on the original 6 PR commits + the v0.34.0.0 bump commit (4f533c72 → 6b47db7e) intentionally NOT rewritten — those are history. `git log` finds the implementation by message subject, not by version tag. 6275 unit tests pass, typecheck clean, migration chain v58-v63 unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) --------- Co-authored-by: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 239 ++++++++++++++++++++ CLAUDE.md | 20 +- README.md | 8 +- SECURITY.md | 18 +- TODOS.md | 16 ++ VERSION | 2 +- docs/architecture/topologies.md | 11 +- docs/mcp/DEPLOY.md | 30 +++ llms-full.txt | 58 ++++- package.json | 2 +- src/commands/auth.ts | 30 ++- src/commands/serve-http.ts | 41 +++- src/commands/serve.ts | 29 ++- src/commands/whoknows.ts | 19 ++ src/core/ai/gateway.ts | 156 ++++++++++++- src/core/ai/recipes/litellm-proxy.ts | 9 + src/core/engine.ts | 17 +- src/core/migrate.ts | 181 +++++++++++++++ src/core/oauth-provider.ts | 247 ++++++++++++++++++--- src/core/operations.ts | 101 ++++++++- src/core/pglite-engine.ts | 117 +++++++++- src/core/pglite-schema.ts | 11 + src/core/postgres-engine.ts | 131 ++++++++++- src/core/schema-embedded.ts | 9 + src/core/search/hybrid.ts | 10 + src/core/types.ts | 21 ++ src/mcp/http-transport.ts | 17 ++ src/mcp/server.ts | 11 +- src/schema.sql | 9 + test/book-mirror.test.ts | 50 +++-- test/e2e/source-isolation-pglite.test.ts | 267 +++++++++++++++++++++++ test/oauth.test.ts | 103 +++++++++ test/openai-compat-multimodal.test.ts | 248 +++++++++++++++++++++ test/serve-stdio-lifecycle.test.ts | 61 ++++++ 34 files changed, 2187 insertions(+), 112 deletions(-) create mode 100644 test/e2e/source-isolation-pglite.test.ts create mode 100644 test/openai-compat-multimodal.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 74257b4..644ad5e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,245 @@ All notable changes to GBrain will be documented in this file. +## [0.34.1.0] - 2026-05-14 + +**MCP hardening wave: stricter source isolation on the read path, PKCE +DCR works, loopback-by-default, and federated read scopes for shared +brains. Six community PRs land as one release.** + +The v0.34.1 wave consolidates six community PRs into a single ship. +Source-isolation tightening is the centerpiece — an authenticated OAuth +client scoped to one source no longer sees rows from neighboring sources +through the read path. The wave also seals smaller papercuts that have +been costing real users: gateway-managed stdio MCP servers no longer +exit on the post-handshake EOF, PKCE-only DCR clients can register +without inheriting a phantom secret, and `gbrain serve --http` defaults +to loopback so a personal-laptop brain isn't one accidental config away +from publishing itself to the LAN. Federated_read adds the read-scope +axis that shared-brain deployments need: a department client can write +to one source and read across a curated set without becoming a +super-reader. + +### What you can now do + +**Scope OAuth clients to a single source.** `gbrain auth register-client +my-agent --source dept-x` registers a client whose write authority is +`dept-x`. Read paths only return rows whose `source_id` matches. The +auth-layer thread is on `verifyAccessToken` so every per-request +dispatch starts with the scope in `AuthInfo.sourceId`; ops consume it +through the canonical `ctx.sourceId` pattern that v0.31.8 established. +Pre-v0.34 clients without an explicit source default to `default` on +upgrade — the v0.33 effective behavior is preserved verbatim. + +**Federate read scope independently.** `gbrain auth register-client +my-l3-dept --source dept-x --federated-read dept-x,wecare,shared` +registers a client that writes to `dept-x` but reads from the union of +`dept-x`, `wecare` parent canon, and `shared` org canon. The two scope +axes are orthogonal — `source_id` is the write authority, `federated_read` +is the read authority. Engine read paths apply +`WHERE source_id = ANY($1::text[])` at SQL when the array is set; scalar +single-source clients keep the v0.31.12 fast path. + +**Run gateway-piped stdio MCP without race-killing your server.** Set +`MCP_STDIO=1` in the env and the server skips the stdin EOF shutdown +hooks. Signal handlers (SIGTERM / SIGINT / SIGHUP) and parent-process +watchdog still cover legitimate disconnects. OpenClaw's bundle-mcp +gateway and similar wrappers pipe the handshake then close their stdin +half; pre-fix this killed the server before the first tool call landed. + +**Register PKCE-only public clients.** `POST /register` with +`token_endpoint_auth_method: "none"` (Claude Code, Cursor, every other +PKCE-first MCP client) now returns RFC 7591-compliant response shape: +no `client_secret` field on public clients, secret-bearing on default +`client_secret_post` clients. The `/token` exchange accepts the public +client through the SDK's clientAuth path because `getClient` correctly +normalizes a NULL `client_secret_hash` to JS undefined. + +**Bind the HTTP MCP server to loopback by default.** `gbrain serve +--http` now listens on `127.0.0.1` unless you pass `--bind 0.0.0.0` (or +a specific interface IP). Personal-laptop installs are no longer one +default-config away from publishing the brain to a LAN. Self-hosted +server operators who actually want remote access pass `--bind 0.0.0.0` +once; a stderr WARN fires when `--public-url` is set without `--bind` so +the operator doesn't silently bind loopback at startup. + +**Embed images through LiteLLM / openai-compatible multimodal models.** +The gateway's `embedMultimodal` no longer hardcodes Voyage; recipes with +`implementation: 'openai-compatible'` route through the standard +`/embeddings` endpoint with content arrays carrying `image_url` entries. +Runtime dimension validation throws a clear error pre-storage if the +provider returns a vector that doesn't match the brain's embedding +column width — no more cryptic `vector dimension mismatch` at INSERT +time. + +### The migration block + +`gbrain upgrade` applies migrations v60-v65 automatically. The migration +chain adds two columns to `oauth_clients` and an index, plus a FK flip +once federated_read is in place. + +### To take advantage of v0.34.1 + +`gbrain upgrade` should do this automatically. If it didn't, or if +`gbrain doctor` warns about a partial migration: + +1. **Run the orchestrator manually:** + ```bash + gbrain apply-migrations --yes + ``` +2. **Verify the schema landed:** + ```bash + gbrain doctor --json | jq '.checks.schema_version' + # version should be >= 60 + ``` +3. **Existing OAuth clients keep working.** Pre-v0.34 clients without an + explicit source scope are backfilled to `source_id='default'` so + their effective scope matches v0.33. To narrow a specific client's + scope, re-register it with `--source `: + ```bash + gbrain auth revoke-client + gbrain auth register-client --source --scopes read,write + ``` +4. **`gbrain serve --http` default changed.** Existing self-hosted + server deployments must add `--bind 0.0.0.0` to keep accepting remote + connections. Personal-laptop users see no behavior change (loopback + is now the default). +5. **If `gbrain apply-migrations` refuses with an `oauth_clients` stale + source_id error** (possible only if an operator hand-poked the column + before upgrading): the error message names the offending client IDs. + Either revoke + re-register them with a valid source, or re-run with + `GBRAIN_ACCEPT_SILENT_WIDEN=1` to NULL the stale values (widens those + clients to super-reader; re-scope via `gbrain auth register-client` + after). +6. **If any step fails or the numbers look wrong,** file an issue: + https://github.com/garrytan/gbrain/issues with `gbrain doctor --json` + output and the failing step. + +### Itemized changes + +**Source-isolation hardening:** +- `OperationContext.auth` now carries `AuthInfo.sourceId` (write scope) + and `AuthInfo.allowedSources` (federated read scope), threaded from + `oauth_clients` rows at token-verification time. +- New helper `sourceScopeOpts(ctx)` in `src/core/operations.ts` encodes + the precedence ladder: federated array wins over scalar over nothing. + Every read-side op handler routes through it so future ops can't + silently drift from the canonical thread. +- `src/core/search/hybrid.ts` inner `SearchOpts` rebuild now includes + `sourceId` + `sourceIds` fields — the structural fix that prevents + the explicit-pick footgun that motivated the wave. +- `src/core/types.ts` adds `sourceIds?: string[]` to `SearchOpts` and + `PageFilters` for the federated read axis. Both Postgres and PGLite + engines apply `WHERE source_id = ANY($N::text[])` when the array is + set; scalar fast path preserved when unset. +- Engine method signatures `traverseGraph(slug, depth, opts?)` and + `traversePaths(slug, opts?)` accept `opts.sourceId` / + `opts.sourceIds` so graph walks respect the caller's scope. +- `src/core/oauth-provider.ts:verifyAccessToken` JOINs + `oauth_clients.source_id` + `federated_read` and surfaces both on the + returned `AuthInfo`. Pre-v60 / pre-v61 brains degrade gracefully via + `isUndefinedColumnError` fallback. +- `src/commands/serve-http.ts` drops the `(authInfo as AuthInfo & + {sourceId?: string}).sourceId ?? env ?? 'default'` cast chain. The + typed field is the source of truth now. +- Legacy `src/mcp/http-transport.ts` (v0.22.7-style access_tokens path) + threads `sourceId: 'default'` through DispatchOpts so legacy tokens + stay source-scoped. + +**Migration chain v60-v65 (six new migrations on top of v54):** +- v60 (`oauth_clients_source_id_fk`) — ALTER TABLE ... ADD COLUMN + source_id TEXT, backfill NULL→'default', install FK with + ON DELETE SET NULL. +- v61 (`oauth_clients_federated_read_column`) — ALTER TABLE ... ADD + COLUMN federated_read TEXT[] NOT NULL DEFAULT '{}'. +- v62 (`oauth_clients_federated_read_backfill`) — explicit CASE backfill + so `source_id IS NULL` produces `'{}'` not an array-containing-NULL. +- v63 (`oauth_clients_federated_read_validate`) — fail-loud check that + every row's source_id is in its federated_read array post-backfill. +- v64 (`oauth_clients_source_id_fk_restrict`) — flip FK to + ON DELETE RESTRICT now that federated_read provides the alternative + scope-loss path. Source delete is refused if any client references it. +- v65 (`oauth_clients_federated_read_gin_index`) — GIN index for the + array-containment queries the read paths run. + +**OAuth + auth surface:** +- `auth.ts` CLI adds `--source ` and `--federated-read ` + flags to `register-client`. The output now prints the resolved + `Write source` and `Federated reads` for the registered client. +- DCR `/register` endpoint now writes `source_id='default'` and + `federated_read=['default']` on the inserted row so new public clients + start in a sane scope. +- `registerClient` honors `token_endpoint_auth_method: "none"` (RFC 7591 + §3.2.1): public clients store `client_secret_hash = NULL` and the + response payload omits `client_secret` entirely. Confidential clients + (default `client_secret_post` and explicit `client_secret_basic`) keep + their one-time-reveal shape. + +**MCP transports:** +- `src/mcp/server.ts` + `src/commands/serve.ts` skip stdin + `'end'`/`'close'` shutdown hooks when `process.env.MCP_STDIO === '1'`. + `ServeOptions` gains a `mcpStdio?: boolean` test seam so the runtime + guard is exercisable without process.env mutation. +- `src/commands/serve-http.ts` adds `--bind HOST` (default `127.0.0.1`). + Stderr WARN fires when `--public-url` is set without `--bind`. + Startup banner prints the resolved `Bind:` line. + +**Multimodal embedding:** +- `gateway.ts` adds `embedMultimodalOpenAICompat()` that POSTs to the + standard `/embeddings` endpoint with content arrays. Routes by + `recipe.implementation === 'openai-compatible'` so LiteLLM-fronted + providers (Anyscale, vLLM, Gemini multimodal via proxy) work alongside + Voyage's existing `/multimodalembeddings` path. +- `recipes/litellm-proxy.ts` declares `supports_multimodal: true` so the + recipe accepts multimodal calls without a model allow-list (LiteLLM is + a passthrough; user-owned model id selection). +- Runtime dimension validation: the returned vector length is checked + against the recipe's declared `default_dims` or the brain's + `embedding_dimensions` config. Mismatch throws `AIConfigError` with + model id + observed + expected before the vector reaches storage. + +**Tests:** +- `test/e2e/source-isolation-pglite.test.ts` — 14 cases pinning the + scope filter at the engine layer plus op-handler threading for both + `ctx.sourceId` and `ctx.auth.allowedSources` paths. +- `test/openai-compat-multimodal.test.ts` — 11 cases covering the + openai-compatible multimodal path: happy-path single + multi-input, + unauthenticated proxy, D12 dim mismatch + default-dim fallback, + 401 / 400 / malformed-JSON / non-array error paths, and a Voyage + regression test. +- `test/oauth.test.ts` — 5 new cases for PKCE DCR public-client gate + (no secret for public; secret unchanged for default; getClient + NULL→undefined normalization; full PKCE `/authorize` → `/token` + round-trip). +- `test/serve-stdio-lifecycle.test.ts` — 3 new cases for the + `MCP_STDIO=1` guard (stdin EOF does NOT trigger shutdown; SIGTERM + still does; unset env preserves CLI behavior). +- `test/book-mirror.test.ts` — `runCli` helper now uses a fresh + `GBRAIN_HOME` tempdir so the test isn't sensitive to the developer's + local `~/.gbrain/config.json`. Pre-fix this could silently inherit a + real Postgres connection and hang past the default 5s test timeout. + +**Test results: 6045 unit tests pass / 0 fail. typecheck clean. PGLite +initSchema runs the v60-v65 chain in ~786ms total.** + +### For contributors + +- The wave bundled six community PRs (#870, #909, #864, #861, #875, + #876) with cross-model plan review (codex outside-voice) before + cherry-pick. Codex caught three structural bugs the original PRs + missed: a 6th source-isolation leak surface (the `query` image path + in `operations.ts:1071-1082`), a 5th read-side op missing from #861's + thread (`find_experts`), and migration-numbering collisions on + v47-v53 that would have wedged on cherry-pick (the branch had grown + to v57 by ship time after master shipped its own v55-v57 search-lite + migrations). The wave's migration chain renumbers to v60-v65. +- The single PR carries `Co-Authored-By:` for the six external + contributors. PRs originated against earlier base branches and the + diffs were re-implemented on the collector branch rather than merged + directly (per CLAUDE.md's PR wave process). + +Contributed by @Hansen1018 (#870), @ding-modding (#909), @DukeDawg +(#864), @toilalesondev (#861 + #876), @yoelgal (#875). ## [0.34.0.0] - 2026-05-14 **Recursive code intelligence ships. Plan-mode subagents get one-call blast and flow.** diff --git a/CLAUDE.md b/CLAUDE.md index 5c6ec36..4269771 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -40,8 +40,8 @@ strict behavior when unset. ## Key files -- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (v0.23 glob matcher: `/*` matches recursive children; bare `` matches exact only). `OperationContext.remote` flags untrusted callers; `OperationContext.allowedSlugPrefixes` (v0.23) is the trusted-workspace allow-list set by the dream cycle. `put_page` enforces: when `viaSubagent` and `allowedSlugPrefixes` is set, slug must match the allow-list; else the legacy `wiki/agents//...` namespace check applies. Auto-link enabled for trusted-workspace writes (skipped only when `remote=true && !trustedWorkspace`). As of v0.26.0, every `Operation` also carries `scope?: 'read' | 'write' | 'admin'` + `localOnly?: boolean`. All ops are annotated; `sync_brain`, `file_upload`, `file_list`, and `file_url` are `admin + localOnly` (rejected over HTTP). `OperationContext.auth?: AuthInfo` is threaded through HTTP dispatch for scope enforcement in `serve-http.ts` before the op runs. **v0.26.9 (D12 + F7b):** `OperationContext.remote` is now a REQUIRED field in the TypeScript type — the compiler is the first defense against transports that forget to set it. Four trust-boundary call sites (`put_page` allowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) flipped from falsy-default (`!ctx.remote`) to fail-closed semantics (`ctx.remote === false` for "trusted-only" sites and `ctx.remote !== false` for "untrust unless explicit-false"). Anything that isn't strictly `false` is now treated as remote. Closed an HTTP MCP shell-job RCE: a `read+write`-scoped OAuth token could submit `shell` jobs because the HTTP request handler's literal context skipped `remote: true` and `submit_job`'s protected-name guard saw a falsy undefined. Stdio MCP set the field correctly via dispatch.ts; HTTP inlined a parallel context-builder for several releases and lost it. -- `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput` / `TimelineBatchInput` for the v0.12.1 bulk-insert API (`addLinksBatch` / `addTimelineEntriesBatch`). As of v0.13.1, `BrainEngine` has a `readonly kind: 'postgres' | 'pglite'` discriminator so migrations (`src/core/migrate.ts`) and other consumers can branch on engine without `instanceof` + dynamic imports. **v0.29:** four new methods — `batchLoadEmotionalInputs(slugs?)` (CTE-shaped read with per-table aggregates so a page × N tags × M takes never produces N×M rows), `setEmotionalWeightBatch(rows)` (`UPDATE FROM unnest($1::text[], $2::text[], $3::real[])` composite-keyed on `(slug, source_id)` for multi-source safety), `getRecentSalience(opts)`, `findAnomalies(opts)`. `PageFilters` extended with `sort?: 'updated_desc' | 'updated_asc' | 'created_desc' | 'slug'` + `PAGE_SORT_SQL` whitelist consumed by both engines (was hardcoded `ORDER BY updated_at DESC`). **v0.32.8 (PR #860):** new `listAllPageRefs(): Promise>` ordered by `(source_id, slug)`. Cheap cross-source enumeration for hot loops on large brains — replaces the `getAllSlugs()→getPage(slug)` N+1 pattern in extract-takes, extract, integrity, which silently defaulted to `source_id='default'` for non-default-source pages. Implementation parity across postgres-engine.ts + pglite-engine.ts. Pinned by `test/e2e/multi-source-bug-class.test.ts`. +- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (v0.23 glob matcher: `/*` matches recursive children; bare `` matches exact only). `OperationContext.remote` flags untrusted callers; `OperationContext.allowedSlugPrefixes` (v0.23) is the trusted-workspace allow-list set by the dream cycle. `put_page` enforces: when `viaSubagent` and `allowedSlugPrefixes` is set, slug must match the allow-list; else the legacy `wiki/agents//...` namespace check applies. Auto-link enabled for trusted-workspace writes (skipped only when `remote=true && !trustedWorkspace`). As of v0.26.0, every `Operation` also carries `scope?: 'read' | 'write' | 'admin'` + `localOnly?: boolean`. All ops are annotated; `sync_brain`, `file_upload`, `file_list`, and `file_url` are `admin + localOnly` (rejected over HTTP). `OperationContext.auth?: AuthInfo` is threaded through HTTP dispatch for scope enforcement in `serve-http.ts` before the op runs. **v0.26.9 (D12 + F7b):** `OperationContext.remote` is now a REQUIRED field in the TypeScript type — the compiler is the first defense against transports that forget to set it. Four trust-boundary call sites (`put_page` allowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) flipped from falsy-default (`!ctx.remote`) to fail-closed semantics (`ctx.remote === false` for "trusted-only" sites and `ctx.remote !== false` for "untrust unless explicit-false"). Anything that isn't strictly `false` is now treated as remote. Closed an HTTP MCP shell-job RCE: a `read+write`-scoped OAuth token could submit `shell` jobs because the HTTP request handler's literal context skipped `remote: true` and `submit_job`'s protected-name guard saw a falsy undefined. Stdio MCP set the field correctly via dispatch.ts; HTTP inlined a parallel context-builder for several releases and lost it. **v0.34.1.0 (#861 + #876):** new helper `sourceScopeOpts(ctx)` encodes the precedence ladder for source-scoped reads — federated array (`ctx.auth.allowedSources`) wins over scalar (`ctx.sourceId` / `ctx.auth.sourceId`) over nothing. Every read-side op handler routes through it so future ops can't silently drift from the canonical v0.31.8 thread. Closes the source-isolation leak on the read path: a `read+write`-scoped OAuth client bound to `--source dept-x` no longer sees rows from neighboring sources via `search` / `query` / `list_pages` / `get_page` / `find_experts` / `query`'s image path. +- `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput` / `TimelineBatchInput` for the v0.12.1 bulk-insert API (`addLinksBatch` / `addTimelineEntriesBatch`). As of v0.13.1, `BrainEngine` has a `readonly kind: 'postgres' | 'pglite'` discriminator so migrations (`src/core/migrate.ts`) and other consumers can branch on engine without `instanceof` + dynamic imports. **v0.29:** four new methods — `batchLoadEmotionalInputs(slugs?)` (CTE-shaped read with per-table aggregates so a page × N tags × M takes never produces N×M rows), `setEmotionalWeightBatch(rows)` (`UPDATE FROM unnest($1::text[], $2::text[], $3::real[])` composite-keyed on `(slug, source_id)` for multi-source safety), `getRecentSalience(opts)`, `findAnomalies(opts)`. `PageFilters` extended with `sort?: 'updated_desc' | 'updated_asc' | 'created_desc' | 'slug'` + `PAGE_SORT_SQL` whitelist consumed by both engines (was hardcoded `ORDER BY updated_at DESC`). **v0.32.8 (PR #860):** new `listAllPageRefs(): Promise>` ordered by `(source_id, slug)`. Cheap cross-source enumeration for hot loops on large brains — replaces the `getAllSlugs()→getPage(slug)` N+1 pattern in extract-takes, extract, integrity, which silently defaulted to `source_id='default'` for non-default-source pages. Implementation parity across postgres-engine.ts + pglite-engine.ts. Pinned by `test/e2e/multi-source-bug-class.test.ts`. **v0.34.1.0 (#861):** `SearchOpts` + `PageFilters` add `sourceIds?: string[]` for the federated read axis; both engines apply `WHERE source_id = ANY($N::text[])` when the array is set and preserve the scalar `sourceId` fast path when unset. `traverseGraph(slug, depth, opts?)` and `traversePaths(slug, opts?)` accept `opts.sourceId` / `opts.sourceIds` so graph walks respect the caller's scope. - `src/core/engine-factory.ts` — Engine factory with dynamic imports (`'pglite'` | `'postgres'`) - `src/core/pglite-engine.ts` — PGLite (embedded Postgres 17.5 via WASM) implementation, all 40 BrainEngine methods. `addLinksBatch` / `addTimelineEntriesBatch` use multi-row `unnest()` with manual `$N` placeholders. As of v0.13.1, `connect()` wraps `PGlite.create()` in a try/catch that emits an actionable error naming the macOS 26.3 WASM bug (#223) and pointing at `gbrain doctor`; the lock is released on failure so the next process can retry cleanly. v0.22.0: `searchKeyword` and `searchKeywordChunks` multiply `ts_rank` by the source-factor CASE expression at the chunk-grain level; `searchVector` becomes a two-stage CTE — inner CTE keeps `ORDER BY cc.embedding <=> vec` so HNSW stays usable, outer SELECT re-ranks by `raw_score * source_factor`. Inner LIMIT scales with offset to preserve pagination contract. As of v0.22.6.1, `initSchema()` calls `applyForwardReferenceBootstrap()` BEFORE replaying SCHEMA_SQL — probes for the specific forward-referenced state the embedded schema blob needs (`pages.source_id`, `links.link_source`, `links.origin_page_id`, `content_chunks.symbol_name`, `content_chunks.language`, `sources` FK target table) and adds only what's missing. Closes the upgrade-wedge bug class that bit users 10+ times across 6 schema versions over 2 years (#239/#243/#266/#357/#366/#374/#375/#378/#395/#396). No-op on fresh installs and modern brains. - `src/core/pglite-schema.ts` — PGLite-specific DDL (pgvector, pg_trgm, triggers) @@ -94,7 +94,7 @@ strict behavior when unset. - `src/core/embedding.ts` — OpenAI text-embedding-3-large, batch, retry, backoff. **v0.28.7:** `BATCH_SIZE` reverted 50→100 — the original Voyage safety guard halved OpenAI throughput on every page. Per-recipe pre-split + recursive halving + adaptive shrink-on-miss now live in the gateway, so the outer paginator goes back to its original purpose: progress-callback granularity, not batch protection. - `src/core/ai/dims.ts` (v0.33.1.1, PR #962 + #866) — per-provider `providerOptions` resolver for embed-time dimension passthrough. The single source of truth for "which provider needs which knob to produce `vector(N)`". Exports `dimsProviderOptions(implementation, modelId, dims)` (called by `embed()` in `gateway.ts`), `VOYAGE_OUTPUT_DIMENSION_MODELS` (private const — the 7 hosted Voyage models that accept `output_dimension`: `voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-3-large`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-code-3` — nano deliberately excluded), `VOYAGE_VALID_OUTPUT_DIMS = [256, 512, 1024, 2048] as const`, `supportsVoyageOutputDimension(modelId)`, and `isValidVoyageOutputDim(dims)`. **Voyage path uses the SDK-supported `dimensions` field** (`{ openaiCompatible: { dimensions: N } }`), NOT Voyage's `output_dimension` wire-key — the existing `voyageCompatFetch` shim in `gateway.ts:541` translates `dimensions → output_dimension` before the HTTP body is built. The reverse (sending `output_dimension` from here) was the v0.33.1.0 bug class: the AI SDK's openai-compatible adapter doesn't recognize the wire-key so it was silently dropped, Voyage returned its default 1024-dim, and the gateway dimension check threw on every embed call. Runtime guard: when a Voyage flexible-dim model is configured with `dims` outside `VOYAGE_VALID_OUTPUT_DIMS`, throws `AIConfigError` with a paste-ready `gbrain config set embedding_dimensions <256|512|1024|2048>` hint at the embed boundary — fail-loud instead of opaque Voyage HTTP 400. Most common trigger: `embedding_model: voyage:voyage-4-large` configured without `embedding_dimensions` (falls back to `DEFAULT_EMBEDDING_DIMENSIONS=1536`, an OpenAI default not a Voyage one). Eva (@100yenadmin) shipped the wire-key fix in #866; Codex P3 follow-up landed the validator + valid-dims allowlist in #962. - `src/core/ai/types.ts` — provider/recipe types. **v0.28.7 (#680):** `EmbeddingTouchpoint` extended with optional `chars_per_token` (default 4 chars/token, matching OpenAI tiktoken on English) and `safety_factor` (default 0.8, budget-utilization ceiling). Both consulted only when `max_batch_tokens` is also set. Voyage declares `chars_per_token=1` + `safety_factor=0.5` to handle dense payloads (CJK/JSON/base64) that overshoot tiktoken. The pre-split budget is `max_batch_tokens × safety_factor / chars_per_token`. **v0.28.11 (#719):** `EmbeddingTouchpoint.multimodal_models?: string[]` model-level allow-list for recipes that mix text-only + multimodal models under one touchpoint (Voyage's 12 models share `supports_multimodal: true` but only `voyage-multimodal-3` accepts `/multimodalembeddings`). When omitted, recipe-level `supports_multimodal` is sufficient. `AIGatewayConfig.embedding_multimodal_model?: string` lets `embedMultimodal()` route to a different model than `embedding_model` — brains using OpenAI for text can use Voyage for images without flipping the primary embedding pipeline. -- `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state. **v0.33.1.1 (#962, Codex P3 follow-up):** new exported `VoyageResponseTooLargeError` tagged class at the top of the file. `voyageCompatFetch`'s two OOM-defense caps (Layer 1 Content-Length check at `:595`, Layer 2 per-embedding base64 cap at `:619`) now throw `VoyageResponseTooLargeError` instead of a generic `Error`. The inbound response-rewriter's surrounding try/catch (which intentionally swallows parse failures so misshaped Voyage responses fall through to the SDK's JSON parser) checks `instanceof VoyageResponseTooLargeError` and rethrows. Pre-fix, the Layer 2 throw was silently swallowed and the oversized response returned to the AI SDK anyway — Layer 2 was theatrical. Source-shape regression assertion in `test/voyage-response-cap.test.ts` pins the `instanceof ⇒ throw err` line. +- `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state. **v0.33.1.1 (#962, Codex P3 follow-up):** new exported `VoyageResponseTooLargeError` tagged class at the top of the file. `voyageCompatFetch`'s two OOM-defense caps (Layer 1 Content-Length check at `:595`, Layer 2 per-embedding base64 cap at `:619`) now throw `VoyageResponseTooLargeError` instead of a generic `Error`. The inbound response-rewriter's surrounding try/catch (which intentionally swallows parse failures so misshaped Voyage responses fall through to the SDK's JSON parser) checks `instanceof VoyageResponseTooLargeError` and rethrows. Pre-fix, the Layer 2 throw was silently swallowed and the oversized response returned to the AI SDK anyway — Layer 2 was theatrical. Source-shape regression assertion in `test/voyage-response-cap.test.ts` pins the `instanceof ⇒ throw err` line. **v0.34.1.0 (#875):** new `embedMultimodalOpenAICompat()` routes recipes with `implementation: 'openai-compatible'` (LiteLLM, Anyscale, vLLM, Gemini multimodal via proxy) through the standard `/embeddings` endpoint with content arrays carrying `image_url` entries. The pre-existing Voyage `/multimodalembeddings` path is unchanged; the gateway selects by recipe `implementation` tag. Runtime dimension validation throws `AIConfigError` (with model id + observed + expected) before the vector reaches storage when the provider returns a width that doesn't match the recipe's `default_dims` or the brain's `embedding_dimensions` config — no more cryptic `vector dimension mismatch` at INSERT time. Pinned by 11 cases in `test/openai-compat-multimodal.test.ts`. - `src/core/ai/recipes/voyage.ts` — Voyage AI openai-compatible recipe. **v0.28.7 (#680):** declares `chars_per_token=1` + `safety_factor=0.5` so the gateway pre-splits Voyage batches at a 60K-character budget (50% of 120K-token cap with the dense-tokenizer ratio). Closes the v0.27 backfill loop where ~26% of the corpus stayed un-embedded because tiktoken-grounded budgeting silently undercounted Voyage's actual token usage. **v0.28.11 (#719):** declares `multimodal_models: ['voyage-multimodal-3']` so the gateway rejects text-only Voyage models pointed at the multimodal endpoint with a clear `AIConfigError` instead of waiting for Voyage's HTTP 400. **v0.33.1.1 (#962, fixup):** recipe docstring at `:7-16` tightened to name the seven hosted flexible-dim models that accept `output_dimension` explicitly (`voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-3-large`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-code-3`) and call out that `voyage-4-nano` is the open-weight variant listed separately by Voyage as fixed 1024-dim — does NOT accept the parameter. The "all v4 variants are flexible" misread is what caused the original PR to include nano in `VOYAGE_OUTPUT_DIMENSION_MODELS`; the negative regression assertion in `test/ai/gateway.test.ts` (`dimsProviderOptions` returns `undefined` for `voyage-4-nano`) pins the contract. - `src/core/ai/recipes/anthropic.ts` — Anthropic recipe (chat + expansion touchpoints). **v0.31.12:** chat and expansion `models:` lists drop the v0.31.6 phantom `claude-sonnet-4-6-20250929` date suffix — canonical id is `claude-sonnet-4-6`. The wrong-direction alias `claude-sonnet-4-6 → claude-sonnet-4-6-20250929` is removed; a reverse alias `claude-sonnet-4-6-20250929 → claude-sonnet-4-6` keeps stale user configs working (rescues `facts.extraction_model` and `models.dream.synthesize` set by v0.31.6 installs). Recipe-shape regression pinned by `test/anthropic-model-ids.test.ts` (6 cases, verbatim cherry-pick of PR #830 plus the reverse-alias rescue case). - `src/core/anthropic-pricing.ts` — Single source of truth for Anthropic model pricing (per-MTok input/output). **v0.31.12:** Opus 4.7 corrected from `$15/$75` to `$5/$25` (the old number was from Opus 4 generation, never refreshed when 4.7 shipped); Opus 4.6 also corrected. Consumed by `src/core/budget-meter.ts` and `src/core/cross-modal-eval/runner.ts` — the cross-modal estimator now reads `ANTHROPIC_PRICING` for Anthropic models instead of duplicating the table, killing the v0.31.6 drift bug class. @@ -153,15 +153,15 @@ strict behavior when unset. - `src/commands/jobs.ts` — `gbrain jobs` CLI subcommands + `gbrain jobs work` daemon. **v0.28.1:** `case 'work'` now wraps `worker.start()` in try/finally and owns engine lifecycle — calls `engine.disconnect()` on shutdown with loud error logging on failure. Replaces the prior call inside `MinionWorker.start()` (which violated engine ownership: the worker disconnected an engine it didn't own, and clobbered the module-level singleton on PostgresEngine via the now-fixed idempotency bug). Pool slots now free immediately on shutdown instead of waiting for TCP keepalive (~minutes). v0.13.1 surfaces the full `MinionJobInput` retry/backoff/timeout/idempotency surface as first-class CLI flags on `jobs submit`: `--max-stalled`, `--backoff-type fixed|exponential`, `--backoff-delay`, `--backoff-jitter`, `--timeout-ms`, `--idempotency-key`. `jobs smoke --sigkill-rescue` is the opt-in regression guard for #219. v0.16 wires `registerBuiltinHandlers` to always register `subagent` + `subagent_aggregator` (no env flag — `ANTHROPIC_API_KEY` is the natural cost gate, trust is via `PROTECTED_JOB_NAMES`) and loads `GBRAIN_PLUGIN_PATH` plugins at worker startup with a loud startup-line per plugin. `shell` handler still gated by `GBRAIN_ALLOW_SHELL_JOBS=1` (RCE surface, separate concern). v0.22.10 (#521): the `autopilot-cycle` handler now forwards `job.data.phases` to `runCycle` (was previously discarded — caller-supplied phase selection silently became a full cycle). Phases are validated against `ALL_PHASES` from `src/core/cycle.ts`; invalid names are filtered out and an empty/missing array falls back to the default 6-phase cycle. v0.22.13 (PR #490 CODEX-1+CODEX-4): `sync` handler now resolves `sourceId` at entry by looking up `sources.local_path` (mirrors `cycle.ts:480`'s autopilot fix from PR #475) so multi-source brains read the per-source `last_commit` anchor instead of the global config key. Concurrency routed through the shared `autoConcurrency()` policy in `src/core/sync-concurrency.ts` instead of the prior hardcoded `4`; PGLite stays serial. `noEmbed` default is `true` (embed is a separate job — submit `gbrain embed --stale` after sync, or rely on the autopilot cycle's embed phase). - `src/commands/features.ts` — `gbrain features --json --auto-fix`: usage scan + feature adoption salesman - `src/commands/autopilot.ts` — `gbrain autopilot --install`: self-maintaining brain daemon (sync+extract+embed). **v0.28.1:** consumes `detectTini()` from `src/core/minions/spawn-helpers.ts` and resolves it once at startup instead of per worker respawn (was paying an `execFileSync` cost on every restart). -- `src/mcp/server.ts` — MCP stdio server (generated from operations). v0.22.7: tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path. +- `src/mcp/server.ts` — MCP stdio server (generated from operations). v0.22.7: tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path. **v0.34.1.0 (#870):** stdin `'end'` / `'close'` shutdown hooks are skipped when `process.env.MCP_STDIO === '1'`. Gateway-piped stdio MCP wrappers (OpenClaw's `bundle-mcp`, similar) pipe the JSON-RPC handshake then close their stdin half; pre-fix this killed the server before the first tool call landed. Signal handlers (SIGTERM / SIGINT / SIGHUP) and the parent-process watchdog still cover legitimate disconnects. `src/commands/serve.ts` exposes `ServeOptions.mcpStdio?: boolean` as a test seam so the runtime guard is exercisable without process.env mutation. Pinned by `test/serve-stdio-lifecycle.test.ts`. - `src/mcp/dispatch.ts` (v0.22.7) — Shared tool-call dispatch consumed by both stdio (`server.ts`) and HTTP transports. Exports `dispatchToolCall(engine, name, params, opts)`, `buildOperationContext(engine, params, opts)`, and `validateParams(op, params)`. Single source of truth for `(ctx, params)` handler arg order and the 5-field `OperationContext` shape (engine + config + logger + dryRun + remote). Defaults to `remote: true` (untrusted); local CLI callers pass `remote: false`. Closed F1/F2/F3 drift bugs in the original v0.22.5 HTTP transport. **v0.26.9 (F8):** adds `summarizeMcpParams(opName, params)` — privacy-preserving redactor for `mcp_request_log` and the admin SSE feed. Returns `{redacted, kind, declared_keys, unknown_key_count, approx_bytes}`. Intersects submitted top-level keys against the operation's declared `params` allow-list (declared keys preserved as a sorted array for debug visibility; unknown keys counted but never named, closing the attacker-controlled-key-name leak). Byte counts bucketed up to nearest 1KB so an attacker can't binary-search secret-content sizes via repeated probes. Operators on a personal laptop who want raw payload visibility opt back in with `gbrain serve --http --log-full-params` (loud stderr warning at startup). Canonical helper — new logging code paths route through it rather than `JSON.stringify(params)`. - `src/mcp/rate-limit.ts` (v0.22.7) — Bounded-LRU token-bucket limiter. `buildDefaultLimiters()` returns the two-bucket pipeline: pre-auth IP (30/60s, fires BEFORE the DB lookup so brute-force load against `access_tokens` is actually capped) + post-auth token-id (60/60s). Tracks `lastTouchedMs` separately from `lastRefillMs` so an exhausted key can't be reset by hammering past the TTL. LRU cap bounds memory under attacker-controlled key growth. -- `src/commands/serve-http.ts` (v0.26.0) — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. Started via `gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--log-full-params]`. Supersedes the v0.22.7 `src/mcp/http-transport.ts` simple bearer-auth path. Combines MCP SDK's `mcpAuthRouter` (authorize / token / register / revoke endpoints), a custom `client_credentials` handler (SDK's token endpoint throws `UnsupportedGrantTypeError` for CC; the custom handler runs BEFORE the router and falls through for `auth_code` / `refresh_token`), `requireBearerAuth` middleware for `/mcp` with scope enforcement before op dispatch, `localOnly` rejection, and `express-rate-limit` at 50 req / 15 min on `/token`. Serves the built admin SPA from `admin/dist/` with SPA fallback. `/admin/events` SSE endpoint broadcasts every MCP request to connected admin browsers. `cookie-parser` middleware wired (Express 5 has no built-in). Startup logging prints port, engine, configured issuer URL (honors `--public-url`), registered-client count, DCR status, and admin bootstrap token. **v0.26.9 hardening pass:** F7 sets `remote: true` explicitly on the `/mcp` request handler's OperationContext literal (closes the HTTP shell-job RCE — without this, `submit_job`'s protected-name guard at `operations.ts:1391` saw a falsy undefined and skipped, letting a `read+write`-scoped OAuth token submit `shell` jobs). F8 wires `summarizeMcpParams` from `src/mcp/dispatch.ts` into both `mcp_request_log` writes and the admin SSE feed by default (raw payloads opt-in via `--log-full-params` with stderr warning). F9 sets cookie `Secure` flag when behind HTTPS or a public-URL proxy. F10 caps the magic-link nonce store with an LRU bound. F12 routes DCR disable through the `GBrainOAuthProvider` constructor's `dcrDisabled` option instead of the prior monkey-patch on the express router. F14 wraps `transport.handleRequest` in try/catch so SDK throws return a JSON-RPC 500 envelope instead of express's default HTML error page. F15 unifies OperationError + unexpected exceptions through `buildError` / `serializeError` so `/mcp` always returns the same envelope shape. **v0.28.1:** `/health` endpoint extracted into pure `probeHealth(engine)` async function with `HEALTH_TIMEOUT_MS = 3000` exported constant — drops the timeout from 5s to 3s so Fly.io's 5s health-check deadline gets 2s of headroom for TCP, response framing, and clock skew. Races `engine.getStats()` against the timeout via `Promise.race`; saturated pool returns 503 with `Health check timed out (database pool may be saturated)` instead of hanging. `clearTimeout` in finally block prevents pending-timer pile-up under high probe rates (race-leak fix from adversarial review). **v0.28.10:** `/health` is now liveness-only via the new `probeLiveness(sql, engineName, version, timeoutMs)` helper that races `sql\`SELECT 1\`` against `HEALTH_TIMEOUT_MS` and returns the same `ProbeHealthResult` tagged-union as `probeHealth` (single timer-cleanup site, single 503 envelope). Body shape: `{status, version, engine}` only — engine stats are no longer spread on the public route. Full stats moved to a new admin endpoint `/admin/api/full-stats` (sibling to `/admin/api/stats` and `/admin/api/health-indicators`) gated by the existing `requireAdmin` middleware; that route calls `probeHealth(engine, ...)` and returns the original spread-stats body. `?full=true` query param removed entirely. Closes the original DoS surface where `getStats()`'s 6× count(*) on 96K-page brains through PgBouncer exceeded `HEALTH_TIMEOUT_MS` and triggered orchestrator restart cascades (Fly.io / k8s seeing 503 → restart loop → advisory-lock pile-up on the migration lock). Outside-voice review (Codex) caught that `/admin/api/health-indicators` is NOT a full-stats endpoint (returns only `{expiring_soon, error_rate}`), and that an alternative loopback-IP gate would have depended on `app.set('trust proxy', 'loopback')` semantics holding under proxy/XFF misconfiguration; the shipped admin-cookie design avoids both. **v0.31.3 (#681):** every OAuth/admin/audit SQL call routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` so `gbrain serve --http` works against PGLite brains. The four `mcp_request_log.params` INSERT sites (success path, auth_failed path, scope_denied path, server-error path) all go through `executeRawJsonb(engine, ...)` so the JSONB column stores real objects, not JSON-encoded strings — closes the bug where `params->>'op'` returned the encoded string `"search"` (with quotes) instead of `search`. Migration v46 normalizes any pre-v0.31.3 string-shaped backlog rows on first start. +- `src/commands/serve-http.ts` (v0.26.0) — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. Started via `gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--log-full-params]`. Supersedes the v0.22.7 `src/mcp/http-transport.ts` simple bearer-auth path. Combines MCP SDK's `mcpAuthRouter` (authorize / token / register / revoke endpoints), a custom `client_credentials` handler (SDK's token endpoint throws `UnsupportedGrantTypeError` for CC; the custom handler runs BEFORE the router and falls through for `auth_code` / `refresh_token`), `requireBearerAuth` middleware for `/mcp` with scope enforcement before op dispatch, `localOnly` rejection, and `express-rate-limit` at 50 req / 15 min on `/token`. Serves the built admin SPA from `admin/dist/` with SPA fallback. `/admin/events` SSE endpoint broadcasts every MCP request to connected admin browsers. `cookie-parser` middleware wired (Express 5 has no built-in). Startup logging prints port, engine, configured issuer URL (honors `--public-url`), registered-client count, DCR status, and admin bootstrap token. **v0.26.9 hardening pass:** F7 sets `remote: true` explicitly on the `/mcp` request handler's OperationContext literal (closes the HTTP shell-job RCE — without this, `submit_job`'s protected-name guard at `operations.ts:1391` saw a falsy undefined and skipped, letting a `read+write`-scoped OAuth token submit `shell` jobs). F8 wires `summarizeMcpParams` from `src/mcp/dispatch.ts` into both `mcp_request_log` writes and the admin SSE feed by default (raw payloads opt-in via `--log-full-params` with stderr warning). F9 sets cookie `Secure` flag when behind HTTPS or a public-URL proxy. F10 caps the magic-link nonce store with an LRU bound. F12 routes DCR disable through the `GBrainOAuthProvider` constructor's `dcrDisabled` option instead of the prior monkey-patch on the express router. F14 wraps `transport.handleRequest` in try/catch so SDK throws return a JSON-RPC 500 envelope instead of express's default HTML error page. F15 unifies OperationError + unexpected exceptions through `buildError` / `serializeError` so `/mcp` always returns the same envelope shape. **v0.28.1:** `/health` endpoint extracted into pure `probeHealth(engine)` async function with `HEALTH_TIMEOUT_MS = 3000` exported constant — drops the timeout from 5s to 3s so Fly.io's 5s health-check deadline gets 2s of headroom for TCP, response framing, and clock skew. Races `engine.getStats()` against the timeout via `Promise.race`; saturated pool returns 503 with `Health check timed out (database pool may be saturated)` instead of hanging. `clearTimeout` in finally block prevents pending-timer pile-up under high probe rates (race-leak fix from adversarial review). **v0.28.10:** `/health` is now liveness-only via the new `probeLiveness(sql, engineName, version, timeoutMs)` helper that races `sql\`SELECT 1\`` against `HEALTH_TIMEOUT_MS` and returns the same `ProbeHealthResult` tagged-union as `probeHealth` (single timer-cleanup site, single 503 envelope). Body shape: `{status, version, engine}` only — engine stats are no longer spread on the public route. Full stats moved to a new admin endpoint `/admin/api/full-stats` (sibling to `/admin/api/stats` and `/admin/api/health-indicators`) gated by the existing `requireAdmin` middleware; that route calls `probeHealth(engine, ...)` and returns the original spread-stats body. `?full=true` query param removed entirely. Closes the original DoS surface where `getStats()`'s 6× count(*) on 96K-page brains through PgBouncer exceeded `HEALTH_TIMEOUT_MS` and triggered orchestrator restart cascades (Fly.io / k8s seeing 503 → restart loop → advisory-lock pile-up on the migration lock). Outside-voice review (Codex) caught that `/admin/api/health-indicators` is NOT a full-stats endpoint (returns only `{expiring_soon, error_rate}`), and that an alternative loopback-IP gate would have depended on `app.set('trust proxy', 'loopback')` semantics holding under proxy/XFF misconfiguration; the shipped admin-cookie design avoids both. **v0.31.3 (#681):** every OAuth/admin/audit SQL call routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` so `gbrain serve --http` works against PGLite brains. The four `mcp_request_log.params` INSERT sites (success path, auth_failed path, scope_denied path, server-error path) all go through `executeRawJsonb(engine, ...)` so the JSONB column stores real objects, not JSON-encoded strings — closes the bug where `params->>'op'` returned the encoded string `"search"` (with quotes) instead of `search`. Migration v46 normalizes any pre-v0.31.3 string-shaped backlog rows on first start. **v0.34.1.0 (#864):** new `--bind HOST` CLI flag with default `127.0.0.1`. Personal-laptop installs no longer publish the brain to the LAN by accident. Self-hosted operators pass `--bind 0.0.0.0` (or a specific interface IP) once to accept remote connections. A stderr WARN fires when `--public-url` is set without `--bind` so the operator sees the binding before the first request (common cause of "ngrok forwards to me but the agent can't reach the upstream" misconfigurations). The startup banner prints a `Bind:` line. **v0.34.1.0 (#861):** drops the `(authInfo as AuthInfo & {sourceId?: string}).sourceId ?? env ?? 'default'` cast chain — `AuthInfo.sourceId` and `AuthInfo.allowedSources` are now the typed source of truth, populated by `oauth-provider.ts:verifyAccessToken` from the `oauth_clients` row. - `src/core/sql-query.ts` (v0.31.3) — Engine-aware tagged-template SQL adapter for OAuth/admin/auth infrastructure. `sqlQueryForEngine(engine)` returns a `SqlQuery` (`(strings, ...values) => Promise`) that walks the template, builds `$N` positional SQL, asserts every value is a `SqlValue` (string | number | bigint | boolean | Date | null), and routes through `engine.executeRaw(sql, params)` so Postgres goes via postgres.js's `unsafe(sql, params)` path and PGLite via its embedded `db.query(sql, params)`. Deliberately narrower than postgres.js's `sql` tag: no nested fragments, no `sql.json()`, no `sql.unsafe()`, no `sql.begin()`, no array binding. The narrow surface is the feature — codex finding #7 from the v0.31 plan review argued the adapter should stay scalar-only or it drifts into a partial postgres.js clone. JSONB writes go through the separate `executeRawJsonb(engine, sql, scalarParams, jsonbParams)` helper that composes positional `$N::jsonb` casts and passes JS objects through; the v0.12.0 double-encode bug class doesn't apply because positional binding through `unsafe()` reaches the wire protocol with the correct type oid (verified by `test/sql-query.test.ts` on PGLite and `test/e2e/auth-permissions.test.ts:67` on Postgres). `scripts/check-jsonb-pattern.sh` doesn't fire because `executeRawJsonb(...)` is a method call, not the banned literal-template-tag interpolation pattern. Consumed by `src/commands/auth.ts`, `src/commands/serve-http.ts`, `src/core/oauth-provider.ts`, `src/commands/files.ts`, and `src/mcp/http-transport.ts` so all five sites work against PGLite and Postgres uniformly. Closes the bug where `gbrain auth` + `gbrain serve --http` were silently Postgres-only because they routed every SQL through the postgres.js singleton (community PR #681). - `src/commands/serve.ts` (v0.31.3) — `gbrain serve` stdio MCP entrypoint with idempotent shutdown across every parent-disconnect signal. Stdio EOF, SIGTERM, SIGINT, SIGHUP, and parent-process death (every reparent case — PID 1, launchd subreaper, systemd, tmux, or a parent shell with `PR_SET_CHILD_SUBREAPER`) all funnel into one `cleanup(reason)` path that releases the engine and the PGLite write-lock dir within 5 seconds. Pre-v0.31.3 the stdio MCP server held the lock indefinitely after Claude Desktop / Cursor / launchd-managed gateways disconnected, forcing a 5-minute stale-lock wait on the next start. Watchdog reparent check is `getParentPid() !== initialParentPid` (capturing the initial ppid once at install time and firing on any change); the previous `=== 1` check missed the subreaper case under launchd / systemd. Bun's `process.ppid` cache is stale across reparenting (see [oven-sh/bun#30305](https://github.com/oven-sh/bun/issues/30305)) so `getParentPid()` runs `spawnSync('ps', ['-o', 'ppid=', '-p', PID])` per tick to read the live kernel PPID. Startup probe verifies `ps` is on PATH; if not (stripped containers, busybox without procps), the watchdog skips installing AND emits a loud `[gbrain serve] watchdog disabled: ps unavailable, parent-death detection unavailable — child will rely on stdin EOF / signals only` stderr line so operators see the degraded mode at boot. Pinned by `test/serve-stdio-lifecycle.test.ts` (22 cases). Closes #413, #446. Credit @Aragorn2046 (origin features in #591) and @seungsu-kr (rebased submitter, Bun ppid workaround). -- `src/core/oauth-provider.ts` (v0.26.0) — `GBrainOAuthProvider` implementing the MCP SDK's `OAuthServerProvider` + `OAuthRegisteredClientsStore` interfaces. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1 spec: `authorize` + `exchangeAuthorizationCode` with PKCE (for ChatGPT), `client_credentials` (for Perplexity / Claude), `refresh_token` with rotation, `revokeToken`, `registerClient` (DCR path validates redirect_uri must be `https://` or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Auth codes single-use with 10-minute TTL via atomic `DELETE...RETURNING` (closes RFC 6749 §10.5 TOCTOU race). Refresh rotation also `DELETE...RETURNING` (closes §10.4 stolen-token detection bypass). `pgArray()` escapes commas/quotes/braces in elements so a comma-bearing redirect_uri can't smuggle a second array element. Legacy `access_tokens` fallback in `verifyAccessToken` grandfathers pre-v0.26 bearer tokens as `read+write+admin`. `sweepExpiredTokens()` runs on startup wrapped in try/catch. **v0.26.9 RFC 6749/7009 hardening pass:** F1+F2 fold `client_id` atomically into the `DELETE WHERE` clauses for both auth-code exchange and refresh rotation — pre-fix the post-hoc client compare burned the row on wrong-client paths so the legitimate client couldn't retry. F3 enforces refresh-scope-subset against the original grant on the row (RFC 6749 §6), not the client's currently-allowed scopes — fixes the case where revoking a scope from a client wouldn't shrink the agent's existing refresh tokens. F4 binds `client_id` on `revokeToken` so a client can only revoke its own tokens (RFC 7009 §2.1). F7c validates the `/token` request's `redirect_uri` against the value stored at `/authorize` (RFC 6749 §4.1.3) — empty-string treated as missing rather than wildcard match (adversarial-review fix). F5 swaps bare `catch {}` blocks in `verifyAccessToken` and `getClient` for `isUndefinedColumnError` from `src/core/utils.ts` — only SQLSTATE 42703 falls through to legacy fallback; lock timeouts and network blips throw and surface. F6 makes `sweepExpiredTokens()` actually return the count via `RETURNING 1` + array length, not a fire-and-forget zero. F12 adds `dcrDisabled` constructor option so `serve-http.ts` can disable the `/register` endpoint without monkey-patching the router. **v0.26.2:** module-private `coerceTimestamp()` boundary helper at the top of the file normalizes postgres-driver-as-string BIGINT columns to JS numbers at every read site (5 call sites: `getClient` L112+L113 for DCR `/register` RFC 7591 §3.2.1 numeric timestamps, `exchangeRefreshToken` L274 + `verifyAccessToken` L296+L303 for the SDK's `typeof === 'number'` bearerAuth check). Throws on non-finite input (NaN/Infinity) so corrupt rows fail loud at the boundary instead of riding through as `expiresAt: NaN`; returns undefined for SQL NULL so callers decide NULL semantics explicitly (refresh + access token paths treat NULL as expired). Helper intentionally NOT promoted to `src/core/utils.ts` — codex review flagged repo-wide BIGINT precision-loss risk for a generic helper. +- `src/core/oauth-provider.ts` (v0.26.0) — `GBrainOAuthProvider` implementing the MCP SDK's `OAuthServerProvider` + `OAuthRegisteredClientsStore` interfaces. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1 spec: `authorize` + `exchangeAuthorizationCode` with PKCE (for ChatGPT), `client_credentials` (for Perplexity / Claude), `refresh_token` with rotation, `revokeToken`, `registerClient` (DCR path validates redirect_uri must be `https://` or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Auth codes single-use with 10-minute TTL via atomic `DELETE...RETURNING` (closes RFC 6749 §10.5 TOCTOU race). Refresh rotation also `DELETE...RETURNING` (closes §10.4 stolen-token detection bypass). `pgArray()` escapes commas/quotes/braces in elements so a comma-bearing redirect_uri can't smuggle a second array element. Legacy `access_tokens` fallback in `verifyAccessToken` grandfathers pre-v0.26 bearer tokens as `read+write+admin`. `sweepExpiredTokens()` runs on startup wrapped in try/catch. **v0.26.9 RFC 6749/7009 hardening pass:** F1+F2 fold `client_id` atomically into the `DELETE WHERE` clauses for both auth-code exchange and refresh rotation — pre-fix the post-hoc client compare burned the row on wrong-client paths so the legitimate client couldn't retry. F3 enforces refresh-scope-subset against the original grant on the row (RFC 6749 §6), not the client's currently-allowed scopes — fixes the case where revoking a scope from a client wouldn't shrink the agent's existing refresh tokens. F4 binds `client_id` on `revokeToken` so a client can only revoke its own tokens (RFC 7009 §2.1). F7c validates the `/token` request's `redirect_uri` against the value stored at `/authorize` (RFC 6749 §4.1.3) — empty-string treated as missing rather than wildcard match (adversarial-review fix). F5 swaps bare `catch {}` blocks in `verifyAccessToken` and `getClient` for `isUndefinedColumnError` from `src/core/utils.ts` — only SQLSTATE 42703 falls through to legacy fallback; lock timeouts and network blips throw and surface. F6 makes `sweepExpiredTokens()` actually return the count via `RETURNING 1` + array length, not a fire-and-forget zero. F12 adds `dcrDisabled` constructor option so `serve-http.ts` can disable the `/register` endpoint without monkey-patching the router. **v0.26.2:** module-private `coerceTimestamp()` boundary helper at the top of the file normalizes postgres-driver-as-string BIGINT columns to JS numbers at every read site (5 call sites: `getClient` L112+L113 for DCR `/register` RFC 7591 §3.2.1 numeric timestamps, `exchangeRefreshToken` L274 + `verifyAccessToken` L296+L303 for the SDK's `typeof === 'number'` bearerAuth check). Throws on non-finite input (NaN/Infinity) so corrupt rows fail loud at the boundary instead of riding through as `expiresAt: NaN`; returns undefined for SQL NULL so callers decide NULL semantics explicitly (refresh + access token paths treat NULL as expired). Helper intentionally NOT promoted to `src/core/utils.ts` — codex review flagged repo-wide BIGINT precision-loss risk for a generic helper. **v0.34.1.0 (#909):** `registerClient` honors `token_endpoint_auth_method: "none"` (RFC 7591 §3.2.1) — public PKCE clients (Claude Code, Cursor, every other PKCE-first MCP client) store `client_secret_hash = NULL` and the response payload omits `client_secret` entirely. Confidential clients (default `client_secret_post` and explicit `client_secret_basic`) keep their one-time-reveal shape. `getClient` correctly normalizes a NULL `client_secret_hash` to JS `undefined` so the SDK's clientAuth path accepts the public client at `/token`. **v0.34.1.0 (#861 + #876):** `verifyAccessToken` JOINs `oauth_clients.source_id` (write scope, scalar) + `oauth_clients.federated_read` (read scope, TEXT[]) and surfaces both on the returned `AuthInfo`. Pre-v60 / pre-v61 brains degrade gracefully via `isUndefinedColumnError` fallback so the upgrade chain is non-blocking on legacy DBs. - `admin/` (v0.26.0) — React 19 + Vite + TypeScript admin SPA embedded in the binary via `admin/dist/` served by `serve-http.ts`. 7 screens: Login (bootstrap token → session cookie), Dashboard (metrics + SSE feed + token health), Agents (sortable table + sparklines + Register button), Register (modal with scope checkboxes + grant type selector), Credentials reveal (full-screen modal with Copy + Download JSON + yellow one-time-only warning), Request Log (filterable paginated), Agent Detail drawer (Details / Activity / Config Export tabs + Revoke). Design tokens: `#0a0a0f` bg, Inter for UI, JetBrains Mono for data, 4-32px spacing scale, rounded pill badges. HTTP-only SameSite=Strict cookie auth. 65KB gzip. Build: `cd admin && bun install && bun run build`; output at `admin/dist/` is committed for self-contained binaries. -- `src/commands/auth.ts` — Token management. `gbrain auth create/list/revoke/test` for legacy bearer tokens (v0.22.7 wired as a first-class CLI subcommand) plus `gbrain auth register-client` (v0.26.0) and `gbrain auth revoke-client ` (v0.26.2) for OAuth 2.1 client lifecycle. `revoke-client` runs an atomic `DELETE...RETURNING` on `oauth_clients`; FK `ON DELETE CASCADE` on `oauth_tokens.client_id` and `oauth_codes.client_id` purges every active token + authorization code in a single transaction. `process.exit(1)` on no-such-client (idempotent — re-running on the same id produces the same exit-1 message). Legacy tokens stored as SHA-256 hashes in `access_tokens`; OAuth clients in `oauth_clients`. As of v0.26.0, legacy tokens grandfather to `read+write+admin` scopes on the OAuth HTTP server, so pre-v0.26 deployments keep working with no migration. **v0.31.3 (#681):** every SQL site routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` (and `executeRawJsonb` for the takes-holders `permissions` JSONB column) so `gbrain auth` works against PGLite brains. Pre-fix, every call hit the postgres.js singleton via `getConn()` and silently failed (or wrote to the wrong DB) when the active engine was PGLite. The takes-holders write goes through `executeRawJsonb(engine, sql, [name, hash], [{takes_holders:[...]}])` which round-trips with `jsonb_typeof = 'object'` instead of the pre-v0.31.3 quoted-string shape. +- `src/commands/auth.ts` — Token management. `gbrain auth create/list/revoke/test` for legacy bearer tokens (v0.22.7 wired as a first-class CLI subcommand) plus `gbrain auth register-client` (v0.26.0) and `gbrain auth revoke-client ` (v0.26.2) for OAuth 2.1 client lifecycle. `revoke-client` runs an atomic `DELETE...RETURNING` on `oauth_clients`; FK `ON DELETE CASCADE` on `oauth_tokens.client_id` and `oauth_codes.client_id` purges every active token + authorization code in a single transaction. `process.exit(1)` on no-such-client (idempotent — re-running on the same id produces the same exit-1 message). Legacy tokens stored as SHA-256 hashes in `access_tokens`; OAuth clients in `oauth_clients`. As of v0.26.0, legacy tokens grandfather to `read+write+admin` scopes on the OAuth HTTP server, so pre-v0.26 deployments keep working with no migration. **v0.31.3 (#681):** every SQL site routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` (and `executeRawJsonb` for the takes-holders `permissions` JSONB column) so `gbrain auth` works against PGLite brains. Pre-fix, every call hit the postgres.js singleton via `getConn()` and silently failed (or wrote to the wrong DB) when the active engine was PGLite. The takes-holders write goes through `executeRawJsonb(engine, sql, [name, hash], [{takes_holders:[...]}])` which round-trips with `jsonb_typeof = 'object'` instead of the pre-v0.31.3 quoted-string shape. **v0.34.1.0 (#876):** `register-client` accepts `--source ` (write authority, scalar) and `--federated-read ` (read scope, array). The output prints the resolved `Write source` and `Federated reads` for the registered client. Pre-v0.34 clients backfill to `source_id='default'` via migration v60 so existing deployments keep their v0.33 effective behavior verbatim. - `src/commands/upgrade.ts` — Self-update CLI. `runPostUpgrade()` enumerates migrations from the TS registry (src/commands/migrations/index.ts) and tail-calls `runApplyMigrations(['--yes', '--non-interactive'])` so the mechanical side of every outstanding migration runs unconditionally. - `src/commands/migrations/` — TS migration registry (compiled into the binary; no filesystem walk of `skills/migrations/*.md` needed at runtime). `index.ts` lists migrations in semver order. `v0_11_0.ts` = Minions adoption orchestrator (8 phases). `v0_12_0.ts` = Knowledge Graph auto-wire orchestrator (5 phases: schema → config check → backfill links → backfill timeline → verify). `phaseASchema` has a 600s timeout (bumped from 60s in v0.12.1 for duplicate-heavy brains). `v0_12_2.ts` = JSONB double-encode repair orchestrator (4 phases: schema → repair-jsonb → verify → record). `v0_14_0.ts` = shell-jobs + autopilot cooperative (2 phases: schema ALTER minion_jobs.max_stalled SET DEFAULT 3 — superseded by v0.14.3's schema-level DEFAULT 5 + UPDATE backfill; pending-host-work ping for skills/migrations/v0.14.0.md). All orchestrators are idempotent and resumable from `partial` status. As of v0.14.2 (Bug 3), the RUNNER owns all ledger writes — orchestrators return `OrchestratorResult` and `apply-migrations.ts` persists a canonical `{version, status, phases}` shape after return. Orchestrators no longer call `appendCompletedMigration` directly. `statusForVersion` prefers `complete` over `partial` (never regresses). 3 consecutive partials → wedged → `--force-retry ` writes a `'retry'` reset marker. v0.14.3 (fix wave) ships schema-only migrations v14 (`pages_updated_at_index`) + v15 (`minion_jobs_max_stalled_default_5` with UPDATE backfill) via the `MIGRATIONS` array in `src/core/migrate.ts` — no orchestrator phases needed. - `src/commands/repair-jsonb.ts` — `gbrain repair-jsonb [--dry-run] [--json]`: rewrites `jsonb_typeof='string'` rows in place across 5 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter). Fixes v0.12.0 double-encode bug on Postgres; PGLite no-ops. Idempotent. @@ -174,7 +174,7 @@ strict behavior when unset. - `src/commands/transcripts.ts` (v0.29) — `gbrain transcripts recent [--days N] [--full] [--json]`: recent raw `.txt` transcripts from the dream-cycle corpus dirs. Imports `listRecentTranscripts` from `src/core/transcripts.ts` (the same library the gated `get_recent_transcripts` MCP op uses). Local-only by construction — the CLI always runs with `ctx.remote=false`. - `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s). Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable. **v0.32.8 (PR #860):** batch projection switched from `SELECT DISTINCT ON (slug)` to `SELECT ... ORDER BY source_id, slug` so multi-source brains scan each `(source, slug)` row independently (pre-fix the DISTINCT collapsed same-slug-different-source pages into one scan, the same bug class this PR fixes). Sequential and auto-repair loops use `listAllPageRefs()` to enumerate `(slug, source_id)` pairs and thread `sourceId` to `getPage`. Batch + sequential paths now report the same page count on multi-source brains. - `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count. v0.26.7 (#612): `rls_event_trigger` check (post-install drift detector for migration v35's auto-RLS event trigger). Lives outside the `// 5. RLS` slice that the structural doctor.test.ts guards anchor on, so the existing test guards stay intact. Healthy `evtenabled` set is `('O','A')` only — `R` is replica-only and would not fire in normal sessions; `D` is disabled. Fix hint is `gbrain apply-migrations --force-retry 35`. **v0.30.2:** `queue_health` gains a fourth subcheck — surfaces dead-lettered subagent jobs with `last_error` matching the `prompt_too_long` classifier within the last 24h. Fix hint points at `gbrain dream --phase synthesize --dry-run --json` to identify the offending transcript and `gbrain jobs prune --status dead --queue default` to clean up. Postgres-only. **v0.31.7:** `runDoctor` switches to `autoDetectSkillsDirReadOnly` (from `src/core/repo-root.ts`) so `bun install -g github:garrytan/gbrain && cd ~ && gbrain doctor` finds the bundled `skills/` via the install-path fallback instead of warning "Could not find skills directory" + docking the health score. `--fix` carries a D6 safety gate: when `detected.source === 'install_path'`, the command refuses auto-repair with a stderr message pointing at `$GBRAIN_SKILLS_DIR` / `$OPENCLAW_WORKSPACE` / `--skills-dir`, because `autoFixDryViolations` writes to SKILL.md files and would otherwise silently rewrite the install tree. The `graph_coverage` check now short-circuits to `ok: 'No entity pages — graph_coverage not applicable (markdown-only brain)'` when `SELECT COUNT(*) FROM pages WHERE type IN ('entity','person','company','organization')` returns 0 (closes #530); the entity count is woven into the warn message and the WARN hint switches from the long-deprecated `gbrain link-extract && gbrain timeline-extract` (gone since v0.16) to the canonical `gbrain extract all`. Pinned by an IRON-RULE regression assertion in `test/doctor.test.ts` that bans the stale verb names from the source string. **v0.32.4:** new `sync_freshness` check (exported `checkSyncFreshness` at the same file) added to both `runDoctor` (local) and `doctorReportRemote` (thin-client). Pure staleness probe — queries `sources.last_sync_at` only, no filesystem access. Warns at 24h, fails at 72h (or never-synced). Future-`last_sync_at` warns ("clock skew or corrupted timestamp") instead of silently falling through as ok — codex outside-voice caught the negative-ageMs bug pre-merge. Env-var overrides `GBRAIN_SYNC_FRESHNESS_WARN_HOURS` / `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS`; invalid values fall back to defaults with a once-per-process stderr warn (`_resolveSyncFreshnessHours`). Failure messages embed `source.id` (not `source.name`) so the printed fix command `gbrain sync --source ` matches what the user copy-pastes. Filesystem-vs-DB page drift detection was deliberately stripped from the v0.32.4 scope — `doctorReportRemote` runs in the HTTP MCP server (`src/commands/serve-http.ts`), and walking DB-supplied `local_path` from a remote-callable endpoint crosses a trust boundary (OAuth write scope could mutate `sources.local_path`). Drift detection will resurface in a separate PR routed through `multi_source_drift`'s existing guard infrastructure (`GBRAIN_DRIFT_LIMIT` / `GBRAIN_DRIFT_TIMEOUT_MS`) with slug normalization tests and a meta-file allow-list. Pinned by 12 cases in `test/doctor.test.ts` ("v0.32.4 — sync_freshness check" describe block): empty sources, never-synced fail, >72h fail, exact 72h boundary, 24h-72h warn, exact 24h boundary, <24h ok, future-timestamp warn, mixed sources (highest severity wins), `executeRaw` throws → outer-catch warn, `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS=6` override fires at 7h, source.id-in-message regression. -- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). **v40 (v0.29):** `pages_emotional_weight` adds `pages.emotional_weight REAL NOT NULL DEFAULT 0.0`. Column-only (no index). On Postgres 11+ and PGLite, `ADD COLUMN` with a constant DEFAULT is metadata-only — instant on tables of any size. v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs. **v35 (v0.26.7):** auto-RLS event trigger + one-time backfill. `auto_rls_on_create_table` fires on `ddl_command_end` for `WHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO')` and runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY` on every new `public.*` table — no FORCE (matches v24/v29/schema.sql posture so non-BYPASSRLS apps can still read their own tables). The same migration backfills RLS on every existing `public.*` base table whose comment doesn't match the doctor regex (`^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,}`). Per-table failure aborts the offending CREATE TABLE (event triggers fire inside the DDL transaction); no EXCEPTION wrap — that would convert loud rollback into silent permissive default. PGLite no-op via `sqlFor.pglite: ''`. Breaking change: operators with intentionally-RLS-off public tables must add the GBRAIN:RLS_EXEMPT comment BEFORE upgrade or the backfill will flip them on. **v46 (v0.31.3):** `mcp_request_log_params_jsonb_normalize` rewrites pre-v0.31.3 rows where `mcp_request_log.params` was stored as a JSON-encoded string (`jsonb_typeof = 'string'`) up to a real JSONB object via `UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string'`. Single statement, idempotent — second-run finds no string-shaped rows and is a no-op. Closes the bug where `/admin/api/requests` returned a quoted string instead of the parsed object. +- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). **v40 (v0.29):** `pages_emotional_weight` adds `pages.emotional_weight REAL NOT NULL DEFAULT 0.0`. Column-only (no index). On Postgres 11+ and PGLite, `ADD COLUMN` with a constant DEFAULT is metadata-only — instant on tables of any size. v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs. **v35 (v0.26.7):** auto-RLS event trigger + one-time backfill. `auto_rls_on_create_table` fires on `ddl_command_end` for `WHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO')` and runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY` on every new `public.*` table — no FORCE (matches v24/v29/schema.sql posture so non-BYPASSRLS apps can still read their own tables). The same migration backfills RLS on every existing `public.*` base table whose comment doesn't match the doctor regex (`^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,}`). Per-table failure aborts the offending CREATE TABLE (event triggers fire inside the DDL transaction); no EXCEPTION wrap — that would convert loud rollback into silent permissive default. PGLite no-op via `sqlFor.pglite: ''`. Breaking change: operators with intentionally-RLS-off public tables must add the GBRAIN:RLS_EXEMPT comment BEFORE upgrade or the backfill will flip them on. **v46 (v0.31.3):** `mcp_request_log_params_jsonb_normalize` rewrites pre-v0.31.3 rows where `mcp_request_log.params` was stored as a JSON-encoded string (`jsonb_typeof = 'string'`) up to a real JSONB object via `UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string'`. Single statement, idempotent — second-run finds no string-shaped rows and is a no-op. Closes the bug where `/admin/api/requests` returned a quoted string instead of the parsed object. **v0.34.1.0 (#861 + #876, v60-v65):** six-migration chain wires source-scoping into the OAuth client table. v60 (`oauth_clients_source_id_fk`) adds `oauth_clients.source_id TEXT` with NULL→`'default'` backfill and an FK to `sources(id) ON DELETE SET NULL`. v61 (`oauth_clients_federated_read_column`) adds `federated_read TEXT[] NOT NULL DEFAULT '{}'`. v62 (`oauth_clients_federated_read_backfill`) explicit-CASE backfills so `source_id IS NULL` produces `'{}'` not an array-containing-NULL. v63 (`oauth_clients_federated_read_validate`) is the fail-loud check that every row's source_id is in its federated_read array post-backfill. v64 (`oauth_clients_source_id_fk_restrict`) flips the FK to `ON DELETE RESTRICT` now that federated_read provides the alternative scope-loss path — source delete is refused if any client references it. v65 (`oauth_clients_federated_read_gin_index`) is the GIN index for the array-containment queries the read paths run. PGLite parity via `sqlFor.pglite` where needed. - `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2. - `src/core/cli-options.ts` — Global CLI flag parser. `parseGlobalFlags(argv)` returns `{cliOpts, rest}` with `--quiet` / `--progress-json` / `--progress-interval=` stripped. `getCliOptions()` / `setCliOptions()` expose a module-level singleton so commands reach the resolved flags without parameter threading. `cliOptsToProgressOptions()` maps to reporter options. `childGlobalFlags()` returns the flag suffix to append to `execSync('gbrain ...')` calls in migration orchestrators. `OperationContext.cliOpts` extends shared-op dispatch for MCP callers. - `src/core/db-lock.ts` (v0.22.13) — generic `tryAcquireDbLock(engine, lockId, ttlMinutes)` over the existing `gbrain_cycle_locks` table. Parameterized lock id so different scopes can nest cleanly: `gbrain-cycle` for the broad cycle (held by `cycle.ts`) and `gbrain-sync` (`SYNC_LOCK_ID` constant) for `performSync`'s narrower writer window. Same UPSERT-with-TTL semantics as the prior cycle-only helper, just generalized. Survives PgBouncer transaction pooling (unlike session-scoped `pg_try_advisory_lock`); crashed holders auto-release once their TTL expires. @@ -649,6 +649,10 @@ E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_U - `test/e2e/serve-http-oauth.test.ts` (v0.26.0, expanded v0.26.2, expanded v0.26.9) — real-Postgres E2E against `gbrain serve --http` with full OAuth 2.1. Spawns a subprocess server, registers a client via the CLI, mints `client_credentials` tokens, exercises the `/mcp` JSON-RPC pipeline. **v0.26.2 adds:** real DCR `/register` HTTP-level response-shape test (asserts `typeof body.client_id_issued_at === 'number'` over the wire — RFC 7591 §3.2.1 spec compliance, not just internal-store shape); real CLI subprocess test for `revoke-client` (registers → mints token → revokes via `execSync` → asserts token rejected at `/mcp` → asserts re-run exits 1); server fixture flips on `--enable-dcr` so `/register` is reachable. **bun execSync env-inheritance fix:** bun's `execSync` does NOT inherit env mutations done via `process.env.X = ...`, only OS-level env from before bun started. helpers.ts loads `.env.testing` and sets `DATABASE_URL` via `process.env` mutation, which is invisible to subprocesses unless `env: { ...process.env }` is passed explicitly — every subprocess call in this file passes `env: { ...process.env }` for that reason. Reference fix for the next maintainer hitting the same failure mode in sibling sync/cycle/dream/claw-test E2Es. `afterAll` cleanup is guarded on `clientId` (won't throw if `beforeAll` failed before registration); cleanup errors surface to stderr without throwing so real test failures aren't masked. Tracks DCR-registered clients alongside the manual one. **v0.26.9** adds 2 regressions for the F7 trust-boundary fix: an HTTP MCP `submit_job` for `name: "shell"` MUST reject with a permission error (proving the request handler now sets `remote: true` and `submit_job`'s protected-name guard fires), and the same guard rejects subagent submission. Closes the OAuth-token-to-RCE escalation path. Skips gracefully when `DATABASE_URL` is unset. - `test/e2e/sync-parallel.test.ts` (v0.22.13 PR #490) — DATABASE_URL-gated. T2: 60-file Postgres sync at concurrency=4 imports all + no connection leak (probes `pg_stat_activity` before/after to confirm worker engines disconnected). P4: 120-file serial-vs-parallel benchmark prints `SYNC_PARALLEL_BENCH N files | serial=Xms | parallel(4)=Yms | speedup=Zx` for CHANGELOG quoting. Asserts parallel ≤ serial × 1.5 (CI-noise tolerant; not a strict speedup gate). - `test/e2e/multi-source-bug-class.test.ts` (v0.32.8, PR #860) — 7-case PGLite in-memory regression suite pinning every bug site fixed in this PR: `listAllPageRefs` ordering by `(source_id, slug)` (F11), `getPage` with sourceId picks the right `(source, slug)` row (F2), `extract-takes` processes both overlapping `people/alice` rows independently, `listPages` filters correctly with `PageFilters.sourceId`, `addLinksBatch` with `from/to_source_id` targets the right rows (F4), `validateSourceId` rejects path traversal (F6), reverse-write disk layout uses `brainDir/.sources//.md` for non-default sources (F6). No DATABASE_URL needed. Wired into `scripts/e2e-test-map.ts` so changes to extract-takes / patterns / synthesize / embed / extract / migrate-engine auto-trigger this test. Companion: `test/e2e/integrity-batch.test.ts`'s "multi-source duplicate slugs scan once" case was pinning the pre-fix bug — assertion flipped in v0.32.8 to expect both batch + sequential paths report 2. +- `test/e2e/source-isolation-pglite.test.ts` (v0.34.1.0, #861) — 14-case PGLite in-memory regression suite pinning the source-isolation P0 seal at two layers. Engine layer: `searchKeyword` / `searchVector` / `searchKeywordChunks` / `listPages` / `getPage` / `traverseGraph` / `traversePaths` apply `sourceId` (scalar fast path) and `sourceIds` (array path) correctly across both engines. Op-handler layer: routes through `sourceScopeOpts(ctx)` so a `read+write`-scoped OAuth client bound to `--source dept-x` cannot see rows from neighboring sources via `search`, `query`, `list_pages`, `get_page`, or `find_experts`. Covers both `ctx.sourceId` (single-source clients) and `ctx.auth.allowedSources` (federated_read clients) precedence; federated array wins over scalar wins over nothing. No DATABASE_URL needed. +- `test/openai-compat-multimodal.test.ts` (v0.34.1.0, #875) — 11-case unit suite for the gateway's openai-compatible multimodal path: happy-path single + multi-input embedding, unauthenticated proxy mode, dimension-mismatch guard (D12; throws `AIConfigError` with model id + observed + expected pre-storage), default-dim fallback when recipe declares `default_dims`, HTTP 401 / 400 / malformed-JSON / non-array error paths, plus a regression test that the existing Voyage `/multimodalembeddings` recipe still routes through its dedicated path (not the openai-compatible one). Hermetic via the `__setEmbedTransportForTests` seam. +- `test/serve-stdio-lifecycle.test.ts` (extended v0.34.1.0, #870) — adds 3 new cases for the `MCP_STDIO=1` env guard: stdin EOF does NOT trigger shutdown when the env is set, SIGTERM still does (guard scope is correct), unset env preserves the pre-v0.34 CLI lifecycle. Exercises the `ServeOptions.mcpStdio?: boolean` test seam directly so tests don't mutate `process.env`. +- `test/oauth.test.ts` (extended v0.34.1.0, #909) — 5 new cases for the PKCE DCR public-client gate: `registerClient` with `token_endpoint_auth_method: "none"` returns no `client_secret` field on the public client, default `client_secret_post` clients still get the one-time-reveal secret, `getClient` NULL→undefined normalization so the SDK's clientAuth path accepts public clients, full PKCE `/authorize` → `/token` round-trip against a public client (no client_secret presented), and a regression test that the public-vs-confidential branch doesn't break confidential client `client_secret_post` exchange. - Tier 2 (`skills.test.ts`) requires OpenClaw + API keys, runs nightly in CI - If `.env.testing` doesn't exist in this directory, check sibling worktrees for one: `find ../ -maxdepth 2 -name .env.testing -print -quit` and copy it here if found. diff --git a/README.md b/README.md index d86cc71..482c9bd 100644 --- a/README.md +++ b/README.md @@ -139,7 +139,9 @@ pick scopes, save the credentials shown once in the reveal modal. Programmatic registration via `oauthProvider.registerClientManual(...)` and the `gbrain auth register-client` CLI are also available. -- **OAuth 2.1 via the MCP SDK** — client credentials (machine-to-machine: Perplexity, Claude), authorization code + PKCE (browser-based: ChatGPT), refresh token rotation, revocation, protected resource metadata. Optional Dynamic Client Registration behind `--enable-dcr` (DCR redirect_uris must be `https://` or loopback per RFC 6749 §3.1.2.1). +- **OAuth 2.1 via the MCP SDK** — client credentials (machine-to-machine: Perplexity, Claude), authorization code + PKCE (browser-based: ChatGPT), refresh token rotation, revocation, protected resource metadata. PKCE-only public clients (`token_endpoint_auth_method: "none"`) register without a secret per RFC 7591 §3.2.1 (v0.34). Optional Dynamic Client Registration behind `--enable-dcr` (DCR redirect_uris must be `https://` or loopback per RFC 6749 §3.1.2.1). +- **Source-scoped OAuth clients (v0.34)** — `gbrain auth register-client my-agent --source dept-x` ties the client's write authority to one source; read paths only return rows matching that source. `--federated-read S1,S2,S3` adds an orthogonal read-scope axis for shared brains (departments writing to one canon while reading the union). Pre-v0.34 clients are backfilled to `source_id='default'` on upgrade. +- **Loopback default for `serve --http` (v0.34)** — listens on `127.0.0.1` unless `--bind 0.0.0.0` (or a specific interface IP). Personal-laptop installs no longer publish the brain to the LAN by accident. A stderr WARN fires when `--public-url` is set without `--bind` so the operator sees the binding before the first request. - **Scoped operations** — 30 operations tagged `read | write | admin`. `sync_brain` and `file_upload` are `localOnly`, rejected over HTTP. - **React admin dashboard** — 7 screens baked into the binary (~65KB gzip). Live SSE activity feed, agents table, credential reveal, filterable request log, per-client config export. - **Legacy bearer tokens still work** — pre-v0.26 `gbrain auth create` tokens continue to authenticate as `read+write+admin`. v0.22.7's simpler `src/mcp/http-transport.ts` path stays compiled in for backward compat callers; v0.26+ deployments use the OAuth-aware `serve-http.ts`. @@ -788,12 +790,16 @@ ADMIN [--skip=] [--json] gbrain serve MCP server (stdio) gbrain serve --http [--port 3131] HTTP MCP server with OAuth 2.1 + admin dashboard + [--bind HOST] (v0.34: default 127.0.0.1; pass + --bind 0.0.0.0 for LAN/remote access) [--token-ttl 3600] [--enable-dcr] [--public-url URL] [--log-full-params] gbrain auth create|list|revoke|test Legacy bearer token management gbrain auth register-client Register an OAuth 2.1 client --grant-types client_credentials,authorization_code --scopes "read write admin" + --source v0.34: write authority for source-scoped clients + --federated-read v0.34: read scope across multiple sources gbrain auth revoke-client Revoke an OAuth 2.1 client (cascade purges active tokens + auth codes via FK CASCADE) # OAuth 2.1 clients can also be registered from the /admin dashboard or diff --git a/SECURITY.md b/SECURITY.md index a271c63..7d89947 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -68,6 +68,16 @@ The built-in HTTP transport ships with several layers of hardening on by default. All env vars below are optional; the defaults are intentionally conservative. +### Bind address (v0.34: loopback by default) + +`gbrain serve --http` listens on `127.0.0.1` by default. Personal-laptop +installs cannot accidentally publish the brain to the LAN. Self-hosted +deployments that need remote access pass `--bind 0.0.0.0` (all +interfaces) or `--bind ` (specific NIC). A stderr WARN +fires when `--public-url` is set without `--bind` so the operator sees +the binding before the first request — common cause of "ngrok forwards +to me but the agent can't reach the upstream" misconfigurations. + ### Postgres-only `gbrain serve --http` requires a Postgres engine. PGLite is local-only by @@ -125,9 +135,11 @@ GBRAIN_HTTP_TRUST_PROXY=1 gbrain serve --http --port 8787 **both** of these are true: 1. gbrain is reachable only via a trusted reverse proxy (not directly - exposed to the internet on the configured port). The simplest - guarantee is to bind gbrain to `127.0.0.1` or a private interface - and have the proxy forward to it. + exposed to the internet on the configured port). As of v0.34 + `gbrain serve --http` binds `127.0.0.1` by default, so the + reverse-proxy-only posture is the out-of-the-box shape; only + override with `--bind 0.0.0.0` (or a specific interface IP) when + gbrain itself needs to accept remote connections directly. 2. The proxy strips any client-supplied `X-Forwarded-For` and `X-Real-IP` headers, then sets them itself. (nginx with `proxy_set_header X-Forwarded-For $remote_addr` does this; Cloudflare and most cloud diff --git a/TODOS.md b/TODOS.md index 46e5950..786ccf5 100644 --- a/TODOS.md +++ b/TODOS.md @@ -1,5 +1,21 @@ # TODOS +## MCP fix wave follow-ups (v0.34.1) + +- [ ] **v0.34.x: Source-scope `takes_*` ops (pre-existing leak surfaced during v0.34.1 adversarial review).** `takes_list`, `takes_search`, `takes_scorecard`, `takes_calibration` in `src/core/operations.ts:1248-1335` thread `ctx.takesHoldersAllowList` but never `ctx.sourceId`. An auth'd OAuth client scoped to `source_id='canon-a'` can call `takes_list --page_slug=foo` (slug in `canon-b`) and read takes attached to foreign-source pages. Pre-existing, not introduced by v0.34.1, but the wave was framed as "P0 source-isolation seal on the read path" and `takes_*` surfaces were missed. Fix: extend `TakesListOpts` in `src/core/engine.ts:186` with `sourceId?: string` + `sourceIds?: string[]`; thread `sourceScopeOpts(ctx)` at each op handler; engine `listTakes`/`searchTakes` filter via the `pages` JOIN. + +- [ ] **v0.34.x: Extend `sourceScopeOpts(ctx)` to the 14 read-side ops PR #861 didn't touch.** `get_page`, `get_tags`, `get_links`, `get_backlinks`, `get_timeline`, `list_files`, `get_file`, and the four `takes_*` ops (above) still use the v0.31.8-era `const sourceOpts = ctx.sourceId ? { sourceId: ctx.sourceId } : {}` pattern. NOT a leak (scalar `ctx.sourceId` IS threaded), but federated_read (#876, `ctx.auth?.allowedSources`) is silently dropped. A "WeCare L3 dept" client gets correct federated results from `search`/`query`/`list_pages`/`traverse_graph`/`find_experts` but only sees its scalar `source_id` for `get_page`/`get_tags`/etc. Fix: route all 14 sites through `sourceScopeOpts(ctx)`. + +- [ ] **v0.34.x: Migration v60 idempotency guard against `--force-retry` race with v64.** `gbrain apply-migrations --force-retry 58` after v64 has already run will re-install the FK with `ON DELETE SET NULL`, silently downgrading the v64 RESTRICT posture. Probability low (operator has to explicitly force-retry 58) but failure mode is invisible. Fix: v60 should probe `pg_constraint.confdeltype` before re-adding and refuse to clobber `'r'` (RESTRICT) with `'n'` (SET NULL). + +- [ ] **v0.34.x: `embedMultimodalOpenAICompat` batching + partial-failure handling.** `src/core/ai/gateway.ts:1180-1255` sends one HTTP request per input. Multi-input callers (10 images) get 10 sequential round-trips with no parallelism; a 401 on input #5 throws and discards inputs #1-#4's already-computed embeddings (wasted spend, no surfacing of the partial array). Voyage's existing path batches. Fix: batch via the provider's `input: [...]` array shape; on partial failure, return successful embeddings + failed-index array. + +- [ ] **v0.34.x: Doctor check `oauth_orphan_source_id`** — surfaces OAuth clients whose source_id was nulled by the v60 D10 silent-widen path (`GBRAIN_ACCEPT_SILENT_WIDEN=1`). Closes the observability gap from v0.34.1's D4 decision. Sibling to the `rls_event_trigger` check pattern in `src/commands/doctor.ts`. + +- [ ] **v0.34.x: `gbrain sources purge` FK error UX.** Post-v0.34, deleting a source is refused if any oauth_client references it (v64 ON DELETE RESTRICT). The CLI currently surfaces the raw Postgres FK violation. Fix: pre-check via `SELECT client_id, client_name FROM oauth_clients WHERE source_id = $1`, print "N OAuth clients reference this source: ... Revoke first via `gbrain auth revoke-client `." Mirrors `assessDestructiveImpact` in destructive-guard.ts (v0.26.5). + +- [ ] **v0.34.x: `hybrid.ts:223` explicit-pick refactor.** The SearchOpts rebuild manually picks fields from HybridSearchOpts. This is the bug shape that caused the original v0.34.1 P0 leak — a new SearchOpts field is silently dropped if not manually added here. The wave added `sourceId` + `sourceIds` to the pick; future fields will keep hitting this footgun. Fix: refactor to spread + TypeScript `Pick<>` helper that narrows HybridSearchOpts → SearchOpts type-safely. + ## functional-area-resolver follow-ups (v0.32.3.0) - [ ] **v0.33.x: Dogfood `functional-area-resolver` on gbrain's own `skills/RESOLVER.md`** when it crosses ~12KB (currently 8KB). Apply the pattern to the Operational section first (largest). Filed during v0.32.3.0 CEO review. diff --git a/VERSION b/VERSION index cedba06..a68a9c7 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.34.0.0 +0.34.1.0 \ No newline at end of file diff --git a/docs/architecture/topologies.md b/docs/architecture/topologies.md index 8e4e0dd..8782eea 100644 --- a/docs/architecture/topologies.md +++ b/docs/architecture/topologies.md @@ -120,10 +120,19 @@ at the remote host. `gbrain doctor` runs a dedicated thin-client check set ```bash gbrain init --supabase # or --pglite, doesn't matter -gbrain serve --http --port 3001 # exposes /mcp + OAuth +gbrain serve --http --port 3001 --bind 0.0.0.0 # v0.34: bind explicitly for remote access + # (defaults to 127.0.0.1 since v0.34) gbrain auth register-client neuromancer \ --grant-types client_credentials \ --scopes read,write,admin # admin needed for ping/doctor + +# v0.34: source-scoped client (write to one source, federate reads across +# multiple sources). Omit both flags for a v0.33-compatible super-client. +gbrain auth register-client neuromancer-dept \ + --grant-types client_credentials \ + --scopes read,write \ + --source dept-x \ + --federated-read dept-x,shared,parent-canon ``` The `register-client` command prints a `client_id` and `client_secret`. diff --git a/docs/mcp/DEPLOY.md b/docs/mcp/DEPLOY.md index 4f508bf..deec780 100644 --- a/docs/mcp/DEPLOY.md +++ b/docs/mcp/DEPLOY.md @@ -117,6 +117,24 @@ gbrain auth register-client perplexity \ --scopes "read write" ``` +**v0.34 — source-scoped clients.** Multi-source brains can scope a client's +write authority to one source and its read scope to a curated set with the +new `--source` and `--federated-read` flags: + +```bash +gbrain auth register-client dept-x-agent \ + --grant-types client_credentials \ + --scopes "read write" \ + --source dept-x \ + --federated-read dept-x,shared,parent-canon +``` + +`--source` controls the write authority — `put_page` / `add_link` / etc only +land in `dept-x`. `--federated-read` controls the read axis independently; +queries return rows from any of the listed sources. Omit both flags for the +v0.33-compatible super-client shape. Pre-v0.34 clients are backfilled to +`source_id='default'` on `gbrain upgrade`. + Host-repo wrappers can register programmatically: ```ts @@ -133,6 +151,18 @@ start the server with `--enable-dcr`. DCR is off by default. ### 3. Expose the server +**v0.34 — bind explicitly.** `gbrain serve --http` defaults to `127.0.0.1`. +To accept connections from the ngrok tunnel (or any non-loopback source), +restart with `--bind`: + +```bash +gbrain serve --http --port 3131 --bind 0.0.0.0 --public-url https://your-brain.ngrok.app +``` + +When `--public-url` is set without `--bind`, a stderr WARN fires at +startup so the misconfiguration ("the tunnel is up but my agent gets +ECONNREFUSED") is loud. + ```bash brew install ngrok ngrok config add-authtoken YOUR_TOKEN diff --git a/llms-full.txt b/llms-full.txt index 2cbcf1c..eae6f6b 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -148,8 +148,8 @@ strict behavior when unset. ## Key files -- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (v0.23 glob matcher: `/*` matches recursive children; bare `` matches exact only). `OperationContext.remote` flags untrusted callers; `OperationContext.allowedSlugPrefixes` (v0.23) is the trusted-workspace allow-list set by the dream cycle. `put_page` enforces: when `viaSubagent` and `allowedSlugPrefixes` is set, slug must match the allow-list; else the legacy `wiki/agents//...` namespace check applies. Auto-link enabled for trusted-workspace writes (skipped only when `remote=true && !trustedWorkspace`). As of v0.26.0, every `Operation` also carries `scope?: 'read' | 'write' | 'admin'` + `localOnly?: boolean`. All ops are annotated; `sync_brain`, `file_upload`, `file_list`, and `file_url` are `admin + localOnly` (rejected over HTTP). `OperationContext.auth?: AuthInfo` is threaded through HTTP dispatch for scope enforcement in `serve-http.ts` before the op runs. **v0.26.9 (D12 + F7b):** `OperationContext.remote` is now a REQUIRED field in the TypeScript type — the compiler is the first defense against transports that forget to set it. Four trust-boundary call sites (`put_page` allowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) flipped from falsy-default (`!ctx.remote`) to fail-closed semantics (`ctx.remote === false` for "trusted-only" sites and `ctx.remote !== false` for "untrust unless explicit-false"). Anything that isn't strictly `false` is now treated as remote. Closed an HTTP MCP shell-job RCE: a `read+write`-scoped OAuth token could submit `shell` jobs because the HTTP request handler's literal context skipped `remote: true` and `submit_job`'s protected-name guard saw a falsy undefined. Stdio MCP set the field correctly via dispatch.ts; HTTP inlined a parallel context-builder for several releases and lost it. -- `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput` / `TimelineBatchInput` for the v0.12.1 bulk-insert API (`addLinksBatch` / `addTimelineEntriesBatch`). As of v0.13.1, `BrainEngine` has a `readonly kind: 'postgres' | 'pglite'` discriminator so migrations (`src/core/migrate.ts`) and other consumers can branch on engine without `instanceof` + dynamic imports. **v0.29:** four new methods — `batchLoadEmotionalInputs(slugs?)` (CTE-shaped read with per-table aggregates so a page × N tags × M takes never produces N×M rows), `setEmotionalWeightBatch(rows)` (`UPDATE FROM unnest($1::text[], $2::text[], $3::real[])` composite-keyed on `(slug, source_id)` for multi-source safety), `getRecentSalience(opts)`, `findAnomalies(opts)`. `PageFilters` extended with `sort?: 'updated_desc' | 'updated_asc' | 'created_desc' | 'slug'` + `PAGE_SORT_SQL` whitelist consumed by both engines (was hardcoded `ORDER BY updated_at DESC`). **v0.32.8 (PR #860):** new `listAllPageRefs(): Promise>` ordered by `(source_id, slug)`. Cheap cross-source enumeration for hot loops on large brains — replaces the `getAllSlugs()→getPage(slug)` N+1 pattern in extract-takes, extract, integrity, which silently defaulted to `source_id='default'` for non-default-source pages. Implementation parity across postgres-engine.ts + pglite-engine.ts. Pinned by `test/e2e/multi-source-bug-class.test.ts`. +- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (v0.23 glob matcher: `/*` matches recursive children; bare `` matches exact only). `OperationContext.remote` flags untrusted callers; `OperationContext.allowedSlugPrefixes` (v0.23) is the trusted-workspace allow-list set by the dream cycle. `put_page` enforces: when `viaSubagent` and `allowedSlugPrefixes` is set, slug must match the allow-list; else the legacy `wiki/agents//...` namespace check applies. Auto-link enabled for trusted-workspace writes (skipped only when `remote=true && !trustedWorkspace`). As of v0.26.0, every `Operation` also carries `scope?: 'read' | 'write' | 'admin'` + `localOnly?: boolean`. All ops are annotated; `sync_brain`, `file_upload`, `file_list`, and `file_url` are `admin + localOnly` (rejected over HTTP). `OperationContext.auth?: AuthInfo` is threaded through HTTP dispatch for scope enforcement in `serve-http.ts` before the op runs. **v0.26.9 (D12 + F7b):** `OperationContext.remote` is now a REQUIRED field in the TypeScript type — the compiler is the first defense against transports that forget to set it. Four trust-boundary call sites (`put_page` allowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) flipped from falsy-default (`!ctx.remote`) to fail-closed semantics (`ctx.remote === false` for "trusted-only" sites and `ctx.remote !== false` for "untrust unless explicit-false"). Anything that isn't strictly `false` is now treated as remote. Closed an HTTP MCP shell-job RCE: a `read+write`-scoped OAuth token could submit `shell` jobs because the HTTP request handler's literal context skipped `remote: true` and `submit_job`'s protected-name guard saw a falsy undefined. Stdio MCP set the field correctly via dispatch.ts; HTTP inlined a parallel context-builder for several releases and lost it. **v0.34.1.0 (#861 + #876):** new helper `sourceScopeOpts(ctx)` encodes the precedence ladder for source-scoped reads — federated array (`ctx.auth.allowedSources`) wins over scalar (`ctx.sourceId` / `ctx.auth.sourceId`) over nothing. Every read-side op handler routes through it so future ops can't silently drift from the canonical v0.31.8 thread. Closes the source-isolation leak on the read path: a `read+write`-scoped OAuth client bound to `--source dept-x` no longer sees rows from neighboring sources via `search` / `query` / `list_pages` / `get_page` / `find_experts` / `query`'s image path. +- `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput` / `TimelineBatchInput` for the v0.12.1 bulk-insert API (`addLinksBatch` / `addTimelineEntriesBatch`). As of v0.13.1, `BrainEngine` has a `readonly kind: 'postgres' | 'pglite'` discriminator so migrations (`src/core/migrate.ts`) and other consumers can branch on engine without `instanceof` + dynamic imports. **v0.29:** four new methods — `batchLoadEmotionalInputs(slugs?)` (CTE-shaped read with per-table aggregates so a page × N tags × M takes never produces N×M rows), `setEmotionalWeightBatch(rows)` (`UPDATE FROM unnest($1::text[], $2::text[], $3::real[])` composite-keyed on `(slug, source_id)` for multi-source safety), `getRecentSalience(opts)`, `findAnomalies(opts)`. `PageFilters` extended with `sort?: 'updated_desc' | 'updated_asc' | 'created_desc' | 'slug'` + `PAGE_SORT_SQL` whitelist consumed by both engines (was hardcoded `ORDER BY updated_at DESC`). **v0.32.8 (PR #860):** new `listAllPageRefs(): Promise>` ordered by `(source_id, slug)`. Cheap cross-source enumeration for hot loops on large brains — replaces the `getAllSlugs()→getPage(slug)` N+1 pattern in extract-takes, extract, integrity, which silently defaulted to `source_id='default'` for non-default-source pages. Implementation parity across postgres-engine.ts + pglite-engine.ts. Pinned by `test/e2e/multi-source-bug-class.test.ts`. **v0.34.1.0 (#861):** `SearchOpts` + `PageFilters` add `sourceIds?: string[]` for the federated read axis; both engines apply `WHERE source_id = ANY($N::text[])` when the array is set and preserve the scalar `sourceId` fast path when unset. `traverseGraph(slug, depth, opts?)` and `traversePaths(slug, opts?)` accept `opts.sourceId` / `opts.sourceIds` so graph walks respect the caller's scope. - `src/core/engine-factory.ts` — Engine factory with dynamic imports (`'pglite'` | `'postgres'`) - `src/core/pglite-engine.ts` — PGLite (embedded Postgres 17.5 via WASM) implementation, all 40 BrainEngine methods. `addLinksBatch` / `addTimelineEntriesBatch` use multi-row `unnest()` with manual `$N` placeholders. As of v0.13.1, `connect()` wraps `PGlite.create()` in a try/catch that emits an actionable error naming the macOS 26.3 WASM bug (#223) and pointing at `gbrain doctor`; the lock is released on failure so the next process can retry cleanly. v0.22.0: `searchKeyword` and `searchKeywordChunks` multiply `ts_rank` by the source-factor CASE expression at the chunk-grain level; `searchVector` becomes a two-stage CTE — inner CTE keeps `ORDER BY cc.embedding <=> vec` so HNSW stays usable, outer SELECT re-ranks by `raw_score * source_factor`. Inner LIMIT scales with offset to preserve pagination contract. As of v0.22.6.1, `initSchema()` calls `applyForwardReferenceBootstrap()` BEFORE replaying SCHEMA_SQL — probes for the specific forward-referenced state the embedded schema blob needs (`pages.source_id`, `links.link_source`, `links.origin_page_id`, `content_chunks.symbol_name`, `content_chunks.language`, `sources` FK target table) and adds only what's missing. Closes the upgrade-wedge bug class that bit users 10+ times across 6 schema versions over 2 years (#239/#243/#266/#357/#366/#374/#375/#378/#395/#396). No-op on fresh installs and modern brains. - `src/core/pglite-schema.ts` — PGLite-specific DDL (pgvector, pg_trgm, triggers) @@ -202,7 +202,7 @@ strict behavior when unset. - `src/core/embedding.ts` — OpenAI text-embedding-3-large, batch, retry, backoff. **v0.28.7:** `BATCH_SIZE` reverted 50→100 — the original Voyage safety guard halved OpenAI throughput on every page. Per-recipe pre-split + recursive halving + adaptive shrink-on-miss now live in the gateway, so the outer paginator goes back to its original purpose: progress-callback granularity, not batch protection. - `src/core/ai/dims.ts` (v0.33.1.1, PR #962 + #866) — per-provider `providerOptions` resolver for embed-time dimension passthrough. The single source of truth for "which provider needs which knob to produce `vector(N)`". Exports `dimsProviderOptions(implementation, modelId, dims)` (called by `embed()` in `gateway.ts`), `VOYAGE_OUTPUT_DIMENSION_MODELS` (private const — the 7 hosted Voyage models that accept `output_dimension`: `voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-3-large`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-code-3` — nano deliberately excluded), `VOYAGE_VALID_OUTPUT_DIMS = [256, 512, 1024, 2048] as const`, `supportsVoyageOutputDimension(modelId)`, and `isValidVoyageOutputDim(dims)`. **Voyage path uses the SDK-supported `dimensions` field** (`{ openaiCompatible: { dimensions: N } }`), NOT Voyage's `output_dimension` wire-key — the existing `voyageCompatFetch` shim in `gateway.ts:541` translates `dimensions → output_dimension` before the HTTP body is built. The reverse (sending `output_dimension` from here) was the v0.33.1.0 bug class: the AI SDK's openai-compatible adapter doesn't recognize the wire-key so it was silently dropped, Voyage returned its default 1024-dim, and the gateway dimension check threw on every embed call. Runtime guard: when a Voyage flexible-dim model is configured with `dims` outside `VOYAGE_VALID_OUTPUT_DIMS`, throws `AIConfigError` with a paste-ready `gbrain config set embedding_dimensions <256|512|1024|2048>` hint at the embed boundary — fail-loud instead of opaque Voyage HTTP 400. Most common trigger: `embedding_model: voyage:voyage-4-large` configured without `embedding_dimensions` (falls back to `DEFAULT_EMBEDDING_DIMENSIONS=1536`, an OpenAI default not a Voyage one). Eva (@100yenadmin) shipped the wire-key fix in #866; Codex P3 follow-up landed the validator + valid-dims allowlist in #962. - `src/core/ai/types.ts` — provider/recipe types. **v0.28.7 (#680):** `EmbeddingTouchpoint` extended with optional `chars_per_token` (default 4 chars/token, matching OpenAI tiktoken on English) and `safety_factor` (default 0.8, budget-utilization ceiling). Both consulted only when `max_batch_tokens` is also set. Voyage declares `chars_per_token=1` + `safety_factor=0.5` to handle dense payloads (CJK/JSON/base64) that overshoot tiktoken. The pre-split budget is `max_batch_tokens × safety_factor / chars_per_token`. **v0.28.11 (#719):** `EmbeddingTouchpoint.multimodal_models?: string[]` model-level allow-list for recipes that mix text-only + multimodal models under one touchpoint (Voyage's 12 models share `supports_multimodal: true` but only `voyage-multimodal-3` accepts `/multimodalembeddings`). When omitted, recipe-level `supports_multimodal` is sufficient. `AIGatewayConfig.embedding_multimodal_model?: string` lets `embedMultimodal()` route to a different model than `embedding_model` — brains using OpenAI for text can use Voyage for images without flipping the primary embedding pipeline. -- `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state. **v0.33.1.1 (#962, Codex P3 follow-up):** new exported `VoyageResponseTooLargeError` tagged class at the top of the file. `voyageCompatFetch`'s two OOM-defense caps (Layer 1 Content-Length check at `:595`, Layer 2 per-embedding base64 cap at `:619`) now throw `VoyageResponseTooLargeError` instead of a generic `Error`. The inbound response-rewriter's surrounding try/catch (which intentionally swallows parse failures so misshaped Voyage responses fall through to the SDK's JSON parser) checks `instanceof VoyageResponseTooLargeError` and rethrows. Pre-fix, the Layer 2 throw was silently swallowed and the oversized response returned to the AI SDK anyway — Layer 2 was theatrical. Source-shape regression assertion in `test/voyage-response-cap.test.ts` pins the `instanceof ⇒ throw err` line. +- `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state. **v0.33.1.1 (#962, Codex P3 follow-up):** new exported `VoyageResponseTooLargeError` tagged class at the top of the file. `voyageCompatFetch`'s two OOM-defense caps (Layer 1 Content-Length check at `:595`, Layer 2 per-embedding base64 cap at `:619`) now throw `VoyageResponseTooLargeError` instead of a generic `Error`. The inbound response-rewriter's surrounding try/catch (which intentionally swallows parse failures so misshaped Voyage responses fall through to the SDK's JSON parser) checks `instanceof VoyageResponseTooLargeError` and rethrows. Pre-fix, the Layer 2 throw was silently swallowed and the oversized response returned to the AI SDK anyway — Layer 2 was theatrical. Source-shape regression assertion in `test/voyage-response-cap.test.ts` pins the `instanceof ⇒ throw err` line. **v0.34.1.0 (#875):** new `embedMultimodalOpenAICompat()` routes recipes with `implementation: 'openai-compatible'` (LiteLLM, Anyscale, vLLM, Gemini multimodal via proxy) through the standard `/embeddings` endpoint with content arrays carrying `image_url` entries. The pre-existing Voyage `/multimodalembeddings` path is unchanged; the gateway selects by recipe `implementation` tag. Runtime dimension validation throws `AIConfigError` (with model id + observed + expected) before the vector reaches storage when the provider returns a width that doesn't match the recipe's `default_dims` or the brain's `embedding_dimensions` config — no more cryptic `vector dimension mismatch` at INSERT time. Pinned by 11 cases in `test/openai-compat-multimodal.test.ts`. - `src/core/ai/recipes/voyage.ts` — Voyage AI openai-compatible recipe. **v0.28.7 (#680):** declares `chars_per_token=1` + `safety_factor=0.5` so the gateway pre-splits Voyage batches at a 60K-character budget (50% of 120K-token cap with the dense-tokenizer ratio). Closes the v0.27 backfill loop where ~26% of the corpus stayed un-embedded because tiktoken-grounded budgeting silently undercounted Voyage's actual token usage. **v0.28.11 (#719):** declares `multimodal_models: ['voyage-multimodal-3']` so the gateway rejects text-only Voyage models pointed at the multimodal endpoint with a clear `AIConfigError` instead of waiting for Voyage's HTTP 400. **v0.33.1.1 (#962, fixup):** recipe docstring at `:7-16` tightened to name the seven hosted flexible-dim models that accept `output_dimension` explicitly (`voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-3-large`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-code-3`) and call out that `voyage-4-nano` is the open-weight variant listed separately by Voyage as fixed 1024-dim — does NOT accept the parameter. The "all v4 variants are flexible" misread is what caused the original PR to include nano in `VOYAGE_OUTPUT_DIMENSION_MODELS`; the negative regression assertion in `test/ai/gateway.test.ts` (`dimsProviderOptions` returns `undefined` for `voyage-4-nano`) pins the contract. - `src/core/ai/recipes/anthropic.ts` — Anthropic recipe (chat + expansion touchpoints). **v0.31.12:** chat and expansion `models:` lists drop the v0.31.6 phantom `claude-sonnet-4-6-20250929` date suffix — canonical id is `claude-sonnet-4-6`. The wrong-direction alias `claude-sonnet-4-6 → claude-sonnet-4-6-20250929` is removed; a reverse alias `claude-sonnet-4-6-20250929 → claude-sonnet-4-6` keeps stale user configs working (rescues `facts.extraction_model` and `models.dream.synthesize` set by v0.31.6 installs). Recipe-shape regression pinned by `test/anthropic-model-ids.test.ts` (6 cases, verbatim cherry-pick of PR #830 plus the reverse-alias rescue case). - `src/core/anthropic-pricing.ts` — Single source of truth for Anthropic model pricing (per-MTok input/output). **v0.31.12:** Opus 4.7 corrected from `$15/$75` to `$5/$25` (the old number was from Opus 4 generation, never refreshed when 4.7 shipped); Opus 4.6 also corrected. Consumed by `src/core/budget-meter.ts` and `src/core/cross-modal-eval/runner.ts` — the cross-modal estimator now reads `ANTHROPIC_PRICING` for Anthropic models instead of duplicating the table, killing the v0.31.6 drift bug class. @@ -261,15 +261,15 @@ strict behavior when unset. - `src/commands/jobs.ts` — `gbrain jobs` CLI subcommands + `gbrain jobs work` daemon. **v0.28.1:** `case 'work'` now wraps `worker.start()` in try/finally and owns engine lifecycle — calls `engine.disconnect()` on shutdown with loud error logging on failure. Replaces the prior call inside `MinionWorker.start()` (which violated engine ownership: the worker disconnected an engine it didn't own, and clobbered the module-level singleton on PostgresEngine via the now-fixed idempotency bug). Pool slots now free immediately on shutdown instead of waiting for TCP keepalive (~minutes). v0.13.1 surfaces the full `MinionJobInput` retry/backoff/timeout/idempotency surface as first-class CLI flags on `jobs submit`: `--max-stalled`, `--backoff-type fixed|exponential`, `--backoff-delay`, `--backoff-jitter`, `--timeout-ms`, `--idempotency-key`. `jobs smoke --sigkill-rescue` is the opt-in regression guard for #219. v0.16 wires `registerBuiltinHandlers` to always register `subagent` + `subagent_aggregator` (no env flag — `ANTHROPIC_API_KEY` is the natural cost gate, trust is via `PROTECTED_JOB_NAMES`) and loads `GBRAIN_PLUGIN_PATH` plugins at worker startup with a loud startup-line per plugin. `shell` handler still gated by `GBRAIN_ALLOW_SHELL_JOBS=1` (RCE surface, separate concern). v0.22.10 (#521): the `autopilot-cycle` handler now forwards `job.data.phases` to `runCycle` (was previously discarded — caller-supplied phase selection silently became a full cycle). Phases are validated against `ALL_PHASES` from `src/core/cycle.ts`; invalid names are filtered out and an empty/missing array falls back to the default 6-phase cycle. v0.22.13 (PR #490 CODEX-1+CODEX-4): `sync` handler now resolves `sourceId` at entry by looking up `sources.local_path` (mirrors `cycle.ts:480`'s autopilot fix from PR #475) so multi-source brains read the per-source `last_commit` anchor instead of the global config key. Concurrency routed through the shared `autoConcurrency()` policy in `src/core/sync-concurrency.ts` instead of the prior hardcoded `4`; PGLite stays serial. `noEmbed` default is `true` (embed is a separate job — submit `gbrain embed --stale` after sync, or rely on the autopilot cycle's embed phase). - `src/commands/features.ts` — `gbrain features --json --auto-fix`: usage scan + feature adoption salesman - `src/commands/autopilot.ts` — `gbrain autopilot --install`: self-maintaining brain daemon (sync+extract+embed). **v0.28.1:** consumes `detectTini()` from `src/core/minions/spawn-helpers.ts` and resolves it once at startup instead of per worker respawn (was paying an `execFileSync` cost on every restart). -- `src/mcp/server.ts` — MCP stdio server (generated from operations). v0.22.7: tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path. +- `src/mcp/server.ts` — MCP stdio server (generated from operations). v0.22.7: tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path. **v0.34.1.0 (#870):** stdin `'end'` / `'close'` shutdown hooks are skipped when `process.env.MCP_STDIO === '1'`. Gateway-piped stdio MCP wrappers (OpenClaw's `bundle-mcp`, similar) pipe the JSON-RPC handshake then close their stdin half; pre-fix this killed the server before the first tool call landed. Signal handlers (SIGTERM / SIGINT / SIGHUP) and the parent-process watchdog still cover legitimate disconnects. `src/commands/serve.ts` exposes `ServeOptions.mcpStdio?: boolean` as a test seam so the runtime guard is exercisable without process.env mutation. Pinned by `test/serve-stdio-lifecycle.test.ts`. - `src/mcp/dispatch.ts` (v0.22.7) — Shared tool-call dispatch consumed by both stdio (`server.ts`) and HTTP transports. Exports `dispatchToolCall(engine, name, params, opts)`, `buildOperationContext(engine, params, opts)`, and `validateParams(op, params)`. Single source of truth for `(ctx, params)` handler arg order and the 5-field `OperationContext` shape (engine + config + logger + dryRun + remote). Defaults to `remote: true` (untrusted); local CLI callers pass `remote: false`. Closed F1/F2/F3 drift bugs in the original v0.22.5 HTTP transport. **v0.26.9 (F8):** adds `summarizeMcpParams(opName, params)` — privacy-preserving redactor for `mcp_request_log` and the admin SSE feed. Returns `{redacted, kind, declared_keys, unknown_key_count, approx_bytes}`. Intersects submitted top-level keys against the operation's declared `params` allow-list (declared keys preserved as a sorted array for debug visibility; unknown keys counted but never named, closing the attacker-controlled-key-name leak). Byte counts bucketed up to nearest 1KB so an attacker can't binary-search secret-content sizes via repeated probes. Operators on a personal laptop who want raw payload visibility opt back in with `gbrain serve --http --log-full-params` (loud stderr warning at startup). Canonical helper — new logging code paths route through it rather than `JSON.stringify(params)`. - `src/mcp/rate-limit.ts` (v0.22.7) — Bounded-LRU token-bucket limiter. `buildDefaultLimiters()` returns the two-bucket pipeline: pre-auth IP (30/60s, fires BEFORE the DB lookup so brute-force load against `access_tokens` is actually capped) + post-auth token-id (60/60s). Tracks `lastTouchedMs` separately from `lastRefillMs` so an exhausted key can't be reset by hammering past the TTL. LRU cap bounds memory under attacker-controlled key growth. -- `src/commands/serve-http.ts` (v0.26.0) — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. Started via `gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--log-full-params]`. Supersedes the v0.22.7 `src/mcp/http-transport.ts` simple bearer-auth path. Combines MCP SDK's `mcpAuthRouter` (authorize / token / register / revoke endpoints), a custom `client_credentials` handler (SDK's token endpoint throws `UnsupportedGrantTypeError` for CC; the custom handler runs BEFORE the router and falls through for `auth_code` / `refresh_token`), `requireBearerAuth` middleware for `/mcp` with scope enforcement before op dispatch, `localOnly` rejection, and `express-rate-limit` at 50 req / 15 min on `/token`. Serves the built admin SPA from `admin/dist/` with SPA fallback. `/admin/events` SSE endpoint broadcasts every MCP request to connected admin browsers. `cookie-parser` middleware wired (Express 5 has no built-in). Startup logging prints port, engine, configured issuer URL (honors `--public-url`), registered-client count, DCR status, and admin bootstrap token. **v0.26.9 hardening pass:** F7 sets `remote: true` explicitly on the `/mcp` request handler's OperationContext literal (closes the HTTP shell-job RCE — without this, `submit_job`'s protected-name guard at `operations.ts:1391` saw a falsy undefined and skipped, letting a `read+write`-scoped OAuth token submit `shell` jobs). F8 wires `summarizeMcpParams` from `src/mcp/dispatch.ts` into both `mcp_request_log` writes and the admin SSE feed by default (raw payloads opt-in via `--log-full-params` with stderr warning). F9 sets cookie `Secure` flag when behind HTTPS or a public-URL proxy. F10 caps the magic-link nonce store with an LRU bound. F12 routes DCR disable through the `GBrainOAuthProvider` constructor's `dcrDisabled` option instead of the prior monkey-patch on the express router. F14 wraps `transport.handleRequest` in try/catch so SDK throws return a JSON-RPC 500 envelope instead of express's default HTML error page. F15 unifies OperationError + unexpected exceptions through `buildError` / `serializeError` so `/mcp` always returns the same envelope shape. **v0.28.1:** `/health` endpoint extracted into pure `probeHealth(engine)` async function with `HEALTH_TIMEOUT_MS = 3000` exported constant — drops the timeout from 5s to 3s so Fly.io's 5s health-check deadline gets 2s of headroom for TCP, response framing, and clock skew. Races `engine.getStats()` against the timeout via `Promise.race`; saturated pool returns 503 with `Health check timed out (database pool may be saturated)` instead of hanging. `clearTimeout` in finally block prevents pending-timer pile-up under high probe rates (race-leak fix from adversarial review). **v0.28.10:** `/health` is now liveness-only via the new `probeLiveness(sql, engineName, version, timeoutMs)` helper that races `sql\`SELECT 1\`` against `HEALTH_TIMEOUT_MS` and returns the same `ProbeHealthResult` tagged-union as `probeHealth` (single timer-cleanup site, single 503 envelope). Body shape: `{status, version, engine}` only — engine stats are no longer spread on the public route. Full stats moved to a new admin endpoint `/admin/api/full-stats` (sibling to `/admin/api/stats` and `/admin/api/health-indicators`) gated by the existing `requireAdmin` middleware; that route calls `probeHealth(engine, ...)` and returns the original spread-stats body. `?full=true` query param removed entirely. Closes the original DoS surface where `getStats()`'s 6× count(*) on 96K-page brains through PgBouncer exceeded `HEALTH_TIMEOUT_MS` and triggered orchestrator restart cascades (Fly.io / k8s seeing 503 → restart loop → advisory-lock pile-up on the migration lock). Outside-voice review (Codex) caught that `/admin/api/health-indicators` is NOT a full-stats endpoint (returns only `{expiring_soon, error_rate}`), and that an alternative loopback-IP gate would have depended on `app.set('trust proxy', 'loopback')` semantics holding under proxy/XFF misconfiguration; the shipped admin-cookie design avoids both. **v0.31.3 (#681):** every OAuth/admin/audit SQL call routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` so `gbrain serve --http` works against PGLite brains. The four `mcp_request_log.params` INSERT sites (success path, auth_failed path, scope_denied path, server-error path) all go through `executeRawJsonb(engine, ...)` so the JSONB column stores real objects, not JSON-encoded strings — closes the bug where `params->>'op'` returned the encoded string `"search"` (with quotes) instead of `search`. Migration v46 normalizes any pre-v0.31.3 string-shaped backlog rows on first start. +- `src/commands/serve-http.ts` (v0.26.0) — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. Started via `gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--log-full-params]`. Supersedes the v0.22.7 `src/mcp/http-transport.ts` simple bearer-auth path. Combines MCP SDK's `mcpAuthRouter` (authorize / token / register / revoke endpoints), a custom `client_credentials` handler (SDK's token endpoint throws `UnsupportedGrantTypeError` for CC; the custom handler runs BEFORE the router and falls through for `auth_code` / `refresh_token`), `requireBearerAuth` middleware for `/mcp` with scope enforcement before op dispatch, `localOnly` rejection, and `express-rate-limit` at 50 req / 15 min on `/token`. Serves the built admin SPA from `admin/dist/` with SPA fallback. `/admin/events` SSE endpoint broadcasts every MCP request to connected admin browsers. `cookie-parser` middleware wired (Express 5 has no built-in). Startup logging prints port, engine, configured issuer URL (honors `--public-url`), registered-client count, DCR status, and admin bootstrap token. **v0.26.9 hardening pass:** F7 sets `remote: true` explicitly on the `/mcp` request handler's OperationContext literal (closes the HTTP shell-job RCE — without this, `submit_job`'s protected-name guard at `operations.ts:1391` saw a falsy undefined and skipped, letting a `read+write`-scoped OAuth token submit `shell` jobs). F8 wires `summarizeMcpParams` from `src/mcp/dispatch.ts` into both `mcp_request_log` writes and the admin SSE feed by default (raw payloads opt-in via `--log-full-params` with stderr warning). F9 sets cookie `Secure` flag when behind HTTPS or a public-URL proxy. F10 caps the magic-link nonce store with an LRU bound. F12 routes DCR disable through the `GBrainOAuthProvider` constructor's `dcrDisabled` option instead of the prior monkey-patch on the express router. F14 wraps `transport.handleRequest` in try/catch so SDK throws return a JSON-RPC 500 envelope instead of express's default HTML error page. F15 unifies OperationError + unexpected exceptions through `buildError` / `serializeError` so `/mcp` always returns the same envelope shape. **v0.28.1:** `/health` endpoint extracted into pure `probeHealth(engine)` async function with `HEALTH_TIMEOUT_MS = 3000` exported constant — drops the timeout from 5s to 3s so Fly.io's 5s health-check deadline gets 2s of headroom for TCP, response framing, and clock skew. Races `engine.getStats()` against the timeout via `Promise.race`; saturated pool returns 503 with `Health check timed out (database pool may be saturated)` instead of hanging. `clearTimeout` in finally block prevents pending-timer pile-up under high probe rates (race-leak fix from adversarial review). **v0.28.10:** `/health` is now liveness-only via the new `probeLiveness(sql, engineName, version, timeoutMs)` helper that races `sql\`SELECT 1\`` against `HEALTH_TIMEOUT_MS` and returns the same `ProbeHealthResult` tagged-union as `probeHealth` (single timer-cleanup site, single 503 envelope). Body shape: `{status, version, engine}` only — engine stats are no longer spread on the public route. Full stats moved to a new admin endpoint `/admin/api/full-stats` (sibling to `/admin/api/stats` and `/admin/api/health-indicators`) gated by the existing `requireAdmin` middleware; that route calls `probeHealth(engine, ...)` and returns the original spread-stats body. `?full=true` query param removed entirely. Closes the original DoS surface where `getStats()`'s 6× count(*) on 96K-page brains through PgBouncer exceeded `HEALTH_TIMEOUT_MS` and triggered orchestrator restart cascades (Fly.io / k8s seeing 503 → restart loop → advisory-lock pile-up on the migration lock). Outside-voice review (Codex) caught that `/admin/api/health-indicators` is NOT a full-stats endpoint (returns only `{expiring_soon, error_rate}`), and that an alternative loopback-IP gate would have depended on `app.set('trust proxy', 'loopback')` semantics holding under proxy/XFF misconfiguration; the shipped admin-cookie design avoids both. **v0.31.3 (#681):** every OAuth/admin/audit SQL call routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` so `gbrain serve --http` works against PGLite brains. The four `mcp_request_log.params` INSERT sites (success path, auth_failed path, scope_denied path, server-error path) all go through `executeRawJsonb(engine, ...)` so the JSONB column stores real objects, not JSON-encoded strings — closes the bug where `params->>'op'` returned the encoded string `"search"` (with quotes) instead of `search`. Migration v46 normalizes any pre-v0.31.3 string-shaped backlog rows on first start. **v0.34.1.0 (#864):** new `--bind HOST` CLI flag with default `127.0.0.1`. Personal-laptop installs no longer publish the brain to the LAN by accident. Self-hosted operators pass `--bind 0.0.0.0` (or a specific interface IP) once to accept remote connections. A stderr WARN fires when `--public-url` is set without `--bind` so the operator sees the binding before the first request (common cause of "ngrok forwards to me but the agent can't reach the upstream" misconfigurations). The startup banner prints a `Bind:` line. **v0.34.1.0 (#861):** drops the `(authInfo as AuthInfo & {sourceId?: string}).sourceId ?? env ?? 'default'` cast chain — `AuthInfo.sourceId` and `AuthInfo.allowedSources` are now the typed source of truth, populated by `oauth-provider.ts:verifyAccessToken` from the `oauth_clients` row. - `src/core/sql-query.ts` (v0.31.3) — Engine-aware tagged-template SQL adapter for OAuth/admin/auth infrastructure. `sqlQueryForEngine(engine)` returns a `SqlQuery` (`(strings, ...values) => Promise`) that walks the template, builds `$N` positional SQL, asserts every value is a `SqlValue` (string | number | bigint | boolean | Date | null), and routes through `engine.executeRaw(sql, params)` so Postgres goes via postgres.js's `unsafe(sql, params)` path and PGLite via its embedded `db.query(sql, params)`. Deliberately narrower than postgres.js's `sql` tag: no nested fragments, no `sql.json()`, no `sql.unsafe()`, no `sql.begin()`, no array binding. The narrow surface is the feature — codex finding #7 from the v0.31 plan review argued the adapter should stay scalar-only or it drifts into a partial postgres.js clone. JSONB writes go through the separate `executeRawJsonb(engine, sql, scalarParams, jsonbParams)` helper that composes positional `$N::jsonb` casts and passes JS objects through; the v0.12.0 double-encode bug class doesn't apply because positional binding through `unsafe()` reaches the wire protocol with the correct type oid (verified by `test/sql-query.test.ts` on PGLite and `test/e2e/auth-permissions.test.ts:67` on Postgres). `scripts/check-jsonb-pattern.sh` doesn't fire because `executeRawJsonb(...)` is a method call, not the banned literal-template-tag interpolation pattern. Consumed by `src/commands/auth.ts`, `src/commands/serve-http.ts`, `src/core/oauth-provider.ts`, `src/commands/files.ts`, and `src/mcp/http-transport.ts` so all five sites work against PGLite and Postgres uniformly. Closes the bug where `gbrain auth` + `gbrain serve --http` were silently Postgres-only because they routed every SQL through the postgres.js singleton (community PR #681). - `src/commands/serve.ts` (v0.31.3) — `gbrain serve` stdio MCP entrypoint with idempotent shutdown across every parent-disconnect signal. Stdio EOF, SIGTERM, SIGINT, SIGHUP, and parent-process death (every reparent case — PID 1, launchd subreaper, systemd, tmux, or a parent shell with `PR_SET_CHILD_SUBREAPER`) all funnel into one `cleanup(reason)` path that releases the engine and the PGLite write-lock dir within 5 seconds. Pre-v0.31.3 the stdio MCP server held the lock indefinitely after Claude Desktop / Cursor / launchd-managed gateways disconnected, forcing a 5-minute stale-lock wait on the next start. Watchdog reparent check is `getParentPid() !== initialParentPid` (capturing the initial ppid once at install time and firing on any change); the previous `=== 1` check missed the subreaper case under launchd / systemd. Bun's `process.ppid` cache is stale across reparenting (see [oven-sh/bun#30305](https://github.com/oven-sh/bun/issues/30305)) so `getParentPid()` runs `spawnSync('ps', ['-o', 'ppid=', '-p', PID])` per tick to read the live kernel PPID. Startup probe verifies `ps` is on PATH; if not (stripped containers, busybox without procps), the watchdog skips installing AND emits a loud `[gbrain serve] watchdog disabled: ps unavailable, parent-death detection unavailable — child will rely on stdin EOF / signals only` stderr line so operators see the degraded mode at boot. Pinned by `test/serve-stdio-lifecycle.test.ts` (22 cases). Closes #413, #446. Credit @Aragorn2046 (origin features in #591) and @seungsu-kr (rebased submitter, Bun ppid workaround). -- `src/core/oauth-provider.ts` (v0.26.0) — `GBrainOAuthProvider` implementing the MCP SDK's `OAuthServerProvider` + `OAuthRegisteredClientsStore` interfaces. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1 spec: `authorize` + `exchangeAuthorizationCode` with PKCE (for ChatGPT), `client_credentials` (for Perplexity / Claude), `refresh_token` with rotation, `revokeToken`, `registerClient` (DCR path validates redirect_uri must be `https://` or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Auth codes single-use with 10-minute TTL via atomic `DELETE...RETURNING` (closes RFC 6749 §10.5 TOCTOU race). Refresh rotation also `DELETE...RETURNING` (closes §10.4 stolen-token detection bypass). `pgArray()` escapes commas/quotes/braces in elements so a comma-bearing redirect_uri can't smuggle a second array element. Legacy `access_tokens` fallback in `verifyAccessToken` grandfathers pre-v0.26 bearer tokens as `read+write+admin`. `sweepExpiredTokens()` runs on startup wrapped in try/catch. **v0.26.9 RFC 6749/7009 hardening pass:** F1+F2 fold `client_id` atomically into the `DELETE WHERE` clauses for both auth-code exchange and refresh rotation — pre-fix the post-hoc client compare burned the row on wrong-client paths so the legitimate client couldn't retry. F3 enforces refresh-scope-subset against the original grant on the row (RFC 6749 §6), not the client's currently-allowed scopes — fixes the case where revoking a scope from a client wouldn't shrink the agent's existing refresh tokens. F4 binds `client_id` on `revokeToken` so a client can only revoke its own tokens (RFC 7009 §2.1). F7c validates the `/token` request's `redirect_uri` against the value stored at `/authorize` (RFC 6749 §4.1.3) — empty-string treated as missing rather than wildcard match (adversarial-review fix). F5 swaps bare `catch {}` blocks in `verifyAccessToken` and `getClient` for `isUndefinedColumnError` from `src/core/utils.ts` — only SQLSTATE 42703 falls through to legacy fallback; lock timeouts and network blips throw and surface. F6 makes `sweepExpiredTokens()` actually return the count via `RETURNING 1` + array length, not a fire-and-forget zero. F12 adds `dcrDisabled` constructor option so `serve-http.ts` can disable the `/register` endpoint without monkey-patching the router. **v0.26.2:** module-private `coerceTimestamp()` boundary helper at the top of the file normalizes postgres-driver-as-string BIGINT columns to JS numbers at every read site (5 call sites: `getClient` L112+L113 for DCR `/register` RFC 7591 §3.2.1 numeric timestamps, `exchangeRefreshToken` L274 + `verifyAccessToken` L296+L303 for the SDK's `typeof === 'number'` bearerAuth check). Throws on non-finite input (NaN/Infinity) so corrupt rows fail loud at the boundary instead of riding through as `expiresAt: NaN`; returns undefined for SQL NULL so callers decide NULL semantics explicitly (refresh + access token paths treat NULL as expired). Helper intentionally NOT promoted to `src/core/utils.ts` — codex review flagged repo-wide BIGINT precision-loss risk for a generic helper. +- `src/core/oauth-provider.ts` (v0.26.0) — `GBrainOAuthProvider` implementing the MCP SDK's `OAuthServerProvider` + `OAuthRegisteredClientsStore` interfaces. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1 spec: `authorize` + `exchangeAuthorizationCode` with PKCE (for ChatGPT), `client_credentials` (for Perplexity / Claude), `refresh_token` with rotation, `revokeToken`, `registerClient` (DCR path validates redirect_uri must be `https://` or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Auth codes single-use with 10-minute TTL via atomic `DELETE...RETURNING` (closes RFC 6749 §10.5 TOCTOU race). Refresh rotation also `DELETE...RETURNING` (closes §10.4 stolen-token detection bypass). `pgArray()` escapes commas/quotes/braces in elements so a comma-bearing redirect_uri can't smuggle a second array element. Legacy `access_tokens` fallback in `verifyAccessToken` grandfathers pre-v0.26 bearer tokens as `read+write+admin`. `sweepExpiredTokens()` runs on startup wrapped in try/catch. **v0.26.9 RFC 6749/7009 hardening pass:** F1+F2 fold `client_id` atomically into the `DELETE WHERE` clauses for both auth-code exchange and refresh rotation — pre-fix the post-hoc client compare burned the row on wrong-client paths so the legitimate client couldn't retry. F3 enforces refresh-scope-subset against the original grant on the row (RFC 6749 §6), not the client's currently-allowed scopes — fixes the case where revoking a scope from a client wouldn't shrink the agent's existing refresh tokens. F4 binds `client_id` on `revokeToken` so a client can only revoke its own tokens (RFC 7009 §2.1). F7c validates the `/token` request's `redirect_uri` against the value stored at `/authorize` (RFC 6749 §4.1.3) — empty-string treated as missing rather than wildcard match (adversarial-review fix). F5 swaps bare `catch {}` blocks in `verifyAccessToken` and `getClient` for `isUndefinedColumnError` from `src/core/utils.ts` — only SQLSTATE 42703 falls through to legacy fallback; lock timeouts and network blips throw and surface. F6 makes `sweepExpiredTokens()` actually return the count via `RETURNING 1` + array length, not a fire-and-forget zero. F12 adds `dcrDisabled` constructor option so `serve-http.ts` can disable the `/register` endpoint without monkey-patching the router. **v0.26.2:** module-private `coerceTimestamp()` boundary helper at the top of the file normalizes postgres-driver-as-string BIGINT columns to JS numbers at every read site (5 call sites: `getClient` L112+L113 for DCR `/register` RFC 7591 §3.2.1 numeric timestamps, `exchangeRefreshToken` L274 + `verifyAccessToken` L296+L303 for the SDK's `typeof === 'number'` bearerAuth check). Throws on non-finite input (NaN/Infinity) so corrupt rows fail loud at the boundary instead of riding through as `expiresAt: NaN`; returns undefined for SQL NULL so callers decide NULL semantics explicitly (refresh + access token paths treat NULL as expired). Helper intentionally NOT promoted to `src/core/utils.ts` — codex review flagged repo-wide BIGINT precision-loss risk for a generic helper. **v0.34.1.0 (#909):** `registerClient` honors `token_endpoint_auth_method: "none"` (RFC 7591 §3.2.1) — public PKCE clients (Claude Code, Cursor, every other PKCE-first MCP client) store `client_secret_hash = NULL` and the response payload omits `client_secret` entirely. Confidential clients (default `client_secret_post` and explicit `client_secret_basic`) keep their one-time-reveal shape. `getClient` correctly normalizes a NULL `client_secret_hash` to JS `undefined` so the SDK's clientAuth path accepts the public client at `/token`. **v0.34.1.0 (#861 + #876):** `verifyAccessToken` JOINs `oauth_clients.source_id` (write scope, scalar) + `oauth_clients.federated_read` (read scope, TEXT[]) and surfaces both on the returned `AuthInfo`. Pre-v60 / pre-v61 brains degrade gracefully via `isUndefinedColumnError` fallback so the upgrade chain is non-blocking on legacy DBs. - `admin/` (v0.26.0) — React 19 + Vite + TypeScript admin SPA embedded in the binary via `admin/dist/` served by `serve-http.ts`. 7 screens: Login (bootstrap token → session cookie), Dashboard (metrics + SSE feed + token health), Agents (sortable table + sparklines + Register button), Register (modal with scope checkboxes + grant type selector), Credentials reveal (full-screen modal with Copy + Download JSON + yellow one-time-only warning), Request Log (filterable paginated), Agent Detail drawer (Details / Activity / Config Export tabs + Revoke). Design tokens: `#0a0a0f` bg, Inter for UI, JetBrains Mono for data, 4-32px spacing scale, rounded pill badges. HTTP-only SameSite=Strict cookie auth. 65KB gzip. Build: `cd admin && bun install && bun run build`; output at `admin/dist/` is committed for self-contained binaries. -- `src/commands/auth.ts` — Token management. `gbrain auth create/list/revoke/test` for legacy bearer tokens (v0.22.7 wired as a first-class CLI subcommand) plus `gbrain auth register-client` (v0.26.0) and `gbrain auth revoke-client ` (v0.26.2) for OAuth 2.1 client lifecycle. `revoke-client` runs an atomic `DELETE...RETURNING` on `oauth_clients`; FK `ON DELETE CASCADE` on `oauth_tokens.client_id` and `oauth_codes.client_id` purges every active token + authorization code in a single transaction. `process.exit(1)` on no-such-client (idempotent — re-running on the same id produces the same exit-1 message). Legacy tokens stored as SHA-256 hashes in `access_tokens`; OAuth clients in `oauth_clients`. As of v0.26.0, legacy tokens grandfather to `read+write+admin` scopes on the OAuth HTTP server, so pre-v0.26 deployments keep working with no migration. **v0.31.3 (#681):** every SQL site routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` (and `executeRawJsonb` for the takes-holders `permissions` JSONB column) so `gbrain auth` works against PGLite brains. Pre-fix, every call hit the postgres.js singleton via `getConn()` and silently failed (or wrote to the wrong DB) when the active engine was PGLite. The takes-holders write goes through `executeRawJsonb(engine, sql, [name, hash], [{takes_holders:[...]}])` which round-trips with `jsonb_typeof = 'object'` instead of the pre-v0.31.3 quoted-string shape. +- `src/commands/auth.ts` — Token management. `gbrain auth create/list/revoke/test` for legacy bearer tokens (v0.22.7 wired as a first-class CLI subcommand) plus `gbrain auth register-client` (v0.26.0) and `gbrain auth revoke-client ` (v0.26.2) for OAuth 2.1 client lifecycle. `revoke-client` runs an atomic `DELETE...RETURNING` on `oauth_clients`; FK `ON DELETE CASCADE` on `oauth_tokens.client_id` and `oauth_codes.client_id` purges every active token + authorization code in a single transaction. `process.exit(1)` on no-such-client (idempotent — re-running on the same id produces the same exit-1 message). Legacy tokens stored as SHA-256 hashes in `access_tokens`; OAuth clients in `oauth_clients`. As of v0.26.0, legacy tokens grandfather to `read+write+admin` scopes on the OAuth HTTP server, so pre-v0.26 deployments keep working with no migration. **v0.31.3 (#681):** every SQL site routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` (and `executeRawJsonb` for the takes-holders `permissions` JSONB column) so `gbrain auth` works against PGLite brains. Pre-fix, every call hit the postgres.js singleton via `getConn()` and silently failed (or wrote to the wrong DB) when the active engine was PGLite. The takes-holders write goes through `executeRawJsonb(engine, sql, [name, hash], [{takes_holders:[...]}])` which round-trips with `jsonb_typeof = 'object'` instead of the pre-v0.31.3 quoted-string shape. **v0.34.1.0 (#876):** `register-client` accepts `--source ` (write authority, scalar) and `--federated-read ` (read scope, array). The output prints the resolved `Write source` and `Federated reads` for the registered client. Pre-v0.34 clients backfill to `source_id='default'` via migration v60 so existing deployments keep their v0.33 effective behavior verbatim. - `src/commands/upgrade.ts` — Self-update CLI. `runPostUpgrade()` enumerates migrations from the TS registry (src/commands/migrations/index.ts) and tail-calls `runApplyMigrations(['--yes', '--non-interactive'])` so the mechanical side of every outstanding migration runs unconditionally. - `src/commands/migrations/` — TS migration registry (compiled into the binary; no filesystem walk of `skills/migrations/*.md` needed at runtime). `index.ts` lists migrations in semver order. `v0_11_0.ts` = Minions adoption orchestrator (8 phases). `v0_12_0.ts` = Knowledge Graph auto-wire orchestrator (5 phases: schema → config check → backfill links → backfill timeline → verify). `phaseASchema` has a 600s timeout (bumped from 60s in v0.12.1 for duplicate-heavy brains). `v0_12_2.ts` = JSONB double-encode repair orchestrator (4 phases: schema → repair-jsonb → verify → record). `v0_14_0.ts` = shell-jobs + autopilot cooperative (2 phases: schema ALTER minion_jobs.max_stalled SET DEFAULT 3 — superseded by v0.14.3's schema-level DEFAULT 5 + UPDATE backfill; pending-host-work ping for skills/migrations/v0.14.0.md). All orchestrators are idempotent and resumable from `partial` status. As of v0.14.2 (Bug 3), the RUNNER owns all ledger writes — orchestrators return `OrchestratorResult` and `apply-migrations.ts` persists a canonical `{version, status, phases}` shape after return. Orchestrators no longer call `appendCompletedMigration` directly. `statusForVersion` prefers `complete` over `partial` (never regresses). 3 consecutive partials → wedged → `--force-retry ` writes a `'retry'` reset marker. v0.14.3 (fix wave) ships schema-only migrations v14 (`pages_updated_at_index`) + v15 (`minion_jobs_max_stalled_default_5` with UPDATE backfill) via the `MIGRATIONS` array in `src/core/migrate.ts` — no orchestrator phases needed. - `src/commands/repair-jsonb.ts` — `gbrain repair-jsonb [--dry-run] [--json]`: rewrites `jsonb_typeof='string'` rows in place across 5 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter). Fixes v0.12.0 double-encode bug on Postgres; PGLite no-ops. Idempotent. @@ -282,7 +282,7 @@ strict behavior when unset. - `src/commands/transcripts.ts` (v0.29) — `gbrain transcripts recent [--days N] [--full] [--json]`: recent raw `.txt` transcripts from the dream-cycle corpus dirs. Imports `listRecentTranscripts` from `src/core/transcripts.ts` (the same library the gated `get_recent_transcripts` MCP op uses). Local-only by construction — the CLI always runs with `ctx.remote=false`. - `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s). Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable. **v0.32.8 (PR #860):** batch projection switched from `SELECT DISTINCT ON (slug)` to `SELECT ... ORDER BY source_id, slug` so multi-source brains scan each `(source, slug)` row independently (pre-fix the DISTINCT collapsed same-slug-different-source pages into one scan, the same bug class this PR fixes). Sequential and auto-repair loops use `listAllPageRefs()` to enumerate `(slug, source_id)` pairs and thread `sourceId` to `getPage`. Batch + sequential paths now report the same page count on multi-source brains. - `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count. v0.26.7 (#612): `rls_event_trigger` check (post-install drift detector for migration v35's auto-RLS event trigger). Lives outside the `// 5. RLS` slice that the structural doctor.test.ts guards anchor on, so the existing test guards stay intact. Healthy `evtenabled` set is `('O','A')` only — `R` is replica-only and would not fire in normal sessions; `D` is disabled. Fix hint is `gbrain apply-migrations --force-retry 35`. **v0.30.2:** `queue_health` gains a fourth subcheck — surfaces dead-lettered subagent jobs with `last_error` matching the `prompt_too_long` classifier within the last 24h. Fix hint points at `gbrain dream --phase synthesize --dry-run --json` to identify the offending transcript and `gbrain jobs prune --status dead --queue default` to clean up. Postgres-only. **v0.31.7:** `runDoctor` switches to `autoDetectSkillsDirReadOnly` (from `src/core/repo-root.ts`) so `bun install -g github:garrytan/gbrain && cd ~ && gbrain doctor` finds the bundled `skills/` via the install-path fallback instead of warning "Could not find skills directory" + docking the health score. `--fix` carries a D6 safety gate: when `detected.source === 'install_path'`, the command refuses auto-repair with a stderr message pointing at `$GBRAIN_SKILLS_DIR` / `$OPENCLAW_WORKSPACE` / `--skills-dir`, because `autoFixDryViolations` writes to SKILL.md files and would otherwise silently rewrite the install tree. The `graph_coverage` check now short-circuits to `ok: 'No entity pages — graph_coverage not applicable (markdown-only brain)'` when `SELECT COUNT(*) FROM pages WHERE type IN ('entity','person','company','organization')` returns 0 (closes #530); the entity count is woven into the warn message and the WARN hint switches from the long-deprecated `gbrain link-extract && gbrain timeline-extract` (gone since v0.16) to the canonical `gbrain extract all`. Pinned by an IRON-RULE regression assertion in `test/doctor.test.ts` that bans the stale verb names from the source string. **v0.32.4:** new `sync_freshness` check (exported `checkSyncFreshness` at the same file) added to both `runDoctor` (local) and `doctorReportRemote` (thin-client). Pure staleness probe — queries `sources.last_sync_at` only, no filesystem access. Warns at 24h, fails at 72h (or never-synced). Future-`last_sync_at` warns ("clock skew or corrupted timestamp") instead of silently falling through as ok — codex outside-voice caught the negative-ageMs bug pre-merge. Env-var overrides `GBRAIN_SYNC_FRESHNESS_WARN_HOURS` / `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS`; invalid values fall back to defaults with a once-per-process stderr warn (`_resolveSyncFreshnessHours`). Failure messages embed `source.id` (not `source.name`) so the printed fix command `gbrain sync --source ` matches what the user copy-pastes. Filesystem-vs-DB page drift detection was deliberately stripped from the v0.32.4 scope — `doctorReportRemote` runs in the HTTP MCP server (`src/commands/serve-http.ts`), and walking DB-supplied `local_path` from a remote-callable endpoint crosses a trust boundary (OAuth write scope could mutate `sources.local_path`). Drift detection will resurface in a separate PR routed through `multi_source_drift`'s existing guard infrastructure (`GBRAIN_DRIFT_LIMIT` / `GBRAIN_DRIFT_TIMEOUT_MS`) with slug normalization tests and a meta-file allow-list. Pinned by 12 cases in `test/doctor.test.ts` ("v0.32.4 — sync_freshness check" describe block): empty sources, never-synced fail, >72h fail, exact 72h boundary, 24h-72h warn, exact 24h boundary, <24h ok, future-timestamp warn, mixed sources (highest severity wins), `executeRaw` throws → outer-catch warn, `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS=6` override fires at 7h, source.id-in-message regression. -- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). **v40 (v0.29):** `pages_emotional_weight` adds `pages.emotional_weight REAL NOT NULL DEFAULT 0.0`. Column-only (no index). On Postgres 11+ and PGLite, `ADD COLUMN` with a constant DEFAULT is metadata-only — instant on tables of any size. v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs. **v35 (v0.26.7):** auto-RLS event trigger + one-time backfill. `auto_rls_on_create_table` fires on `ddl_command_end` for `WHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO')` and runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY` on every new `public.*` table — no FORCE (matches v24/v29/schema.sql posture so non-BYPASSRLS apps can still read their own tables). The same migration backfills RLS on every existing `public.*` base table whose comment doesn't match the doctor regex (`^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,}`). Per-table failure aborts the offending CREATE TABLE (event triggers fire inside the DDL transaction); no EXCEPTION wrap — that would convert loud rollback into silent permissive default. PGLite no-op via `sqlFor.pglite: ''`. Breaking change: operators with intentionally-RLS-off public tables must add the GBRAIN:RLS_EXEMPT comment BEFORE upgrade or the backfill will flip them on. **v46 (v0.31.3):** `mcp_request_log_params_jsonb_normalize` rewrites pre-v0.31.3 rows where `mcp_request_log.params` was stored as a JSON-encoded string (`jsonb_typeof = 'string'`) up to a real JSONB object via `UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string'`. Single statement, idempotent — second-run finds no string-shaped rows and is a no-op. Closes the bug where `/admin/api/requests` returned a quoted string instead of the parsed object. +- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). **v40 (v0.29):** `pages_emotional_weight` adds `pages.emotional_weight REAL NOT NULL DEFAULT 0.0`. Column-only (no index). On Postgres 11+ and PGLite, `ADD COLUMN` with a constant DEFAULT is metadata-only — instant on tables of any size. v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs. **v35 (v0.26.7):** auto-RLS event trigger + one-time backfill. `auto_rls_on_create_table` fires on `ddl_command_end` for `WHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO')` and runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY` on every new `public.*` table — no FORCE (matches v24/v29/schema.sql posture so non-BYPASSRLS apps can still read their own tables). The same migration backfills RLS on every existing `public.*` base table whose comment doesn't match the doctor regex (`^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,}`). Per-table failure aborts the offending CREATE TABLE (event triggers fire inside the DDL transaction); no EXCEPTION wrap — that would convert loud rollback into silent permissive default. PGLite no-op via `sqlFor.pglite: ''`. Breaking change: operators with intentionally-RLS-off public tables must add the GBRAIN:RLS_EXEMPT comment BEFORE upgrade or the backfill will flip them on. **v46 (v0.31.3):** `mcp_request_log_params_jsonb_normalize` rewrites pre-v0.31.3 rows where `mcp_request_log.params` was stored as a JSON-encoded string (`jsonb_typeof = 'string'`) up to a real JSONB object via `UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string'`. Single statement, idempotent — second-run finds no string-shaped rows and is a no-op. Closes the bug where `/admin/api/requests` returned a quoted string instead of the parsed object. **v0.34.1.0 (#861 + #876, v60-v65):** six-migration chain wires source-scoping into the OAuth client table. v60 (`oauth_clients_source_id_fk`) adds `oauth_clients.source_id TEXT` with NULL→`'default'` backfill and an FK to `sources(id) ON DELETE SET NULL`. v61 (`oauth_clients_federated_read_column`) adds `federated_read TEXT[] NOT NULL DEFAULT '{}'`. v62 (`oauth_clients_federated_read_backfill`) explicit-CASE backfills so `source_id IS NULL` produces `'{}'` not an array-containing-NULL. v63 (`oauth_clients_federated_read_validate`) is the fail-loud check that every row's source_id is in its federated_read array post-backfill. v64 (`oauth_clients_source_id_fk_restrict`) flips the FK to `ON DELETE RESTRICT` now that federated_read provides the alternative scope-loss path — source delete is refused if any client references it. v65 (`oauth_clients_federated_read_gin_index`) is the GIN index for the array-containment queries the read paths run. PGLite parity via `sqlFor.pglite` where needed. - `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2. - `src/core/cli-options.ts` — Global CLI flag parser. `parseGlobalFlags(argv)` returns `{cliOpts, rest}` with `--quiet` / `--progress-json` / `--progress-interval=` stripped. `getCliOptions()` / `setCliOptions()` expose a module-level singleton so commands reach the resolved flags without parameter threading. `cliOptsToProgressOptions()` maps to reporter options. `childGlobalFlags()` returns the flag suffix to append to `execSync('gbrain ...')` calls in migration orchestrators. `OperationContext.cliOpts` extends shared-op dispatch for MCP callers. - `src/core/db-lock.ts` (v0.22.13) — generic `tryAcquireDbLock(engine, lockId, ttlMinutes)` over the existing `gbrain_cycle_locks` table. Parameterized lock id so different scopes can nest cleanly: `gbrain-cycle` for the broad cycle (held by `cycle.ts`) and `gbrain-sync` (`SYNC_LOCK_ID` constant) for `performSync`'s narrower writer window. Same UPSERT-with-TTL semantics as the prior cycle-only helper, just generalized. Survives PgBouncer transaction pooling (unlike session-scoped `pg_try_advisory_lock`); crashed holders auto-release once their TTL expires. @@ -757,6 +757,10 @@ E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_U - `test/e2e/serve-http-oauth.test.ts` (v0.26.0, expanded v0.26.2, expanded v0.26.9) — real-Postgres E2E against `gbrain serve --http` with full OAuth 2.1. Spawns a subprocess server, registers a client via the CLI, mints `client_credentials` tokens, exercises the `/mcp` JSON-RPC pipeline. **v0.26.2 adds:** real DCR `/register` HTTP-level response-shape test (asserts `typeof body.client_id_issued_at === 'number'` over the wire — RFC 7591 §3.2.1 spec compliance, not just internal-store shape); real CLI subprocess test for `revoke-client` (registers → mints token → revokes via `execSync` → asserts token rejected at `/mcp` → asserts re-run exits 1); server fixture flips on `--enable-dcr` so `/register` is reachable. **bun execSync env-inheritance fix:** bun's `execSync` does NOT inherit env mutations done via `process.env.X = ...`, only OS-level env from before bun started. helpers.ts loads `.env.testing` and sets `DATABASE_URL` via `process.env` mutation, which is invisible to subprocesses unless `env: { ...process.env }` is passed explicitly — every subprocess call in this file passes `env: { ...process.env }` for that reason. Reference fix for the next maintainer hitting the same failure mode in sibling sync/cycle/dream/claw-test E2Es. `afterAll` cleanup is guarded on `clientId` (won't throw if `beforeAll` failed before registration); cleanup errors surface to stderr without throwing so real test failures aren't masked. Tracks DCR-registered clients alongside the manual one. **v0.26.9** adds 2 regressions for the F7 trust-boundary fix: an HTTP MCP `submit_job` for `name: "shell"` MUST reject with a permission error (proving the request handler now sets `remote: true` and `submit_job`'s protected-name guard fires), and the same guard rejects subagent submission. Closes the OAuth-token-to-RCE escalation path. Skips gracefully when `DATABASE_URL` is unset. - `test/e2e/sync-parallel.test.ts` (v0.22.13 PR #490) — DATABASE_URL-gated. T2: 60-file Postgres sync at concurrency=4 imports all + no connection leak (probes `pg_stat_activity` before/after to confirm worker engines disconnected). P4: 120-file serial-vs-parallel benchmark prints `SYNC_PARALLEL_BENCH N files | serial=Xms | parallel(4)=Yms | speedup=Zx` for CHANGELOG quoting. Asserts parallel ≤ serial × 1.5 (CI-noise tolerant; not a strict speedup gate). - `test/e2e/multi-source-bug-class.test.ts` (v0.32.8, PR #860) — 7-case PGLite in-memory regression suite pinning every bug site fixed in this PR: `listAllPageRefs` ordering by `(source_id, slug)` (F11), `getPage` with sourceId picks the right `(source, slug)` row (F2), `extract-takes` processes both overlapping `people/alice` rows independently, `listPages` filters correctly with `PageFilters.sourceId`, `addLinksBatch` with `from/to_source_id` targets the right rows (F4), `validateSourceId` rejects path traversal (F6), reverse-write disk layout uses `brainDir/.sources//.md` for non-default sources (F6). No DATABASE_URL needed. Wired into `scripts/e2e-test-map.ts` so changes to extract-takes / patterns / synthesize / embed / extract / migrate-engine auto-trigger this test. Companion: `test/e2e/integrity-batch.test.ts`'s "multi-source duplicate slugs scan once" case was pinning the pre-fix bug — assertion flipped in v0.32.8 to expect both batch + sequential paths report 2. +- `test/e2e/source-isolation-pglite.test.ts` (v0.34.1.0, #861) — 14-case PGLite in-memory regression suite pinning the source-isolation P0 seal at two layers. Engine layer: `searchKeyword` / `searchVector` / `searchKeywordChunks` / `listPages` / `getPage` / `traverseGraph` / `traversePaths` apply `sourceId` (scalar fast path) and `sourceIds` (array path) correctly across both engines. Op-handler layer: routes through `sourceScopeOpts(ctx)` so a `read+write`-scoped OAuth client bound to `--source dept-x` cannot see rows from neighboring sources via `search`, `query`, `list_pages`, `get_page`, or `find_experts`. Covers both `ctx.sourceId` (single-source clients) and `ctx.auth.allowedSources` (federated_read clients) precedence; federated array wins over scalar wins over nothing. No DATABASE_URL needed. +- `test/openai-compat-multimodal.test.ts` (v0.34.1.0, #875) — 11-case unit suite for the gateway's openai-compatible multimodal path: happy-path single + multi-input embedding, unauthenticated proxy mode, dimension-mismatch guard (D12; throws `AIConfigError` with model id + observed + expected pre-storage), default-dim fallback when recipe declares `default_dims`, HTTP 401 / 400 / malformed-JSON / non-array error paths, plus a regression test that the existing Voyage `/multimodalembeddings` recipe still routes through its dedicated path (not the openai-compatible one). Hermetic via the `__setEmbedTransportForTests` seam. +- `test/serve-stdio-lifecycle.test.ts` (extended v0.34.1.0, #870) — adds 3 new cases for the `MCP_STDIO=1` env guard: stdin EOF does NOT trigger shutdown when the env is set, SIGTERM still does (guard scope is correct), unset env preserves the pre-v0.34 CLI lifecycle. Exercises the `ServeOptions.mcpStdio?: boolean` test seam directly so tests don't mutate `process.env`. +- `test/oauth.test.ts` (extended v0.34.1.0, #909) — 5 new cases for the PKCE DCR public-client gate: `registerClient` with `token_endpoint_auth_method: "none"` returns no `client_secret` field on the public client, default `client_secret_post` clients still get the one-time-reveal secret, `getClient` NULL→undefined normalization so the SDK's clientAuth path accepts public clients, full PKCE `/authorize` → `/token` round-trip against a public client (no client_secret presented), and a regression test that the public-vs-confidential branch doesn't break confidential client `client_secret_post` exchange. - Tier 2 (`skills.test.ts`) requires OpenClaw + API keys, runs nightly in CI - If `.env.testing` doesn't exist in this directory, check sibling worktrees for one: `find ../ -maxdepth 2 -name .env.testing -print -quit` and copy it here if found. @@ -2210,7 +2214,9 @@ pick scopes, save the credentials shown once in the reveal modal. Programmatic registration via `oauthProvider.registerClientManual(...)` and the `gbrain auth register-client` CLI are also available. -- **OAuth 2.1 via the MCP SDK** — client credentials (machine-to-machine: Perplexity, Claude), authorization code + PKCE (browser-based: ChatGPT), refresh token rotation, revocation, protected resource metadata. Optional Dynamic Client Registration behind `--enable-dcr` (DCR redirect_uris must be `https://` or loopback per RFC 6749 §3.1.2.1). +- **OAuth 2.1 via the MCP SDK** — client credentials (machine-to-machine: Perplexity, Claude), authorization code + PKCE (browser-based: ChatGPT), refresh token rotation, revocation, protected resource metadata. PKCE-only public clients (`token_endpoint_auth_method: "none"`) register without a secret per RFC 7591 §3.2.1 (v0.34). Optional Dynamic Client Registration behind `--enable-dcr` (DCR redirect_uris must be `https://` or loopback per RFC 6749 §3.1.2.1). +- **Source-scoped OAuth clients (v0.34)** — `gbrain auth register-client my-agent --source dept-x` ties the client's write authority to one source; read paths only return rows matching that source. `--federated-read S1,S2,S3` adds an orthogonal read-scope axis for shared brains (departments writing to one canon while reading the union). Pre-v0.34 clients are backfilled to `source_id='default'` on upgrade. +- **Loopback default for `serve --http` (v0.34)** — listens on `127.0.0.1` unless `--bind 0.0.0.0` (or a specific interface IP). Personal-laptop installs no longer publish the brain to the LAN by accident. A stderr WARN fires when `--public-url` is set without `--bind` so the operator sees the binding before the first request. - **Scoped operations** — 30 operations tagged `read | write | admin`. `sync_brain` and `file_upload` are `localOnly`, rejected over HTTP. - **React admin dashboard** — 7 screens baked into the binary (~65KB gzip). Live SSE activity feed, agents table, credential reveal, filterable request log, per-client config export. - **Legacy bearer tokens still work** — pre-v0.26 `gbrain auth create` tokens continue to authenticate as `read+write+admin`. v0.22.7's simpler `src/mcp/http-transport.ts` path stays compiled in for backward compat callers; v0.26+ deployments use the OAuth-aware `serve-http.ts`. @@ -2859,12 +2865,16 @@ ADMIN [--skip=] [--json] gbrain serve MCP server (stdio) gbrain serve --http [--port 3131] HTTP MCP server with OAuth 2.1 + admin dashboard + [--bind HOST] (v0.34: default 127.0.0.1; pass + --bind 0.0.0.0 for LAN/remote access) [--token-ttl 3600] [--enable-dcr] [--public-url URL] [--log-full-params] gbrain auth create|list|revoke|test Legacy bearer token management gbrain auth register-client Register an OAuth 2.1 client --grant-types client_credentials,authorization_code --scopes "read write admin" + --source v0.34: write authority for source-scoped clients + --federated-read v0.34: read scope across multiple sources gbrain auth revoke-client Revoke an OAuth 2.1 client (cascade purges active tokens + auth codes via FK CASCADE) # OAuth 2.1 clients can also be registered from the /admin dashboard or @@ -5192,6 +5202,24 @@ gbrain auth register-client perplexity \ --scopes "read write" ``` +**v0.34 — source-scoped clients.** Multi-source brains can scope a client's +write authority to one source and its read scope to a curated set with the +new `--source` and `--federated-read` flags: + +```bash +gbrain auth register-client dept-x-agent \ + --grant-types client_credentials \ + --scopes "read write" \ + --source dept-x \ + --federated-read dept-x,shared,parent-canon +``` + +`--source` controls the write authority — `put_page` / `add_link` / etc only +land in `dept-x`. `--federated-read` controls the read axis independently; +queries return rows from any of the listed sources. Omit both flags for the +v0.33-compatible super-client shape. Pre-v0.34 clients are backfilled to +`source_id='default'` on `gbrain upgrade`. + Host-repo wrappers can register programmatically: ```ts @@ -5208,6 +5236,18 @@ start the server with `--enable-dcr`. DCR is off by default. ### 3. Expose the server +**v0.34 — bind explicitly.** `gbrain serve --http` defaults to `127.0.0.1`. +To accept connections from the ngrok tunnel (or any non-loopback source), +restart with `--bind`: + +```bash +gbrain serve --http --port 3131 --bind 0.0.0.0 --public-url https://your-brain.ngrok.app +``` + +When `--public-url` is set without `--bind`, a stderr WARN fires at +startup so the misconfiguration ("the tunnel is up but my agent gets +ECONNREFUSED") is loud. + ```bash brew install ngrok ngrok config add-authtoken YOUR_TOKEN diff --git a/package.json b/package.json index 5e7a7f5..5491452 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gbrain", - "version": "0.34.0.0", + "version": "0.34.1.0", "description": "Postgres-native personal knowledge brain with hybrid RAG search", "type": "module", "main": "src/core/index.ts", diff --git a/src/commands/auth.ts b/src/commands/auth.ts index 4aaca2c..f9ae0cb 100644 --- a/src/commands/auth.ts +++ b/src/commands/auth.ts @@ -329,26 +329,44 @@ async function revokeClient(clientId: string) { } async function registerClient(name: string, args: string[]) { - if (!name) { console.error('Usage: auth register-client [--grant-types G] [--scopes S]'); process.exit(1); } + if (!name) { + console.error('Usage: auth register-client [--grant-types G] [--scopes S] [--source SOURCE] [--federated-read SRC1,SRC2,...]'); + process.exit(1); + } const grantsIdx = args.indexOf('--grant-types'); const scopesIdx = args.indexOf('--scopes'); + const sourceIdx = args.indexOf('--source'); + const federatedIdx = args.indexOf('--federated-read'); const grantTypes = grantsIdx >= 0 && args[grantsIdx + 1] ? args[grantsIdx + 1].split(',').map(s => s.trim()).filter(Boolean) : ['client_credentials']; const scopes = scopesIdx >= 0 && args[scopesIdx + 1] ? args[scopesIdx + 1] : 'read'; + // v0.34.1 (#861, D2): --source flag scopes the OAuth client to a single + // source. Defaults to 'default' to match migration v60's backfill so + // operators upgrading without changing flags see no behavior change. + const sourceId = sourceIdx >= 0 && args[sourceIdx + 1] ? args[sourceIdx + 1] : 'default'; + // v0.34.1 (#876): --federated-read accepts a comma-separated source list + // for federated read scope. When omitted, federated_read defaults to + // [sourceId] (read scope == write scope, the v0.33 default). + const federatedRead = federatedIdx >= 0 && args[federatedIdx + 1] + ? args[federatedIdx + 1].split(',').map(s => s.trim()).filter(Boolean) + : undefined; try { await withConfiguredSql(async (sql) => { const { GBrainOAuthProvider } = await import('../core/oauth-provider.ts'); const provider = new GBrainOAuthProvider({ sql }); const { clientId, clientSecret } = await provider.registerClientManual( - name, grantTypes, scopes, [], + name, grantTypes, scopes, [], sourceId, federatedRead, ); + const effectiveFederated = federatedRead && federatedRead.length > 0 ? federatedRead : [sourceId]; console.log(`OAuth client registered: "${name}"\n`); - console.log(` Client ID: ${clientId}`); - console.log(` Client Secret: ${clientSecret}\n`); - console.log(` Grant types: ${grantTypes.join(', ')}`); - console.log(` Scopes: ${scopes}\n`); + console.log(` Client ID: ${clientId}`); + console.log(` Client Secret: ${clientSecret}\n`); + console.log(` Grant types: ${grantTypes.join(', ')}`); + console.log(` Scopes: ${scopes}`); + console.log(` Write source: ${sourceId}`); + console.log(` Federated reads: ${effectiveFederated.join(', ')}\n`); console.log('Save the client secret — it will not be shown again.'); console.log(`Revoke with: gbrain auth revoke-client "${clientId}"`); }); diff --git a/src/commands/serve-http.ts b/src/commands/serve-http.ts index b8da946..752324b 100644 --- a/src/commands/serve-http.ts +++ b/src/commands/serve-http.ts @@ -162,10 +162,31 @@ interface ServeHttpOptions { * at startup so the privacy posture change is visible. */ logFullParams?: boolean; + /** + * Network interface(s) to bind. Defaults to `127.0.0.1` (loopback only) in + * v0.34.1+ — gbrain's primary use case is a personal-knowledge brain on a + * laptop, and the pre-v0.34 default of `0.0.0.0` made it one accidental + * `--http` invocation away from publishing the brain to a LAN. + * + * Server operators who DO want to accept remote connections pass + * `--bind 0.0.0.0` (or a specific interface IP). When `--public-url` is + * set but `--bind` is unset, a stderr WARN fires at startup recommending + * the explicit flag — defaulting to loopback while declaring a public URL + * is almost always a misconfiguration. + */ + bind?: string; } export async function runServeHttp(engine: BrainEngine, options: ServeHttpOptions) { const { port, tokenTtl, enableDcr, publicUrl, logFullParams } = options; + // v0.34.1 (#864, D11): default bind flipped from 0.0.0.0 to 127.0.0.1. + // gbrain's primary use case is a personal-knowledge brain on a laptop; + // the pre-v0.34 default exposed brains on every interface. Server + // operators who need remote access pass `--bind 0.0.0.0` (or a specific + // interface). Declaring `--public-url` without `--bind` is almost always + // a misconfiguration; we WARN to stderr at startup in that case rather + // than silently binding loopback only. + const bind = options.bind ?? '127.0.0.1'; const config = loadConfig() || { engine: 'pglite' as const }; if (logFullParams) { @@ -174,6 +195,12 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption ); } + if (publicUrl && options.bind === undefined) { + console.error( + '[serve-http] WARNING: --public-url is set but --bind is not. Default bind changed to 127.0.0.1 in v0.34.1; remote clients reaching the public URL will be refused. Pass --bind 0.0.0.0 to accept all interfaces.', + ); + } + // Engine-aware SQL adapter. Routes through engine.executeRaw on both // Postgres and PGLite — the OAuth/admin/auth surface no longer requires // a postgres.js singleton, so `gbrain serve --http` works against PGLite @@ -926,9 +953,14 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption // ToolResult and we read isError + _meta to pick the right branch. const tokenAllowList = (authInfo as AuthInfo & { takesHoldersAllowList?: string[] }).takesHoldersAllowList ?? ['world']; - const tokenSourceId = (authInfo as AuthInfo & { sourceId?: string }).sourceId - ?? process.env.GBRAIN_SOURCE - ?? 'default'; + // v0.34.1 (#861, D13): AuthInfo.sourceId is now a real typed field + // populated from oauth_clients.source_id (migration v60 backfilled + // NULL → 'default'). Pre-fix this site cast through AuthInfo and + // fell back to GBRAIN_SOURCE env / 'default' — the silent-fallback + // path codex flagged in plan review. Post-v60, every OAuth client + // has source_id set; legacy bearer tokens default to 'default' in + // verifyAccessToken. The env-fallback is gone. + const tokenSourceId = authInfo.sourceId ?? 'default'; let toolResult: Awaited>; try { @@ -1058,12 +1090,13 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption // --------------------------------------------------------------------------- const clientCount = await sql`SELECT count(*)::int as count FROM oauth_clients`; - app.listen(port, () => { + app.listen(port, bind, () => { console.error(` ╔══════════════════════════════════════════════════════╗ ║ GBrain MCP Server v${VERSION.padEnd(37)}║ ╠══════════════════════════════════════════════════════╣ ║ Port: ${String(port).padEnd(40)}║ +║ Bind: ${bind.padEnd(40)}║ ║ Engine: ${(config.engine || 'pglite').padEnd(40)}║ ║ Issuer: ${issuerUrl.origin.padEnd(40)}║ ║ Clients: ${String((clientCount[0] as any).count).padEnd(40)}║ diff --git a/src/commands/serve.ts b/src/commands/serve.ts index 1b1651a..3fa1362 100644 --- a/src/commands/serve.ts +++ b/src/commands/serve.ts @@ -56,6 +56,13 @@ export interface ServeOptions { // tick fell through to the cached `process.ppid` and the watchdog // never fired, while still claiming to be installed. probeWatchdog?: () => boolean; + // v0.34.1 (#870): test seam for the MCP_STDIO=1 piped-stdin guard. + // When true, runServe skips the stdin 'end'/'close' shutdown hooks + // because the wrapping gateway (OpenClaw bundle-mcp, others) pipes the + // JSON-RPC handshake and closes stdin immediately. Signal handlers and + // transport.onclose still cover legitimate shutdown. + // Defaults to `process.env.MCP_STDIO === '1'` when omitted. + mcpStdio?: boolean; } export async function runServe( @@ -90,8 +97,18 @@ export async function runServe( // when set so the posture change is visible in stderr. const logFullParams = args.includes('--log-full-params'); + // v0.34.1 (#864, D11): `--bind HOST` lets operators choose the network + // interface to listen on. When unset, runServeHttp defaults to 127.0.0.1 + // (loopback) — server operators who need remote access pass + // `--bind 0.0.0.0` (or a specific interface IP). `bind` is intentionally + // left undefined here when the flag is absent so the WARN-on-public-url + // path in serve-http can distinguish "operator chose loopback explicitly" + // from "operator didn't set the flag at all." + const bindIdx = args.indexOf('--bind'); + const bind = bindIdx >= 0 ? args[bindIdx + 1] : undefined; + const { runServeHttp } = await import('./serve-http.ts'); - await runServeHttp(engine, { port, tokenTtl, enableDcr, publicUrl, logFullParams }); + await runServeHttp(engine, { port, tokenTtl, enableDcr, publicUrl, logFullParams, bind }); return; } @@ -201,7 +218,15 @@ function installStdioLifecycle( // Skip when stdin is a TTY: interactive `gbrain serve` use shouldn't // terminate just because the user hasn't typed anything. Signal / // watchdog paths still cover that case if needed. - if (!deps.stdin.isTTY) { + // v0.34.1 (#870): when MCP_STDIO=1, the wrapping gateway pipes the + // JSON-RPC handshake then closes its stdin half. Treating that as a + // permanent disconnect kills the server before the first tool call. + // Signal handlers (SIGTERM/SIGINT/SIGHUP), transport.onclose, and the + // parent-process watchdog below still cover legitimate shutdown paths. + // `mcpStdio` is the injectable form; default reads the env once at + // install time so tests stay isolated (no process.env mutation). + const mcpStdioMode = opts.mcpStdio ?? (process.env.MCP_STDIO === '1'); + if (!deps.stdin.isTTY && !mcpStdioMode) { deps.stdin.once('end', () => beginShutdown('stdin-end')); deps.stdin.once('close', () => beginShutdown('stdin-close')); } diff --git a/src/commands/whoknows.ts b/src/commands/whoknows.ts index a422568..8460057 100644 --- a/src/commands/whoknows.ts +++ b/src/commands/whoknows.ts @@ -52,6 +52,21 @@ export interface WhoknowsOpts { * ranking function without redefining the type filter. */ types?: PageType[]; + /** + * v0.34.1 (#861, D3 — P0 leak seal): scope expert candidates to a + * single source. The op-handler at operations.ts:find_experts threads + * `ctx.sourceId` here so an authenticated MCP client scoped to src-A + * cannot surface people pages from src-B in the rankings. Pre-fix, the + * whoknows op was authored against v0.33 after PR #861 was drafted and + * the source-scope thread was missing entirely. + */ + sourceId?: string; + /** + * v0.34.1 (#876, D9): federated read — scope candidates to ANY of these + * source ids. Threaded from `ctx.auth?.allowedSources` via + * `sourceScopeOpts` in operations.ts. Array wins over scalar `sourceId`. + */ + sourceIds?: string[]; } export interface WhoknowsResult { @@ -165,11 +180,15 @@ export async function findExperts( // 1. Hybrid search with SQL-level types filter (v0.33 typeFilter parameter). // Disable salience + recency boosts in hybridSearch — we apply our own // locked formula on top of the raw relevance score. + // v0.34.1 (#861, D3): thread source-scope so an authenticated MCP + // client only ranks experts within its accessible sources. const results: SearchResult[] = await hybridSearch(engine, opts.topic, { types, limit: innerLimit, salience: 'off', recency: 'off', + sourceId: opts.sourceId, + sourceIds: opts.sourceIds, }); if (results.length === 0) return []; diff --git a/src/core/ai/gateway.ts b/src/core/ai/gateway.ts index f6deb2e..eef0ee2 100644 --- a/src/core/ai/gateway.ts +++ b/src/core/ai/gateway.ts @@ -1016,11 +1016,18 @@ export async function embedMultimodal(inputs: MultimodalInput[]): Promise { + // Auth resolution via the gateway's canonical helper so LiteLLM-style + // optional-auth recipes (Authorization: Bearer LITELLM_API_KEY) and + // hard-required-auth recipes (OpenAI Authorization: Bearer + // OPENAI_API_KEY) both work via the same code path. Throws AIConfigError + // when required env is missing. + const authResult = recipe.resolveAuth + ? recipe.resolveAuth(cfg.env) + : defaultResolveAuth(recipe, cfg.env, 'embedding'); + const baseUrl = cfg.base_urls?.[recipe.id] ?? recipe.base_url_default; + if (!baseUrl) { + throw new AIConfigError( + `${recipe.name} requires a base URL for multimodal embedding.`, + recipe.setup_hint, + ); + } + + // D12 — dim validation. Prefer recipe's declared default_dims when set; + // fall back to the brain's configured embedding_dimensions. If neither + // is known (LiteLLM recipe with default_dims=0 and no config override), + // we skip the dim check rather than fabricate an expected value — the + // engine's vector(N) column will reject mismatched rows at INSERT time + // with a clearer error than anything we could throw here. + const recipeDims = recipe.touchpoints.embedding?.default_dims ?? 0; + const expectedDims = recipeDims > 0 + ? recipeDims + : (cfg.embedding_dimensions ?? 0); + + // Send each input as one /embeddings request. Most providers cap the + // number of inputs per call at the text-embedding batch limit, but the + // multimodal content array varies per provider. Single-input requests + // are the safe lowest common denominator; LiteLLM's proxy backend + // batches internally if it can. + const allEmbeddings: Float32Array[] = []; + for (const input of inputs) { + const body = { + model: modelId, + input: [ + { + // OpenAI's documented multimodal content shape. The data-URL + // form embeds the image bytes inline so the proxy doesn't need + // network access to fetch the image. + type: 'image_url', + image_url: { url: `data:${input.mime};base64,${input.data}` }, + }, + ], + }; + + let res: Response; + try { + res = await fetch(`${baseUrl}/embeddings`, { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + [authResult.headerName]: authResult.token, + }, + body: JSON.stringify(body), + }); + } catch (err) { + throw normalizeAIError(err, `embedMultimodal(${recipe.id}:${modelId})`); + } + + if (!res.ok) { + const text = await res.text().catch(() => ''); + if (res.status === 401 || res.status === 403) { + const requiredKey = recipe.auth_env?.required[0]; + throw new AIConfigError( + `${recipe.name} multimodal returned ${res.status}: ${text || 'auth failed'}.`, + requiredKey + ? `Re-export ${requiredKey} or rotate the key at ${recipe.auth_env?.setup_url ?? recipe.setup_hint}.` + : recipe.setup_hint, + ); + } + // Surface the upstream error verbatim — 400s here usually mean the + // proxied model doesn't support multimodal input. The error text is + // the user's best signal for picking a different model id. + throw new AITransientError( + `${recipe.name} multimodal returned ${res.status}: ${text || 'transient error'}.`, + ); + } + + let parsedBody: { data?: Array<{ embedding: number[] }> }; + try { + parsedBody = (await res.json()) as { data?: Array<{ embedding: number[] }> }; + } catch (err) { + throw new AITransientError( + `${recipe.name} multimodal returned malformed JSON: ${err instanceof Error ? err.message : String(err)}.`, + ); + } + if (!parsedBody.data || !Array.isArray(parsedBody.data) || parsedBody.data.length < 1) { + throw new AITransientError( + `${recipe.name} multimodal returned no embeddings (expected 1).`, + ); + } + + const row = parsedBody.data[0]; + if (!Array.isArray(row.embedding)) { + throw new AITransientError( + `${recipe.name} multimodal returned non-array embedding payload.`, + ); + } + // D12 — dim validation. Throw EmbedDimensionMismatchError-shape error + // (AIConfigError with model id + observed + expected so the operator + // can diagnose and pick a compatible model OR adjust the brain's + // embedding_dimensions config). Skip the check when expectedDims=0 + // (no recipe declaration AND no config override). + if (expectedDims > 0 && row.embedding.length !== expectedDims) { + throw new AIConfigError( + `${recipe.id}:${modelId} returned ${row.embedding.length}-dim vector; expected ${expectedDims}.`, + `The brain's embedding column is fixed at ${expectedDims} dims; this model is incompatible. ` + + `Either pick a model that returns ${expectedDims} dims, OR set --embedding-dimensions ${row.embedding.length} ` + + `and reinitialize the embedding column at the new width.`, + ); + } + allEmbeddings.push(new Float32Array(row.embedding)); + } + + return allEmbeddings; +} + // ---- Expansion ---- async function resolveExpansionProvider(modelStr: string): Promise<{ model: any; recipe: Recipe; modelId: string }> { diff --git a/src/core/ai/recipes/litellm-proxy.ts b/src/core/ai/recipes/litellm-proxy.ts index d7ee4e9..bc82b63 100644 --- a/src/core/ai/recipes/litellm-proxy.ts +++ b/src/core/ai/recipes/litellm-proxy.ts @@ -30,6 +30,15 @@ export const litellmProxy: Recipe = { // LiteLLM's batch capacity is determined by the backend it proxies; // no static cap to declare here. v0.32 (#779). no_batch_cap: true, + // v0.34.1 (#875): LiteLLM can forward to multimodal providers (OpenAI, + // Gemini, Voyage etc.). embedMultimodal routes openai-compatible + // recipes through embedMultimodalOpenAICompat() — same /embeddings + // endpoint as text, with content arrays carrying image_base64 + // entries. No multimodal_models allow-list: the user knows which of + // their proxied models support multimodal; we trust the model id and + // surface the provider's rejection (D12 dim-validation catches + // mismatched-dim responses pre-storage). + supports_multimodal: true, }, }, setup_hint: 'Run LiteLLM (https://docs.litellm.ai) in front of any provider; set LITELLM_BASE_URL + pass --embedding-model litellm: and --embedding-dimensions .', diff --git a/src/core/engine.ts b/src/core/engine.ts index 0f64a08..24b4703 100644 --- a/src/core/engine.ts +++ b/src/core/engine.ts @@ -629,18 +629,31 @@ export interface BrainEngine { dirPrefix?: string, minSimilarity?: number, ): Promise<{ slug: string; similarity: number } | null>; - traverseGraph(slug: string, depth?: number): Promise; + /** + * v0.34.1 (#861 — P0 leak seal): `opts.sourceId` / `opts.sourceIds` + * constrain visited nodes to a single source or array of sources. + * Pre-fix, the walk ignored source scope and an authenticated MCP + * client could enumerate cross-source topology + page metadata via + * the graph op. MCP-bound callers MUST pass the auth'd scope; local + * CLI callers omit it for the historical unscoped behavior. + */ + traverseGraph( + slug: string, + depth?: number, + opts?: { sourceId?: string; sourceIds?: string[] }, + ): Promise; /** * Edge-based graph traversal with optional type and direction filters. * Returns a list of edges (GraphPath[]) instead of nodes. Supports: * - linkType: per-edge filter, only follows matching edges (per-edge semantics) * - direction: 'in' (follow to->from), 'out' (follow from->to), 'both' * - depth: max depth from root (default 5) + * - sourceId/sourceIds: v0.34.1 source-isolation filter, see traverseGraph * Uses cycle prevention (visited array in recursive CTE). */ traversePaths( slug: string, - opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both' }, + opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] }, ): Promise; /** * For a list of slugs, return how many inbound links each has. diff --git a/src/core/migrate.ts b/src/core/migrate.ts index 54fe792..9077237 100644 --- a/src/core/migrate.ts +++ b/src/core/migrate.ts @@ -2969,6 +2969,187 @@ export const MIGRATIONS: Migration[] = [ ON search_telemetry (date DESC); `, }, + { + version: 60, + name: 'oauth_clients_source_id_fk', + // v0.34.1 (#861 + D4 + D10 + D13 — P0 source-isolation leak seal). + // + // Adds oauth_clients.source_id, validates ALL existing rows can map to a + // real source row, backfills NULL → 'default', and installs the FK with + // ON DELETE SET NULL. PR #861's original migration claimed v47-v51; we + // re-number to v60 because the branch already shipped through v54. + // + // D10 (codex outside-voice push-back): fail loud when stale source_id + // rows exist instead of silently widening to NULL. Pre-fix this column + // didn't exist; the only way a row has source_id IS a manual SQL poke, + // so the stale-row branch fires only on operator-modified brains. The + // GBRAIN_ACCEPT_SILENT_WIDEN=1 env var is the explicit opt-in for + // operators who'd rather upgrade than psql-fix. Doctor surfaces orphan + // rows post-clean via the v0.34.x follow-up TODO. + // + // D13: backfill NULL → 'default' BEFORE the FK ADD preserves the v0.33 + // effective behavior (legacy unscoped clients silently fell back to + // 'default' via serve-http.ts:929 cast). Verify 'default' exists in + // sources first — fresh brains have it from sources schema's default + // seed; brains that scripted it out would otherwise wedge here. + // + // PGLite parity via the same DO blocks. PGLite supports DO/EXCEPTION + // since 0.3; no engine branch needed. + idempotent: true, + sql: ` + -- v0.34.1 (#861 + D2 + D13 — P0 source-isolation leak seal). + -- + -- This migration is intentionally lean: oauth_clients.source_id did + -- NOT exist pre-v60, so the only state we inherit from upgrade is + -- "rows with NULL source_id." Backfill those to 'default' (D13: + -- preserves the pre-v0.34 effective fallback behavior verbatim) and + -- install the FK with ON DELETE SET NULL. + -- + -- D10 pre-clean is NOT NEEDED here: codex flagged the silent-widen + -- footgun assuming source_id was an existing column with possibly-stale + -- values. Since the column is brand new in this migration, the only + -- post-backfill values are 'default' (which we just verified exists + -- via the FK contract) plus any NULL the backfill left untouched + -- because of WHERE-clause filtering — none possible. The + -- GBRAIN_ACCEPT_SILENT_WIDEN env-flag stays in the runner for future + -- migrations that need it; this one doesn't. + + -- 1. Add the column. NULL for every existing row. + ALTER TABLE oauth_clients ADD COLUMN IF NOT EXISTS source_id TEXT; + + -- 2. Backfill NULL → 'default'. Pre-v0.34 legacy clients then map + -- to the same source the serve-http fallback chain used to put + -- them in implicitly. No-op on fresh installs (no rows yet). + UPDATE oauth_clients SET source_id = 'default' WHERE source_id IS NULL; + + -- 3. Install FK if not already present. The PGLite + Postgres fresh- + -- install schemas (src/core/pglite-schema.ts, src/schema.sql) now + -- include the FK inline on the CREATE TABLE, so this DO block + -- skips on fresh installs and only fires on upgrade brains where + -- oauth_clients was created pre-v60 without the FK. ON DELETE SET + -- NULL matches the original PR #861 posture; #876 later flips to + -- RESTRICT once federated_read provides the alternative + -- scope-recovery path. + DO $$ + BEGIN + IF NOT EXISTS ( + SELECT 1 FROM pg_constraint + WHERE conname = 'oauth_clients_source_id_fkey' + ) THEN + ALTER TABLE oauth_clients + ADD CONSTRAINT oauth_clients_source_id_fkey + FOREIGN KEY (source_id) REFERENCES sources(id) ON DELETE SET NULL; + END IF; + END $$; + + -- 4. Index for token-verification lookups (verifyAccessToken's JOIN + -- on oauth_clients.client_id → c.source_id). oauth_clients stays + -- small so plain CREATE INDEX (no CONCURRENTLY) is fine. + CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id + ON oauth_clients(source_id) WHERE source_id IS NOT NULL; + `, + }, + { + version: 61, + name: 'oauth_clients_federated_read_column', + // v0.34.1 (#876): add federated_read TEXT[] for the read-side + // federation feature. source_id (v60) is the WRITE-authority axis; + // federated_read is the READ-scope axis. A client can write to ONE + // source while reading from N (a "WeCare L3 dept" client writes to + // dept-x and reads dept-x + parent canon + shared canon). + // + // Default '{}' (empty array) on column add — pre-existing rows get + // backfilled in v62 with an explicit CASE so the array reflects the + // client's current scope rather than the column default. + idempotent: true, + sql: ` + ALTER TABLE oauth_clients ADD COLUMN IF NOT EXISTS federated_read TEXT[] NOT NULL DEFAULT '{}'; + `, + }, + { + version: 62, + name: 'oauth_clients_federated_read_backfill', + // v0.34.1 (#876, F5 — codex outside-voice fix). Backfill federated_read + // with explicit CASE so source_id IS NULL doesn't produce an ambiguous + // array containing NULL. Three cases: + // - source_id IS NULL → '{}' (empty read scope; legacy unscoped + // clients lost their implicit fallback in v60 backfill to 'default', + // so this branch fires only when an operator explicitly NULL'd + // source_id after migration). + // - source_id IS NOT NULL → ARRAY[source_id] (read scope matches + // write scope, the pre-federation default). + // Only fires on rows where federated_read is still the column default + // ({}). Operators who hand-set federated_read keep their config. + idempotent: true, + sql: ` + UPDATE oauth_clients + SET federated_read = CASE + WHEN source_id IS NULL THEN '{}'::text[] + ELSE ARRAY[source_id] + END + WHERE federated_read = '{}'::text[]; + `, + }, + { + version: 63, + name: 'oauth_clients_federated_read_validate', + // v0.34.1 (#876): post-backfill validation. Every client with a + // non-NULL source_id should now have its source_id reflected in + // federated_read. Fail loud if backfill missed a row — points at a + // logic bug in v62's WHERE clause. + idempotent: true, + sql: ` + DO $$ + DECLARE + bad_count INT; + BEGIN + SELECT count(*) INTO bad_count FROM oauth_clients + WHERE source_id IS NOT NULL + AND NOT (source_id = ANY(federated_read)); + IF bad_count > 0 THEN + RAISE EXCEPTION 'oauth_clients has % rows where source_id is not in federated_read after v62 backfill. This is a bug in v62 — re-run gbrain apply-migrations --force-retry 62.', bad_count; + END IF; + END $$; + `, + }, + { + version: 64, + name: 'oauth_clients_source_id_fk_restrict', + // v0.34.1 (#876): flip the source_id FK from ON DELETE SET NULL (v60 + // posture) to ON DELETE RESTRICT now that federated_read provides + // the alternative scope-loss path. Pre-fix, deleting a source could + // silently widen any oauth_client to super-reader (source_id → NULL). + // Post-flip, source delete is refused if any client references it; + // the operator's path is "revoke or re-scope the clients first." + idempotent: true, + sql: ` + DO $$ + BEGIN + IF EXISTS ( + SELECT 1 FROM pg_constraint + WHERE conname = 'oauth_clients_source_id_fkey' + ) THEN + ALTER TABLE oauth_clients DROP CONSTRAINT oauth_clients_source_id_fkey; + END IF; + ALTER TABLE oauth_clients + ADD CONSTRAINT oauth_clients_source_id_fkey + FOREIGN KEY (source_id) REFERENCES sources(id) ON DELETE RESTRICT; + END $$; + `, + }, + { + version: 65, + name: 'oauth_clients_federated_read_gin_index', + // v0.34.1 (#876): GIN index for array-containment lookups + // (`WHERE p.source_id = ANY(federated_read)` and similar). The five + // read-side ops fall back to scalar sourceId when no auth is set, so + // this index only matters under load on federated-scoped clients. + idempotent: true, + sql: ` + CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read + ON oauth_clients USING GIN (federated_read); + `, + }, ]; export const LATEST_VERSION = MIGRATIONS.length > 0 diff --git a/src/core/oauth-provider.ts b/src/core/oauth-provider.ts index c31188d..501fad2 100644 --- a/src/core/oauth-provider.ts +++ b/src/core/oauth-provider.ts @@ -139,9 +139,15 @@ class GBrainClientsStore implements OAuthRegisteredClientsStore { `; if (rows.length === 0) return undefined; const r = rows[0]; + // v0.34.1 (#909): public clients (token_endpoint_auth_method='none') + // have client_secret_hash = NULL. Normalize SQL NULL to JS undefined + // so SDK middleware that checks `client.client_secret === undefined` + // (not `=== null`) correctly identifies the client as public and + // skips the secret-comparison branch on /token. + const rawSecret = r.client_secret_hash; return { client_id: r.client_id as string, - client_secret: r.client_secret_hash as string | undefined, + client_secret: rawSecret == null ? undefined : (rawSecret as string), client_name: r.client_name as string, redirect_uris: (r.redirect_uris as string[]) || [], grant_types: (r.grant_types as string[]) || ['client_credentials'], @@ -170,27 +176,97 @@ class GBrainClientsStore implements OAuthRegisteredClientsStore { assertAllowedScopes(parseScopeString(client.scope)); const clientId = generateToken('gbrain_cl_'); - const clientSecret = generateToken('gbrain_cs_'); - const secretHash = hashToken(clientSecret); + // v0.34.1 (#909): RFC 7591 §2 — clients that authenticate at the token + // endpoint via PKCE alone declare `token_endpoint_auth_method: "none"`. + // For those clients the authorization server MUST NOT issue a client + // secret. Pre-fix, unconditional secret generation made the MCP SDK's + // clientAuth middleware check `client.client_secret` on every request, + // rejecting valid public-client (Claude Code, Cursor) flows. + // + // We persist secret_hash = NULL for public clients so `getClient` and + // the SDK's clientAuth path can detect them via `client_secret_hash IS + // NULL` and skip the secret comparison. Confidential clients (default + // `client_secret_post` and explicit `client_secret_basic`) still mint + // a secret as before. + const authMethod = client.token_endpoint_auth_method || 'client_secret_post'; + const isPublicClient = authMethod === 'none'; + const clientSecret = isPublicClient ? undefined : generateToken('gbrain_cs_'); + const secretHash = clientSecret ? hashToken(clientSecret) : null; const now = Math.floor(Date.now() / 1000); - await this.sql` - INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, - grant_types, scope, token_endpoint_auth_method, - client_id_issued_at) - VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'}, - ${pgArray((client.redirect_uris || []).map(String))}, - ${pgArray(client.grant_types || ['client_credentials'])}, - ${client.scope || ''}, ${client.token_endpoint_auth_method || 'client_secret_post'}, - ${now}) - `; + // v0.34.1 (#861, D2 + D13 + #876): DCR clients get source_id='default' + // (matches legacy fallback) and federated_read=['default'] (read scope + // == write scope). Operators who need narrower / wider scope rescope + // via the CLI later. Pre-v60/v61 brain falls through to the legacy + // projection (no source_id / federated_read column yet). + try { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, token_endpoint_auth_method, + client_id_issued_at, source_id, federated_read) + VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'}, + ${pgArray((client.redirect_uris || []).map(String))}, + ${pgArray(client.grant_types || ['client_credentials'])}, + ${client.scope || ''}, ${authMethod}, + ${now}, ${'default'}, ${pgArray(['default'])}) + `; + } catch (err) { + if (isUndefinedColumnError(err, 'federated_read')) { + try { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, token_endpoint_auth_method, + client_id_issued_at, source_id) + VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'}, + ${pgArray((client.redirect_uris || []).map(String))}, + ${pgArray(client.grant_types || ['client_credentials'])}, + ${client.scope || ''}, ${authMethod}, + ${now}, ${'default'}) + `; + } catch (err2) { + if (isUndefinedColumnError(err2, 'source_id')) { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, token_endpoint_auth_method, + client_id_issued_at) + VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'}, + ${pgArray((client.redirect_uris || []).map(String))}, + ${pgArray(client.grant_types || ['client_credentials'])}, + ${client.scope || ''}, ${authMethod}, + ${now}) + `; + } else { + throw err2; + } + } + } else if (isUndefinedColumnError(err, 'source_id')) { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, token_endpoint_auth_method, + client_id_issued_at) + VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'}, + ${pgArray((client.redirect_uris || []).map(String))}, + ${pgArray(client.grant_types || ['client_credentials'])}, + ${client.scope || ''}, ${authMethod}, + ${now}) + `; + } else { + throw err; + } + } - return { + // Public clients: omit `client_secret` entirely from the response so + // the wire payload matches RFC 7591 §3.2.1 ("if the client is a + // public client, the authorization server MUST NOT issue a client + // secret"). Confidential clients return the freshly-generated secret + // exactly once — same shape as before. + const response: OAuthClientInformationFull = { ...client, client_id: clientId, - client_secret: clientSecret, client_id_issued_at: now, }; + if (clientSecret) response.client_secret = clientSecret; + return response; } } @@ -403,15 +479,57 @@ export class GBrainOAuthProvider implements OAuthServerProvider { const now = Math.floor(Date.now() / 1000); // Try OAuth tokens first. JOIN oauth_clients in the same query so - // verifyAccessToken returns client_name in AuthInfo — eliminates the - // separate per-request lookup at serve-http.ts that was the N+1 hot - // path (see PR #586 review D14=B). - const oauthRows = await this.sql` - SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name - FROM oauth_tokens t - LEFT JOIN oauth_clients c ON c.client_id = t.client_id - WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' - `; + // verifyAccessToken returns client_name AND source_id in AuthInfo — + // eliminates the separate per-request lookup at serve-http.ts that + // was the N+1 hot path (see PR #586 review D14=B; v0.34.1 #861 D2 + // adds the source_id thread on the same JOIN). + // + // v0.34.1 (#861): the JOIN guards on a c.source_id column that + // migration v60 adds. Pre-v60 brains throw a "column does not exist" + // error here — caught at the boundary via isUndefinedColumnError so + // unmigrated brains degrade to "no source scope" rather than refusing + // every token verification. + let oauthRows: Record[]; + try { + oauthRows = await this.sql` + SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name, + c.source_id, c.federated_read + FROM oauth_tokens t + LEFT JOIN oauth_clients c ON c.client_id = t.client_id + WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' + `; + } catch (err) { + // v0.34.1: pre-v60 brain → source_id column missing. Pre-v61 brain → + // federated_read column missing. Both classes degrade to legacy + // projection so auth keeps working until the operator runs + // apply-migrations. Probe both column names so partial-upgrade brains + // (v60 applied but v61 didn't yet) also fall through cleanly. + if (isUndefinedColumnError(err, 'source_id') || isUndefinedColumnError(err, 'federated_read')) { + // Try the v60-only projection first (source_id but no federated_read). + try { + oauthRows = await this.sql` + SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name, c.source_id + FROM oauth_tokens t + LEFT JOIN oauth_clients c ON c.client_id = t.client_id + WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' + `; + } catch (err2) { + if (isUndefinedColumnError(err2, 'source_id')) { + // Truly pre-v60: no source_id either. Pre-v0.34 projection. + oauthRows = await this.sql` + SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name + FROM oauth_tokens t + LEFT JOIN oauth_clients c ON c.client_id = t.client_id + WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' + `; + } else { + throw err2; + } + } + } else { + throw err; + } + } if (oauthRows.length > 0) { const row = oauthRows[0]; @@ -422,6 +540,15 @@ export class GBrainOAuthProvider implements OAuthServerProvider { if (expiresAt === undefined || expiresAt < now) { throw new Error('Token expired'); } + // v0.34.1 (#876): federated_read normalization. SELECT returns + // either a JS array (Postgres / PGLite text[] driver mapping) or + // undefined when the legacy projection ran (pre-v61 brain). Empty + // array vs undefined matters: empty array = explicit no-federated- + // read; undefined = column missing on this brain. + const federatedRaw = row.federated_read; + const allowedSources = Array.isArray(federatedRaw) + ? (federatedRaw as string[]) + : undefined; return { token, clientId: row.client_id as string, @@ -429,6 +556,14 @@ export class GBrainOAuthProvider implements OAuthServerProvider { scopes: (row.scopes as string[]) || [], expiresAt, resource: row.resource ? new URL(row.resource as string) : undefined, + // v0.34.1 (#861, D2): source-isolation scope from oauth_clients. + // Undefined when the row predates v60 or when the brain itself + // predates v60 (fell through to the legacy projection above). + sourceId: (row.source_id as string | null) ?? undefined, + // v0.34.1 (#876): federated read scope. sourceScopeOpts in + // operations.ts prefers this array over scalar sourceId when set + // and non-empty. + allowedSources, } as AuthInfo; } @@ -452,6 +587,12 @@ export class GBrainOAuthProvider implements OAuthServerProvider { clientName: name, scopes: ['read', 'write', 'admin'], expiresAt: Math.floor(Date.now() / 1000) + 365 * 24 * 3600, // Legacy tokens never expire — set 1yr future + // v0.34.1 (#861, D13): legacy bearer tokens default to 'default' + // source — matches the pre-v0.34 effective behavior where the + // serve-http transport fell back to GBRAIN_SOURCE/'default' for + // any caller without explicit scope. Operators who want a + // narrower scope for legacy tokens migrate to OAuth. + sourceId: 'default', } as AuthInfo; } @@ -570,6 +711,8 @@ export class GBrainOAuthProvider implements OAuthServerProvider { grantTypes: string[], scopes: string, redirectUris: string[] = [], + sourceId: string = 'default', + federatedRead?: string[], ): Promise<{ clientId: string; clientSecret: string }> { // v0.28: ALLOWED_SCOPES allowlist. Reject `--scopes "read flying-unicorn"` // at registration so meaningless scope strings can't pile up in the DB. @@ -582,12 +725,56 @@ export class GBrainOAuthProvider implements OAuthServerProvider { const secretHash = hashToken(clientSecret); const now = Math.floor(Date.now() / 1000); - await this.sql` - INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, - grant_types, scope, client_id_issued_at) - VALUES (${clientId}, ${secretHash}, ${name}, - ${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now}) - `; + // v0.34.1 (#861 + #876): persist source_id AND federated_read so + // verifyAccessToken can populate both AuthInfo fields. Defaults: + // source_id = 'default' (matches v60 backfill) + // federated_read = [source_id] when omitted (a non-federated client + // has read scope == write scope, the v0.33 default) + const federated = federatedRead && federatedRead.length > 0 ? federatedRead : [sourceId]; + try { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, client_id_issued_at, + source_id, federated_read) + VALUES (${clientId}, ${secretHash}, ${name}, + ${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now}, + ${sourceId}, ${pgArray(federated)}) + `; + } catch (err) { + // Pre-v60 / pre-v61 brain: column missing. Fall back through both + // projections so registration still works until apply-migrations. + if (isUndefinedColumnError(err, 'federated_read')) { + // v60-only brain: source_id but no federated_read. + try { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, client_id_issued_at, source_id) + VALUES (${clientId}, ${secretHash}, ${name}, + ${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now}, ${sourceId}) + `; + } catch (err2) { + if (isUndefinedColumnError(err2, 'source_id')) { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, client_id_issued_at) + VALUES (${clientId}, ${secretHash}, ${name}, + ${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now}) + `; + } else { + throw err2; + } + } + } else if (isUndefinedColumnError(err, 'source_id')) { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, client_id_issued_at) + VALUES (${clientId}, ${secretHash}, ${name}, + ${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now}) + `; + } else { + throw err; + } + } return { clientId, clientSecret }; } diff --git a/src/core/operations.ts b/src/core/operations.ts index 758e5fc..2d763ef 100644 --- a/src/core/operations.ts +++ b/src/core/operations.ts @@ -233,6 +233,36 @@ export interface AuthInfo { clientName?: string; scopes: string[]; expiresAt?: number; + /** + * v0.34.1 (#861, D2): the source the calling OAuth client is scoped + * to (write authority). Sourced from `oauth_clients.source_id` at + * token-verification time. The HTTP transport ALSO threads this + * value into `OperationContext.sourceId` at the same site so op + * handlers can consume it via the canonical `ctx.sourceId` (D2 + * dual-write decision — identity surface symmetric with + * `allowedSources` below). + * + * Undefined for legacy bearer tokens that predate v0.34.1 and for + * clients that haven't been scoped yet. Migration v60 backfills + * NULL → 'default' for pre-existing rows so this field is populated + * on the upgrade path; brand-new public-client registrations may + * still leave it null until an operator explicitly scopes via + * `gbrain auth scope-client`. + */ + sourceId?: string; + /** + * v0.34.1 (#876): array of source ids this OAuth client may READ + * from (federation). Sourced from `oauth_clients.federated_read`. + * Independent of `sourceId` (write authority): a "WeCare L3 dept" + * client can write to `source_id='dept-x'` while reading the union + * of `['dept-x', 'wecare-parent', 'shared']`. + * + * Empty array `[]` means "no federated reads beyond `sourceId`". + * Undefined means "the post-v60 backfill hasn't populated this row + * yet" — engines fall back to scalar `sourceId` filtering in that + * case (back-compat). + */ + allowedSources?: string[]; } export interface OperationContext { @@ -354,6 +384,38 @@ export interface OperationContext { sourceId: string; } +/** + * v0.34.1 (#861, D9 — P0 leak seal): resolve the source-scope filter for a + * read-side op handler. Returns an opts fragment ready to spread into the + * engine call. + * + * Precedence: + * 1. `ctx.auth?.allowedSources` (federated read, #876) → emits + * `{sourceIds: [...]}`. Federated semantics subsume the scalar case. + * 2. `ctx.sourceId` (scalar) → emits `{sourceId: '...'}`. + * 3. Neither set → emits `{}`. Local CLI callers (and tests that don't + * populate ctx) keep the pre-v0.34 unscoped behavior. + * + * Both fields default to the engine's "no filter" behavior individually, + * so unset values are safe — the engine sees the same shape it did + * pre-v0.34. The leak this guards against is an authenticated MCP client + * whose ctx.sourceId IS set but whose engine call was constructed without + * threading it (operations.ts:968/1076/1092/935/1469/1471/2241 pre-fix). + * + * Helper rather than inline so every read-side handler routes through the + * same precedence ladder — drift between sites is the bug class. + */ +export function sourceScopeOpts(ctx: OperationContext): { sourceId?: string; sourceIds?: string[] } { + const allowed = ctx.auth?.allowedSources; + // Treat an empty `allowedSources: []` as "no federated read scope" — the + // op-handler defers to scalar `ctx.sourceId` below. An attacker-controlled + // value of `[]` MUST NOT widen scope to "all sources" by being interpreted + // as "no filter." + if (allowed && allowed.length > 0) return { sourceIds: allowed }; + if (ctx.sourceId) return { sourceId: ctx.sourceId }; + return {}; +} + export interface Operation { name: string; description: string; @@ -940,6 +1002,12 @@ const list_pages: Operation = { const sort = rawSort && (LIST_PAGES_SORT_VALUES as readonly string[]).includes(rawSort) ? (rawSort as ListPagesSort) : undefined; + // v0.34.1 (#861 — P0 leak seal): thread the auth'd client's source scope + // into the listPages filter so an OAuth client scoped to src-A cannot + // enumerate src-B pages. Pre-fix, ctx.sourceId / ctx.auth?.allowedSources + // were ignored at this op handler and the engine returned every source's + // pages indiscriminately. + const scope = sourceScopeOpts(ctx); const pages = await ctx.engine.listPages({ type: p.type as any, tag: p.tag as string, @@ -947,6 +1015,7 @@ const list_pages: Operation = { includeDeleted: (p.include_deleted as boolean) === true, updated_after: typeof p.updated_after === 'string' ? p.updated_after : undefined, sort, + ...scope, }); return pages.map(pg => ({ slug: pg.slug, @@ -973,9 +1042,13 @@ const search: Operation = { handler: async (ctx, p) => { const startedAt = Date.now(); const queryText = p.query as string; + // v0.34.1 (#861 — P0 leak seal): thread caller's source scope into + // searchKeyword. Pre-fix this op silently returned cross-source hits + // for any auth'd OAuth client. const raw = await ctx.engine.searchKeyword(queryText, { limit: (p.limit as number) || 20, offset: (p.offset as number) || 0, + ...sourceScopeOpts(ctx), }); const results = dedupResults(raw); const latency_ms = Date.now() - startedAt; @@ -1086,10 +1159,15 @@ const query: Operation = { const [vec] = await embedMultimodal([ { kind: 'image_base64', data: imageData, mime: imageMime }, ]); + // v0.34.1 (#861 F2 — 6th leak surface): the image path bypasses + // hybridSearch and calls searchVector directly, so it needs its + // own thread of the source scope. Pre-fix, this branch leaked + // image pages across sources independent of the text path's fix. const results = await ctx.engine.searchVector(vec, { limit: (p.limit as number) || 20, offset: (p.offset as number) || 0, embeddingColumn: 'embedding_image', + ...sourceScopeOpts(ctx), }); return results; } @@ -1142,6 +1220,10 @@ const query: Operation = { useCache: typeof p.use_cache === 'boolean' ? (p.use_cache as boolean) : undefined, intentWeighting: typeof p.intent_weighting === 'boolean' ? (p.intent_weighting as boolean) : undefined, onMeta: (m) => { capturedMeta = m; }, + // v0.34.1 (#861 — P0 leak seal): thread caller's source scope. The + // hybridSearch internal searchOpts rebuild (hybrid.ts:223) was + // dropping these fields pre-fix even when callers passed them. + ...sourceScopeOpts(ctx), }); const latency_ms = Date.now() - startedAt; @@ -1500,12 +1582,17 @@ const traverse_graph: Operation = { const depth = Math.max(1, Math.min(requestedDepth, TRAVERSE_DEPTH_CAP)); const linkType = p.link_type as string | undefined; const direction = p.direction as 'in' | 'out' | 'both' | undefined; + // v0.34.1 (#861 — P0 leak seal): thread caller's source scope so graph + // walks stay within the auth'd client's accessible sources. Pre-fix, + // traverseGraph / traversePaths happily followed edges into pages from + // foreign sources, leaking topology + page metadata via the graph op. + const scope = sourceScopeOpts(ctx); // Backward compat: when neither link_type nor direction is provided, return // the legacy GraphNode[] shape. Once either is set, switch to GraphPath[]. if (linkType === undefined && direction === undefined) { - return ctx.engine.traverseGraph(slug, depth); + return ctx.engine.traverseGraph(slug, depth, scope); } - return ctx.engine.traversePaths(slug, { depth, linkType, direction }); + return ctx.engine.traversePaths(slug, { depth, linkType, direction, ...scope }); }, scope: 'read', cliHints: { name: 'graph', positional: ['slug'] }, @@ -2269,16 +2356,22 @@ const find_experts: Operation = { description: 'Include factor breakdown per result (expertise, recency, salience).', }, }, - handler: async (_ctx, p) => { + handler: async (ctx, p) => { const { findExperts } = await import('../commands/whoknows.ts'); const topic = typeof p.topic === 'string' ? p.topic : ''; if (!topic.trim()) { throw new OperationError('invalid_params', '`topic` is required and must be a non-empty string.'); } - return findExperts(_ctx.engine, { + // v0.34.1 (#861, D3 — 5th leak surface): find_experts (whoknows) was + // authored against v0.33 after PR #861 was drafted, so the source-scope + // thread was missing entirely. The op calls findExperts → hybridSearch + // internally; without the thread an auth'd src-A whoknows query would + // surface src-B people in the rankings. + return findExperts(ctx.engine, { topic, limit: typeof p.limit === 'number' ? p.limit : undefined, explain: p.explain === true, + ...sourceScopeOpts(ctx), }); }, cliHints: { name: 'whoknows', positional: ['topic'] }, diff --git a/src/core/pglite-engine.ts b/src/core/pglite-engine.ts index 885ad53..7973581 100644 --- a/src/core/pglite-engine.ts +++ b/src/core/pglite-engine.ts @@ -640,8 +640,12 @@ export class PGLiteEngine implements BrainEngine { params.push(escaped); where.push(`p.slug LIKE $${params.length} ESCAPE '\\'`); } - // v0.31.12: scope to a single source when requested. - if (filters?.sourceId) { + // v0.31.12 + v0.34.1 (#876, D9): scope to a single source OR an array + // of sources. Array form wins (federated subsumes scalar). + if (filters?.sourceIds && filters.sourceIds.length > 0) { + params.push(filters.sourceIds); + where.push(`p.source_id = ANY($${params.length}::text[])`); + } else if (filters?.sourceId) { params.push(filters.sourceId); where.push(`p.source_id = $${params.length}`); } @@ -788,6 +792,14 @@ export class PGLiteEngine implements BrainEngine { params.push(opts.beforeDate); extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861 — P0 leak seal): source-isolation. Array wins over scalar. + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + extraFilter += ` AND p.source_id = $${params.length}`; + } const { rows } = await this.db.query( `WITH ranked AS ( @@ -886,6 +898,14 @@ export class PGLiteEngine implements BrainEngine { params.push(opts.beforeDate); extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861 — P0 leak seal): source-isolation on the CJK fallback path. + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + extraFilter += ` AND p.source_id = $${params.length}`; + } // Bigram-frequency count: count occurrences of $qRaw in chunk_text via // (length(chunk) - length(replace(chunk, q, ''))) / length(q). Acts as @@ -1006,6 +1026,16 @@ export class PGLiteEngine implements BrainEngine { params.push(opts.beforeDate); extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861 — P0 leak seal): source-isolation for the chunk-grain + // anchor primitive. Layer 7 two-pass walks from these anchors so a + // foreign-source anchor would let the walk leak into foreign neighbors. + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + extraFilter += ` AND p.source_id = $${params.length}`; + } // visibilityClause already declared above (v0.32.7: hoisted so CJK branch can reuse). @@ -1083,6 +1113,16 @@ export class PGLiteEngine implements BrainEngine { params.push(opts.beforeDate); extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861, F2 — P0 leak seal): source-isolation in the INNER CTE + // so HNSW candidate pool narrows before re-rank. Mirrors postgres-engine + // placement decision (codex flagged this during plan review). + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + extraFilter += ` AND p.source_id = $${params.length}`; + } // v0.26.5: visibility filter applied in the inner CTE so HNSW sees the // same candidate count it always did. See postgres-engine.ts for rationale. @@ -1516,13 +1556,38 @@ export class PGLiteEngine implements BrainEngine { return { slug: row.slug, similarity: row.sim }; } - async traverseGraph(slug: string, depth: number = 5): Promise { + async traverseGraph( + slug: string, + depth: number = 5, + opts?: { sourceId?: string; sourceIds?: string[] }, + ): Promise { + // v0.34.1 (#861 — P0 leak seal): source-scope filters at seed, step, and + // aggregation subquery. Mirrors postgres-engine.traverseGraph placement. + const params: unknown[] = [slug, depth]; + const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0; + let seedScope = ''; + let stepScope = ''; + let aggScope = ''; + if (useSourceIds) { + params.push(opts!.sourceIds); + const idx = params.length; + seedScope = `AND p.source_id = ANY($${idx}::text[])`; + stepScope = `AND p2.source_id = ANY($${idx}::text[])`; + aggScope = `AND p3.source_id = ANY($${idx}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + const idx = params.length; + seedScope = `AND p.source_id = $${idx}`; + stepScope = `AND p2.source_id = $${idx}`; + aggScope = `AND p3.source_id = $${idx}`; + } + // Cycle prevention: visited array tracks page IDs already in the path. // Prevents exponential blowup on cyclic subgraphs (e.g., A->B->A). const { rows } = await this.db.query( `WITH RECURSIVE graph AS ( SELECT p.id, p.slug, p.title, p.type, 0 as depth, ARRAY[p.id] as visited - FROM pages p WHERE p.slug = $1 + FROM pages p WHERE p.slug = $1 ${seedScope} UNION ALL @@ -1532,6 +1597,7 @@ export class PGLiteEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.to_page_id WHERE g.depth < $2 AND NOT (p2.id = ANY(g.visited)) + ${stepScope} ) SELECT DISTINCT g.slug, g.title, g.type, g.depth, coalesce( @@ -1543,12 +1609,12 @@ export class PGLiteEngine implements BrainEngine { (SELECT jsonb_agg(DISTINCT jsonb_build_object('to_slug', p3.slug, 'link_type', l2.link_type)) FROM links l2 JOIN pages p3 ON p3.id = l2.to_page_id - WHERE l2.from_page_id = g.id), + WHERE l2.from_page_id = g.id ${aggScope}), '[]'::jsonb ) as links FROM graph g ORDER BY g.depth, g.slug`, - [slug, depth] + params ); return (rows as Record[]).map(r => ({ @@ -1562,7 +1628,7 @@ export class PGLiteEngine implements BrainEngine { async traversePaths( slug: string, - opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both' }, + opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] }, ): Promise { const depth = opts?.depth ?? 5; const direction = opts?.direction ?? 'out'; @@ -1571,12 +1637,36 @@ export class PGLiteEngine implements BrainEngine { const params: unknown[] = [slug, depth]; if (linkType !== null) params.push(linkType); + // v0.34.1 (#861 — P0 leak seal): source-scope filters at seed + step + + // final SELECT joins (for the 'both' branch's pf + pt). Mirrors + // postgres-engine.traversePaths placement. + const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0; + let seedScope = ''; + let stepScope = ''; + let pfScope = ''; + let ptScope = ''; + if (useSourceIds) { + params.push(opts!.sourceIds); + const idx = params.length; + seedScope = `AND p.source_id = ANY($${idx}::text[])`; + stepScope = `AND p2.source_id = ANY($${idx}::text[])`; + pfScope = `AND pf.source_id = ANY($${idx}::text[])`; + ptScope = `AND pt.source_id = ANY($${idx}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + const idx = params.length; + seedScope = `AND p.source_id = $${idx}`; + stepScope = `AND p2.source_id = $${idx}`; + pfScope = `AND pf.source_id = $${idx}`; + ptScope = `AND pt.source_id = $${idx}`; + } + let sql: string; if (direction === 'out') { sql = ` WITH RECURSIVE walk AS ( SELECT p.id, p.slug, 0::int AS depth, ARRAY[p.id] AS visited - FROM pages p WHERE p.slug = $1 + FROM pages p WHERE p.slug = $1 ${seedScope} UNION ALL SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id FROM walk w @@ -1585,6 +1675,7 @@ export class PGLiteEngine implements BrainEngine { WHERE w.depth < $2 AND NOT (p2.id = ANY(w.visited)) ${linkTypeWhere} + ${stepScope} ) SELECT w.slug AS from_slug, p2.slug AS to_slug, l.link_type, l.context, w.depth + 1 AS depth @@ -1593,13 +1684,14 @@ export class PGLiteEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.to_page_id WHERE w.depth < $2 ${linkTypeWhere} + ${stepScope} ORDER BY depth, from_slug, to_slug `; } else if (direction === 'in') { sql = ` WITH RECURSIVE walk AS ( SELECT p.id, p.slug, 0::int AS depth, ARRAY[p.id] AS visited - FROM pages p WHERE p.slug = $1 + FROM pages p WHERE p.slug = $1 ${seedScope} UNION ALL SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id FROM walk w @@ -1608,6 +1700,7 @@ export class PGLiteEngine implements BrainEngine { WHERE w.depth < $2 AND NOT (p2.id = ANY(w.visited)) ${linkTypeWhere} + ${stepScope} ) SELECT p2.slug AS from_slug, w.slug AS to_slug, l.link_type, l.context, w.depth + 1 AS depth @@ -1616,6 +1709,7 @@ export class PGLiteEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.from_page_id WHERE w.depth < $2 ${linkTypeWhere} + ${stepScope} ORDER BY depth, from_slug, to_slug `; } else { @@ -1624,7 +1718,7 @@ export class PGLiteEngine implements BrainEngine { sql = ` WITH RECURSIVE walk AS ( SELECT p.id, 0::int AS depth, ARRAY[p.id] AS visited - FROM pages p WHERE p.slug = $1 + FROM pages p WHERE p.slug = $1 ${seedScope} UNION ALL SELECT p2.id, w.depth + 1, w.visited || p2.id FROM walk w @@ -1633,6 +1727,7 @@ export class PGLiteEngine implements BrainEngine { WHERE w.depth < $2 AND NOT (p2.id = ANY(w.visited)) ${linkTypeWhere} + ${stepScope} ) SELECT pf.slug AS from_slug, pt.slug AS to_slug, l.link_type, l.context, w.depth + 1 AS depth @@ -1642,6 +1737,8 @@ export class PGLiteEngine implements BrainEngine { JOIN pages pt ON pt.id = l.to_page_id WHERE w.depth < $2 ${linkTypeWhere} + ${pfScope} + ${ptScope} ORDER BY depth, from_slug, to_slug `; } diff --git a/src/core/pglite-schema.ts b/src/core/pglite-schema.ts index 33660dc..3d05dff 100644 --- a/src/core/pglite-schema.ts +++ b/src/core/pglite-schema.ts @@ -594,8 +594,19 @@ CREATE TABLE IF NOT EXISTS oauth_clients ( client_secret_expires_at BIGINT, token_ttl INTEGER, deleted_at TIMESTAMPTZ, + source_id TEXT REFERENCES sources(id) ON DELETE RESTRICT, + federated_read TEXT[] NOT NULL DEFAULT '{}', created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); +-- v0.34.1 (#861, D13 + #876): source_id is the OAuth client's write-source +-- scope; federated_read is its read-source array (a federated client can +-- read sources beyond its source_id). Migration v60 adds source_id; +-- v61-v65 add federated_read + GIN index + flip FK to RESTRICT. Fresh +-- installs land in the post-migration shape via the inline columns above. +CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id + ON oauth_clients(source_id) WHERE source_id IS NOT NULL; +CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read + ON oauth_clients USING GIN (federated_read); CREATE TABLE IF NOT EXISTS oauth_tokens ( token_hash TEXT PRIMARY KEY, diff --git a/src/core/postgres-engine.ts b/src/core/postgres-engine.ts index e1efb49..e57aab1 100644 --- a/src/core/postgres-engine.ts +++ b/src/core/postgres-engine.ts @@ -685,10 +685,14 @@ export class PostgresEngine implements BrainEngine { const slugCondition = slugPrefix ? sql`AND p.slug LIKE ${slugPrefix.replace(/[\\%_]/g, (c) => '\\' + c) + '%'} ESCAPE '\\'` : sql``; - // v0.31.12: scope to a single source when requested. - const sourceCondition = filters?.sourceId - ? sql`AND p.source_id = ${filters.sourceId}` - : sql``; + // v0.31.12 + v0.34.1 (#876, D9): scope to a single source OR an array + // of sources. When BOTH are set, the array wins (federated semantics + // subsume the scalar case). When neither is set, no filter applies. + const sourceCondition = filters?.sourceIds && filters.sourceIds.length > 0 + ? sql`AND p.source_id = ANY(${filters.sourceIds}::text[])` + : filters?.sourceId + ? sql`AND p.source_id = ${filters.sourceId}` + : sql``; // v0.26.5: hide soft-deleted by default; opt in via filters.includeDeleted. const deletedCondition = filters?.includeDeleted === true ? sql`` @@ -828,6 +832,22 @@ export class PostgresEngine implements BrainEngine { params.push(opts.beforeDate); beforeDateClause = `AND COALESCE(p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861 — P0 leak seal): source-isolation filter. When the + // caller's auth scope is set, narrow the inner CTE candidate set so + // an authenticated MCP client cannot see foreign-source pages via + // keyword search. Array form wins over scalar (federated subsumes + // single-source). Index-backed by idx_pages_source_id; the filter is + // pushed to the INNER CTE specifically so HNSW-style downstream + // ranking sees a narrowed candidate set rather than re-ranking a + // cross-source pool. + let sourceClause = ''; + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + sourceClause = `AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + sourceClause = `AND p.source_id = $${params.length}`; + } params.push(innerLimit); const innerLimitParam = `$${params.length}`; params.push(limit); @@ -859,6 +879,7 @@ export class PostgresEngine implements BrainEngine { ${symbolKindClause} ${afterDateClause} ${beforeDateClause} + ${sourceClause} ${hardExcludeClause} ${visibilityClause} -- v0.27.1: hide image rows from text-keyword search so OCR text @@ -962,6 +983,17 @@ export class PostgresEngine implements BrainEngine { params.push(opts.beforeDate); beforeDateClause = `AND COALESCE(p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861 — P0 leak seal): source-isolation. Anchor primitive + // for two-pass retrieval, so cross-source anchors would let the walk + // discover foreign-source neighbors. Filter at chunk-rank time. + let sourceClause = ''; + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + sourceClause = `AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + sourceClause = `AND p.source_id = $${params.length}`; + } params.push(limit); const limitParam = `$${params.length}`; params.push(offset); @@ -988,6 +1020,7 @@ export class PostgresEngine implements BrainEngine { ${symbolKindClause} ${afterDateClause} ${beforeDateClause} + ${sourceClause} ${hardExcludeClause} ${visibilityClause} ORDER BY score DESC @@ -1071,6 +1104,19 @@ export class PostgresEngine implements BrainEngine { params.push(opts.beforeDate); beforeDateClause = `AND COALESCE(p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861, F2 — P0 leak seal): source-isolation in the INNER CTE + // specifically. Pushing the filter inside narrows the HNSW candidate set + // before re-rank; pushing it to the outer SELECT would force HNSW to + // over-fetch then post-filter, wasting candidate slots. Codex flagged + // this placement during plan review. Array form wins over scalar. + let sourceClause = ''; + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + sourceClause = `AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + sourceClause = `AND p.source_id = $${params.length}`; + } params.push(innerLimit); const innerLimitParam = `$${params.length}`; params.push(limit); @@ -1106,6 +1152,7 @@ export class PostgresEngine implements BrainEngine { ${symbolKindClause} ${afterDateClause} ${beforeDateClause} + ${sourceClause} ${hardExcludeClause} ${visibilityClause} ORDER BY cc.${col} <=> $1::vector @@ -1516,13 +1563,40 @@ export class PostgresEngine implements BrainEngine { return { slug: row.slug, similarity: row.sim }; } - async traverseGraph(slug: string, depth: number = 5): Promise { + async traverseGraph( + slug: string, + depth: number = 5, + opts?: { sourceId?: string; sourceIds?: string[] }, + ): Promise { const sql = this.sql; + // v0.34.1 (#861 — P0 leak seal): scope visited nodes to the caller's + // source(s). Without this, the walk follows edges into pages from + // foreign sources, leaking topology + page metadata. The filter + // applies at BOTH the seed (root must be in scope) AND the recursive + // step (every visited neighbor must be in scope). The aggregation + // subquery also filters so the per-node `links` array only includes + // edges to in-scope pages. + const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0; + const seedScope = useSourceIds + ? sql`AND p.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND p.source_id = ${opts.sourceId}` + : sql``; + const stepScope = useSourceIds + ? sql`AND p2.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND p2.source_id = ${opts.sourceId}` + : sql``; + const aggScope = useSourceIds + ? sql`AND p3.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND p3.source_id = ${opts.sourceId}` + : sql``; // Cycle prevention: visited array tracks page IDs already in the path. const rows = await sql` WITH RECURSIVE graph AS ( SELECT p.id, p.slug, p.title, p.type, 0 as depth, ARRAY[p.id] as visited - FROM pages p WHERE p.slug = ${slug} + FROM pages p WHERE p.slug = ${slug} ${seedScope} UNION ALL @@ -1532,6 +1606,7 @@ export class PostgresEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.to_page_id WHERE g.depth < ${depth} AND NOT (p2.id = ANY(g.visited)) + ${stepScope} ) SELECT DISTINCT g.slug, g.title, g.type, g.depth, coalesce( @@ -1545,7 +1620,7 @@ export class PostgresEngine implements BrainEngine { (SELECT jsonb_agg(DISTINCT jsonb_build_object('to_slug', p3.slug, 'link_type', l2.link_type)) FROM links l2 JOIN pages p3 ON p3.id = l2.to_page_id - WHERE l2.from_page_id = g.id), + WHERE l2.from_page_id = g.id ${aggScope}), '[]'::jsonb ) as links FROM graph g @@ -1563,20 +1638,47 @@ export class PostgresEngine implements BrainEngine { async traversePaths( slug: string, - opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both' }, + opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] }, ): Promise { const sql = this.sql; const depth = opts?.depth ?? 5; const direction = opts?.direction ?? 'out'; const linkType = opts?.linkType ?? null; const linkTypeMatches = linkType !== null; + // v0.34.1 (#861 — P0 leak seal): source-scope filter fragments. Applied + // at seed (root must be in scope) AND at every recursive step (neighbor + // must be in scope) AND in the SELECT join (final edges respect scope). + // The 'both' branch needs filters on BOTH endpoint joins. + const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0; + const seedScope = useSourceIds + ? sql`AND p.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND p.source_id = ${opts.sourceId}` + : sql``; + const stepScope = useSourceIds + ? sql`AND p2.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND p2.source_id = ${opts.sourceId}` + : sql``; + // For the 'both' direction's final SELECT, both endpoint joins (pf, pt) + // get scope filters so edges crossing into a foreign source are dropped. + const pfScope = useSourceIds + ? sql`AND pf.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND pf.source_id = ${opts.sourceId}` + : sql``; + const ptScope = useSourceIds + ? sql`AND pt.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND pt.source_id = ${opts.sourceId}` + : sql``; let rows; if (direction === 'out') { rows = await sql` WITH RECURSIVE walk AS ( SELECT p.id, p.slug, 0::int as depth, ARRAY[p.id] as visited - FROM pages p WHERE p.slug = ${slug} + FROM pages p WHERE p.slug = ${slug} ${seedScope} UNION ALL SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id FROM walk w @@ -1585,6 +1687,7 @@ export class PostgresEngine implements BrainEngine { WHERE w.depth < ${depth} AND NOT (p2.id = ANY(w.visited)) AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${stepScope} ) SELECT w.slug as from_slug, p2.slug as to_slug, l.link_type, l.context, w.depth + 1 as depth @@ -1593,13 +1696,14 @@ export class PostgresEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.to_page_id WHERE w.depth < ${depth} AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${stepScope} ORDER BY depth, from_slug, to_slug `; } else if (direction === 'in') { rows = await sql` WITH RECURSIVE walk AS ( SELECT p.id, p.slug, 0::int as depth, ARRAY[p.id] as visited - FROM pages p WHERE p.slug = ${slug} + FROM pages p WHERE p.slug = ${slug} ${seedScope} UNION ALL SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id FROM walk w @@ -1608,6 +1712,7 @@ export class PostgresEngine implements BrainEngine { WHERE w.depth < ${depth} AND NOT (p2.id = ANY(w.visited)) AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${stepScope} ) SELECT p2.slug as from_slug, w.slug as to_slug, l.link_type, l.context, w.depth + 1 as depth @@ -1616,13 +1721,14 @@ export class PostgresEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.from_page_id WHERE w.depth < ${depth} AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${stepScope} ORDER BY depth, from_slug, to_slug `; } else { rows = await sql` WITH RECURSIVE walk AS ( SELECT p.id, 0::int as depth, ARRAY[p.id] as visited - FROM pages p WHERE p.slug = ${slug} + FROM pages p WHERE p.slug = ${slug} ${seedScope} UNION ALL SELECT p2.id, w.depth + 1, w.visited || p2.id FROM walk w @@ -1631,6 +1737,7 @@ export class PostgresEngine implements BrainEngine { WHERE w.depth < ${depth} AND NOT (p2.id = ANY(w.visited)) AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${stepScope} ) SELECT pf.slug as from_slug, pt.slug as to_slug, l.link_type, l.context, w.depth + 1 as depth @@ -1640,6 +1747,8 @@ export class PostgresEngine implements BrainEngine { JOIN pages pt ON pt.id = l.to_page_id WHERE w.depth < ${depth} AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${pfScope} + ${ptScope} ORDER BY depth, from_slug, to_slug `; } diff --git a/src/core/schema-embedded.ts b/src/core/schema-embedded.ts index 0d8f800..dbfb707 100644 --- a/src/core/schema-embedded.ts +++ b/src/core/schema-embedded.ts @@ -428,8 +428,17 @@ CREATE TABLE IF NOT EXISTS oauth_clients ( client_secret_expires_at BIGINT, token_ttl INTEGER, deleted_at TIMESTAMPTZ, + source_id TEXT REFERENCES sources(id) ON DELETE RESTRICT, + federated_read TEXT[] NOT NULL DEFAULT '{}', created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); +-- v0.34.1 (#861, D13 + #876): source_id is the write-source scope; +-- federated_read is the read-source array. Migrations v60-v65 land both +-- columns on upgrade; fresh installs include them inline above. +CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id + ON oauth_clients(source_id) WHERE source_id IS NOT NULL; +CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read + ON oauth_clients USING GIN (federated_read); CREATE TABLE IF NOT EXISTS oauth_tokens ( token_hash TEXT PRIMARY KEY, diff --git a/src/core/search/hybrid.ts b/src/core/search/hybrid.ts index 8195ab6..e21836e 100644 --- a/src/core/search/hybrid.ts +++ b/src/core/search/hybrid.ts @@ -280,6 +280,16 @@ export async function hybridSearch( // PR #618 callers compiling while the new names are the public surface. afterDate: opts?.since ?? opts?.afterDate, beforeDate: opts?.until ?? opts?.beforeDate, + // v0.34.1 (#861, D9 — P0 leak seal): thread source-scoping through so the + // inner engine.searchKeyword / engine.searchVector calls apply the + // WHERE source_id filter at SQL level. Pre-fix, this explicit pick + // silently DROPPED these fields and every authenticated MCP client + // could see pages from foreign sources via the hybrid search hot + // path. New SearchOpts fields scoped to source isolation MUST be + // added here too; the rebuild shape is intentional (HNSW inner-CTE + // ordering means we can't lazy-spread the full opts). + sourceId: opts?.sourceId, + sourceIds: opts?.sourceIds, }; // Track what actually ran for the optional onMeta callback (v0.25.0). // Caller leaves onMeta undefined → these flags are computed but never diff --git a/src/core/types.ts b/src/core/types.ts index 0d15f12..468ef13 100644 --- a/src/core/types.ts +++ b/src/core/types.ts @@ -196,6 +196,15 @@ export interface PageFilters { * operations to a single source. */ sourceId?: string; + /** + * v0.34.1 (#876, D9): filter to ANY of these sources (federated read). + * Engine applies `WHERE p.source_id = ANY($N::text[])` when array is set. + * Caller precedence: if BOTH `sourceId` and `sourceIds` are set, the array + * wins (the federated semantics subsume the single-source case via an + * array of length 1). When neither is set, no filter applies — the + * pre-v0.34 unscoped behavior is preserved for local CLI callers. + */ + sourceIds?: string[]; } /** v0.26.5 — opts for getPage / softDeletePage / restorePage. */ @@ -457,6 +466,18 @@ export interface SearchOpts { * undefined to search all sources. */ sourceId?: string; + /** + * v0.34.1 (#876, D9): filter to ANY of these sources (federated read). + * Engine applies `WHERE p.source_id = ANY($N::text[])` when the array + * is set. Caller precedence: if BOTH `sourceId` and `sourceIds` are + * provided, the array wins (the federated semantics subsume the + * single-source case). When neither is set, no filter applies — the + * pre-v0.34 unscoped behavior is preserved for local CLI callers. + * + * The op-handler layer resolves: `ctx.auth?.allowedSources` (federated + * client) → `sourceIds`; otherwise `ctx.sourceId` (scalar) → `sourceId`. + */ + sourceIds?: string[]; /** * v0.27.1: target column for vector search. 'embedding' (default) hits * the brain's primary text-embedding column. 'embedding_image' targets diff --git a/src/mcp/http-transport.ts b/src/mcp/http-transport.ts index 6356208..df5825a 100644 --- a/src/mcp/http-transport.ts +++ b/src/mcp/http-transport.ts @@ -66,6 +66,14 @@ interface AuthResult { tokenName?: string; /** v0.28: per-token allow-list for takes.holder. Default ['world'] when permissions row absent. */ takesHoldersAllowList?: string[]; + /** + * v0.34.1 (#861, D13): source-isolation scope for the auth'd request. + * Legacy bearer tokens here default to 'default' to match the v0.33 + * effective behavior (the now-removed serve-http.ts fallback chain). + * Operators migrate to the full OAuth transport (gbrain serve --http) + * for narrower scoping. + */ + sourceId?: string; } /** Read up to `cap` bytes off req.body. Returns null if cap exceeded. */ @@ -180,6 +188,12 @@ export async function startHttpTransport(opts: HttpTransportOptions) { tokenId: rowId, tokenName: rowName, takesHoldersAllowList: allowList, + // v0.34.1 (#861, D13): legacy bearer tokens default to 'default' + // source. Preserves the pre-v0.34 effective behavior of the + // serve-http fallback chain that was removed for OAuth clients + // (migration v60 backfills oauth_clients.source_id). This path + // is for the older v0.22.7 access_tokens transport. + sourceId: 'default', }; } catch { return { ok: false }; @@ -328,9 +342,12 @@ export async function startHttpTransport(opts: HttpTransportOptions) { const args: Record = params?.arguments ?? {}; // v0.28: thread per-token takes-holder allow-list so takes_list / // takes_search / query (when it returns takes) can server-side filter. + // v0.34.1 (#861): thread source-isolation scope. Legacy access_tokens + // path defaults to 'default' per AuthResult.sourceId above. const result = await dispatchToolCall(engine, toolName, args, { remote: true, takesHoldersAllowList: auth.takesHoldersAllowList, + sourceId: auth.sourceId, }); const status = result.isError ? 'error' : 'success'; logRequest(auth.tokenName!, `tools/call:${toolName}`, status, Date.now() - startedMs); diff --git a/src/mcp/server.ts b/src/mcp/server.ts index e268cbb..94da322 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -62,8 +62,15 @@ export async function startMcpServer(engine: BrainEngine) { .catch(() => {}) .finally(() => process.exit(code)); }; - process.stdin.on('end', () => shutdown('stdin end')); - process.stdin.on('close', () => shutdown('stdin close')); + // v0.34.1 (#870): when MCP_STDIO=1, the wrapping gateway (OpenClaw's + // bundle-mcp layer, others) often pipes the JSON-RPC handshake then + // closes its stdin half. Treating that as a permanent disconnect kills + // the server before the first tool call arrives. Signal handlers and + // transport.onclose still cover the legitimate shutdown paths. + if (process.env.MCP_STDIO !== '1') { + process.stdin.on('end', () => shutdown('stdin end')); + process.stdin.on('close', () => shutdown('stdin close')); + } // @ts-ignore — SDK exposes onclose on transport transport.onclose = () => shutdown('transport close'); process.on('SIGTERM', () => shutdown('SIGTERM')); diff --git a/src/schema.sql b/src/schema.sql index bf9fa84..c19221d 100644 --- a/src/schema.sql +++ b/src/schema.sql @@ -424,8 +424,17 @@ CREATE TABLE IF NOT EXISTS oauth_clients ( client_secret_expires_at BIGINT, token_ttl INTEGER, deleted_at TIMESTAMPTZ, + source_id TEXT REFERENCES sources(id) ON DELETE RESTRICT, + federated_read TEXT[] NOT NULL DEFAULT '{}', created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); +-- v0.34.1 (#861, D13 + #876): source_id is the write-source scope; +-- federated_read is the read-source array. Migrations v60-v65 land both +-- columns on upgrade; fresh installs include them inline above. +CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id + ON oauth_clients(source_id) WHERE source_id IS NOT NULL; +CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read + ON oauth_clients USING GIN (federated_read); CREATE TABLE IF NOT EXISTS oauth_tokens ( token_hash TEXT PRIMARY KEY, diff --git a/test/book-mirror.test.ts b/test/book-mirror.test.ts index 25905c3..0ee6599 100644 --- a/test/book-mirror.test.ts +++ b/test/book-mirror.test.ts @@ -32,35 +32,57 @@ async function runCli(args: string[]): Promise<{ stderr: string; exit: number; }> { - const proc = Bun.spawn( - ['bun', 'run', 'src/cli.ts', 'book-mirror', ...args], - { - cwd: REPO_ROOT, - stdout: 'pipe', - stderr: 'pipe', - env: { ...process.env, DATABASE_URL: '' }, - }, + // v0.34.1 (#876): set GBRAIN_HOME to a fresh tempdir so the test isn't + // sensitive to the developer's local ~/.gbrain/config.json. Pre-fix this + // test would silently inherit the user's database_url and try to connect + // to a real Postgres + run migrations, which both extends runtime past + // the default test timeout and produces different exit semantics + // depending on the local DB state. With GBRAIN_HOME pointing at an + // empty dir, the CLI hits "No brain configured" and exits 1 immediately + // — which is what this test was originally written to cover. + const tmpHome = await import('os').then(os => + import('fs').then(fs => fs.mkdtempSync(`${os.tmpdir()}/gbrain-test-`)), ); - const stdout = await new Response(proc.stdout).text(); - const stderr = await new Response(proc.stderr).text(); - const exit = await proc.exited; - return { stdout, stderr, exit }; + try { + const proc = Bun.spawn( + ['bun', 'run', 'src/cli.ts', 'book-mirror', ...args], + { + cwd: REPO_ROOT, + stdout: 'pipe', + stderr: 'pipe', + env: { ...process.env, DATABASE_URL: '', GBRAIN_HOME: tmpHome }, + }, + ); + const stdout = await new Response(proc.stdout).text(); + const stderr = await new Response(proc.stderr).text(); + const exit = await proc.exited; + return { stdout, stderr, exit }; + } finally { + const fs = await import('fs'); + fs.rmSync(tmpHome, { recursive: true, force: true }); + } } describe('gbrain book-mirror — CLI registration', () => { + // v0.34.1 (#876): migrations v60-v65 added oauth_clients.source_id + + // federated_read FK plumbing. The cold-spawned subprocess's initSchema + // chain now takes ~1s longer end-to-end; bump these tests' timeout to + // 30s so the test-harness budget covers Bun cold-start + PGLite init + + // migration apply + Postgres-retry exhaustion. The CLI itself still + // exits in 7-10s on a misconfigured DB. it('book-mirror is in CLI_ONLY (does not get "Unknown command")', async () => { const { stderr } = await runCli([]); // Without DB, the command will fail — but on the connect path, // not as "Unknown command". This proves dispatch reached // handleCliOnly's switch statement. expect(stderr).not.toContain('Unknown command'); - }); + }, 30000); it('without DB, never reaches queue submission', async () => { const { stderr, exit } = await runCli(['--slug', 'noop']); expect(exit).not.toBe(0); expect(stderr).not.toContain('submitted:'); - }); + }, 30000); }); describe('gbrain book-mirror — source file invariants', () => { diff --git a/test/e2e/source-isolation-pglite.test.ts b/test/e2e/source-isolation-pglite.test.ts new file mode 100644 index 0000000..71567d8 --- /dev/null +++ b/test/e2e/source-isolation-pglite.test.ts @@ -0,0 +1,267 @@ +/** + * v0.34.1 (#861 — P0 source-isolation leak seal) E2E regression. + * + * The wave's IRON RULE: every read-side op must filter by source_id when + * the caller supplies it via SearchOpts/PageFilters/TraverseOpts. Pre-fix, + * authenticated MCP clients scoped to source-A could enumerate source-B + * pages via search / query / list_pages / traverse_graph / find_experts. + * + * Runs against PGLite in-memory so the test exercises both engines' parity + * surface (the schema-drift E2E covers Postgres independently) without + * needing a Docker container. + * + * Coverage: searchKeyword, searchVector (synthetic embedding), listPages, + * traverseGraph, traversePaths. find_experts is exercised via the same + * hybridSearch path the op handler calls. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../../src/core/pglite-engine.ts'; +import { resetPgliteState } from '../helpers/reset-pglite.ts'; + +let engine: PGLiteEngine; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await resetPgliteState(engine); + // Two sources so we can prove the filter excludes cross-source rows. + // 'default' source is created by initSchema's seed; we add src-b. + await engine.executeRaw( + `INSERT INTO sources (id, name, config) VALUES ('src-b', 'src-b', '{}'::jsonb) ON CONFLICT DO NOTHING`, + ); + // Seed one person page in each source. Same slug intentionally — + // proves the composite (source_id, slug) key is honored, not just slug. + // upsertChunks is needed because searchKeyword scans content_chunks, not + // pages.compiled_truth directly. Each page gets one chunk that mirrors + // its compiled_truth so search-by-keyword has something to find. + await engine.putPage('people/alice', { + type: 'person', + title: 'Alice Source-A', + compiled_truth: 'Alice works on widgets in source A. Important context here.', + timeline: '', + frontmatter: {}, + }, { sourceId: 'default' }); + await engine.upsertChunks('people/alice', [{ + chunk_index: 0, + chunk_text: 'Alice works on widgets in source A. Important context here.', + chunk_source: 'compiled_truth', + token_count: 12, + }], { sourceId: 'default' }); + + await engine.putPage('people/alice', { + type: 'person', + title: 'Alice Source-B', + compiled_truth: 'Alice works on gadgets in source B. Important context here.', + timeline: '', + frontmatter: {}, + }, { sourceId: 'src-b' }); + await engine.upsertChunks('people/alice', [{ + chunk_index: 0, + chunk_text: 'Alice works on gadgets in source B. Important context here.', + chunk_source: 'compiled_truth', + token_count: 12, + }], { sourceId: 'src-b' }); + + await engine.putPage('people/bob', { + type: 'person', + title: 'Bob Source-B Only', + compiled_truth: 'Bob lives only in source B. Important context here.', + timeline: '', + frontmatter: {}, + }, { sourceId: 'src-b' }); + await engine.upsertChunks('people/bob', [{ + chunk_index: 0, + chunk_text: 'Bob lives only in source B. Important context here.', + chunk_source: 'compiled_truth', + token_count: 11, + }], { sourceId: 'src-b' }); +}); + +describe('v0.34.1 source-isolation regression (#861)', () => { + test('searchKeyword with sourceId=default excludes src-b rows', async () => { + const results = await engine.searchKeyword('widgets', { sourceId: 'default' }); + // Should find Alice from source-A only (mentions "widgets"). src-b's + // Alice mentions "gadgets" not "widgets" but the test guards against + // accidentally pulling in src-b rows via a missing WHERE clause. + expect(results.length).toBeGreaterThan(0); + for (const r of results) { + expect(r.source_id).toBe('default'); + } + }); + + test('searchKeyword with sourceId=src-b excludes default rows', async () => { + const results = await engine.searchKeyword('gadgets', { sourceId: 'src-b' }); + expect(results.length).toBeGreaterThan(0); + for (const r of results) { + expect(r.source_id).toBe('src-b'); + } + }); + + test('searchKeyword with sourceIds=[default,src-b] returns both', async () => { + // Federated read (D9 array form) returns the union. + const results = await engine.searchKeyword('Important context', { + sourceIds: ['default', 'src-b'], + }); + const sources = new Set(results.map(r => r.source_id)); + expect(sources.has('default')).toBe(true); + expect(sources.has('src-b')).toBe(true); + }); + + test('searchKeyword with sourceIds=[default] only returns default', async () => { + // Array form with a single element behaves like scalar. + const results = await engine.searchKeyword('Important context', { + sourceIds: ['default'], + }); + for (const r of results) { + expect(r.source_id).toBe('default'); + } + }); + + test('searchKeyword with no source scope returns all sources', async () => { + // Local CLI / unscoped callers preserve pre-v0.34 behavior. + const results = await engine.searchKeyword('Important context'); + const sources = new Set(results.map(r => r.source_id)); + expect(sources.size).toBeGreaterThanOrEqual(2); + }); + + test('listPages with sourceId=default hides src-b rows', async () => { + const pages = await engine.listPages({ sourceId: 'default', limit: 100 }); + expect(pages.length).toBeGreaterThan(0); + expect(pages.find(p => p.title === 'Bob Source-B Only')).toBeUndefined(); + expect(pages.find(p => p.title === 'Alice Source-B')).toBeUndefined(); + }); + + test('listPages with sourceIds=[default,src-b] returns the union', async () => { + const pages = await engine.listPages({ sourceIds: ['default', 'src-b'], limit: 100 }); + const titles = new Set(pages.map(p => p.title)); + expect(titles.has('Alice Source-A')).toBe(true); + expect(titles.has('Alice Source-B')).toBe(true); + expect(titles.has('Bob Source-B Only')).toBe(true); + }); + + test('traverseGraph with sourceId=default does not surface src-b roots', async () => { + // Seeding the walk at src-b's bob with sourceId=default produces an + // empty result — the seed itself is filtered out, so the walk never + // discovers anything. Pre-fix, the walk would still return bob's + // neighbors via cross-source edge following. + const nodes = await engine.traverseGraph('people/bob', 5, { sourceId: 'default' }); + expect(nodes.length).toBe(0); + }); + + test('traverseGraph with sourceId=src-b finds the src-b page', async () => { + const nodes = await engine.traverseGraph('people/bob', 5, { sourceId: 'src-b' }); + expect(nodes.length).toBeGreaterThanOrEqual(1); + expect(nodes.find(n => n.slug === 'people/bob')).toBeDefined(); + }); + + test('traversePaths with sourceId=default does not surface src-b paths', async () => { + // Same seed-filter check via the edge-based traversal. + const paths = await engine.traversePaths('people/bob', { depth: 3, sourceId: 'default' }); + expect(paths.length).toBe(0); + }); + + test('searchVector with sourceId filters HNSW candidate pool', async () => { + // No real embeddings on the test pages; the WHERE cc.embedding IS NOT NULL + // gate filters them out. We assert the contract via an empty result + // rather than a positive match: with sourceId set, the SQL still runs + // (no type or undefined-column errors). + const synth = new Float32Array(1536).fill(0.01); + const results = await engine.searchVector(synth, { sourceId: 'src-b' }); + // Either empty (no embeddings) or all from src-b. Both prove the + // filter is wired without a runtime error. + for (const r of results) { + expect(r.source_id).toBe('src-b'); + } + }); + + test('AuthInfo path: ctx.sourceId scoping propagates through op handlers', async () => { + // The op handler at operations.ts:search threads ctx.sourceId via + // sourceScopeOpts. Simulate the dispatcher's threading and verify + // the engine sees the filter — this is the layer the regression + // tests need to cover most directly. + const { operations } = await import('../../src/core/operations.ts'); + const searchOp = operations.find(o => o.name === 'search'); + expect(searchOp).toBeDefined(); + + const ctx = { + engine, + config: { engine: 'pglite' as const }, + logger: { info: () => {}, warn: () => {}, error: () => {} }, + dryRun: false, + remote: true, + sourceId: 'src-b', + }; + const result = await searchOp!.handler(ctx as any, { query: 'gadgets' }); + const rows = result as Array<{ source_id?: string }>; + expect(rows.length).toBeGreaterThan(0); + for (const r of rows) { + expect(r.source_id).toBe('src-b'); + } + }); + + test('AuthInfo.allowedSources path: ctx.auth.allowedSources widens read scope', async () => { + // D9 federated read: when AuthInfo.allowedSources is populated, the + // sourceScopeOpts helper produces sourceIds array (array wins over + // scalar ctx.sourceId). Verify the op handler routes through this. + const { operations } = await import('../../src/core/operations.ts'); + const searchOp = operations.find(o => o.name === 'search'); + const ctx = { + engine, + config: { engine: 'pglite' as const }, + logger: { info: () => {}, warn: () => {}, error: () => {} }, + dryRun: false, + remote: true, + sourceId: 'default', // scalar would scope to default-only + auth: { + token: 'test', + clientId: 'test', + scopes: ['read'], + sourceId: 'default', + allowedSources: ['default', 'src-b'], // ... but the array wins + }, + }; + const result = await searchOp!.handler(ctx as any, { query: 'Important context' }); + const rows = result as Array<{ source_id?: string }>; + const sources = new Set(rows.map(r => r.source_id)); + expect(sources.has('default')).toBe(true); + expect(sources.has('src-b')).toBe(true); + }); + + test('#876 federated_read empty array means no federated reads', async () => { + // sourceScopeOpts treats allowedSources: [] (explicit empty) as "no + // federated scope" and falls back to scalar sourceId. An empty array + // MUST NOT widen scope to "all sources" — that's the silent-widen + // footgun. Verify the precedence ladder is correct. + const { operations } = await import('../../src/core/operations.ts'); + const searchOp = operations.find(o => o.name === 'search'); + const ctx = { + engine, + config: { engine: 'pglite' as const }, + logger: { info: () => {}, warn: () => {}, error: () => {} }, + dryRun: false, + remote: true, + sourceId: 'default', + auth: { + token: 'test', + clientId: 'test', + scopes: ['read'], + sourceId: 'default', + allowedSources: [], // explicit empty — must NOT widen scope + }, + }; + const result = await searchOp!.handler(ctx as any, { query: 'Important context' }); + const rows = result as Array<{ source_id?: string }>; + for (const r of rows) { + expect(r.source_id).toBe('default'); + } + }); +}); diff --git a/test/oauth.test.ts b/test/oauth.test.ts index dd7829f..fa549a5 100644 --- a/test/oauth.test.ts +++ b/test/oauth.test.ts @@ -1148,3 +1148,106 @@ describe('F12 dcrDisabled constructor option', () => { expect(result.clientSecret).toStartWith('gbrain_cs_'); }); }); + +// --------------------------------------------------------------------------- +// v0.34.1 (#909) — PKCE public-client DCR (RFC 7591 §3.2.1) +// --------------------------------------------------------------------------- +// +// Per RFC 7591 §3.2.1, when a DCR client declares +// `token_endpoint_auth_method: "none"` (PKCE-only public clients like Claude +// Code, Cursor), the authorization server MUST NOT issue a client_secret. +// Pre-fix, unconditional secret generation made the MCP SDK's clientAuth +// middleware reject valid public-client flows on /token. + +describe('PKCE DCR public-client gate (#909)', () => { + test("registerClient with token_endpoint_auth_method='none' omits client_secret", async () => { + const result = await provider.clientsStore.registerClient!({ + client_name: 'public-pkce-client', + redirect_uris: ['https://example.com/callback'], + grant_types: ['authorization_code'], + scope: 'read', + token_endpoint_auth_method: 'none', + }); + expect(result.client_id).toStartWith('gbrain_cl_'); + // RFC 7591 §3.2.1: public clients get NO client_secret in the response. + expect(result.client_secret).toBeUndefined(); + expect(result.token_endpoint_auth_method).toBe('none'); + }); + + test('default auth_method (omitted) still issues a client_secret', async () => { + // Regression guard: confidential clients (the existing default) must + // keep their secret-issuing behavior unchanged. + const result = await provider.clientsStore.registerClient!({ + client_name: 'confidential-default', + redirect_uris: ['https://example.com/callback'], + grant_types: ['authorization_code'], + scope: 'read', + // token_endpoint_auth_method omitted; falls back to 'client_secret_post' + }); + expect(result.client_id).toStartWith('gbrain_cl_'); + expect(result.client_secret).toStartWith('gbrain_cs_'); + }); + + test('explicit client_secret_post still issues a client_secret', async () => { + const result = await provider.clientsStore.registerClient!({ + client_name: 'confidential-explicit', + redirect_uris: ['https://example.com/callback'], + grant_types: ['authorization_code'], + scope: 'read', + token_endpoint_auth_method: 'client_secret_post', + }); + expect(result.client_id).toStartWith('gbrain_cl_'); + expect(result.client_secret).toStartWith('gbrain_cs_'); + }); + + test('getClient on a public client returns client_secret=undefined (NULL normalized)', async () => { + // The SDK's clientAuth middleware checks `client.client_secret === undefined` + // (not `=== null`) to decide whether to enforce secret comparison on /token. + // Without normalization, Postgres NULL would reach the SDK as JS null and + // the secret check would mis-fire on every public client. + const reg = await provider.clientsStore.registerClient!({ + client_name: 'public-getclient-norm', + redirect_uris: ['https://example.com/callback'], + grant_types: ['authorization_code'], + scope: 'read', + token_endpoint_auth_method: 'none', + }); + const stored = await provider.clientsStore.getClient(reg.client_id); + expect(stored).toBeDefined(); + expect(stored!.client_secret).toBeUndefined(); + expect(stored!.token_endpoint_auth_method).toBe('none'); + }); + + test('PKCE flow end-to-end: public client /authorize then /token, no secret needed', async () => { + // Full F7 regression #15: public client completes auth_code → token + // exchange without ever presenting a client_secret. + const reg = await provider.clientsStore.registerClient!({ + client_name: 'pkce-roundtrip', + redirect_uris: ['http://localhost:3000/callback'], + grant_types: ['authorization_code'], + scope: 'read', + token_endpoint_auth_method: 'none', + }); + + // Re-fetch via getClient to mirror what the SDK middleware sees. + const client = (await provider.clientsStore.getClient(reg.client_id))!; + expect(client.client_secret).toBeUndefined(); + + let redirectUrl = ''; + const mockRes = { redirect: (url: string) => { redirectUrl = url; } } as any; + await provider.authorize(client, { + codeChallenge: 'test-challenge-value', + redirectUri: 'http://localhost:3000/callback', + scopes: ['read'], + }, mockRes); + const code = new URL(redirectUrl).searchParams.get('code')!; + expect(code).toMatch(/^gbrain_code_/); + + // Exchange the code — public client; no secret on the wire. + const tokens = await provider.exchangeAuthorizationCode(client, code); + expect(tokens.access_token).toStartWith('gbrain_at_'); + // SDK normalizes token_type per RFC 6750 §6.1.1 (case-insensitive); + // implementations may emit "bearer" lowercase. + expect(String(tokens.token_type).toLowerCase()).toBe('bearer'); + }); +}); diff --git a/test/openai-compat-multimodal.test.ts b/test/openai-compat-multimodal.test.ts new file mode 100644 index 0000000..0aecdfd --- /dev/null +++ b/test/openai-compat-multimodal.test.ts @@ -0,0 +1,248 @@ +// v0.34.1 (#875): multimodal embedding for openai-compatible recipes via +// LiteLLM (or any other openai-compatible proxy). Sibling to +// voyage-multimodal.test.ts; covers the new embedMultimodalOpenAICompat +// path including D12 dim validation. + +import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; +import { configureGateway, embedMultimodal, resetGateway } from '../src/core/ai/gateway.ts'; +import { AIConfigError, AITransientError } from '../src/core/ai/errors.ts'; + +type FetchHandler = (url: string, init: RequestInit) => Promise; +let fetchHandler: FetchHandler | null = null; +const origFetch = globalThis.fetch; + +beforeEach(() => { + fetchHandler = null; + globalThis.fetch = (async (url: string | URL | Request, init?: RequestInit) => { + if (!fetchHandler) { + throw new Error('fetch called but no handler installed'); + } + return fetchHandler(typeof url === 'string' ? url : url.toString(), init ?? {}); + }) as typeof fetch; +}); + +afterEach(() => { + globalThis.fetch = origFetch; + resetGateway(); +}); + +function configureLitellm(env: Record = {}, dims = 1024) { + configureGateway({ + embedding_model: 'litellm:gpt-4o-multimodal', + embedding_dimensions: dims, + env: { + LITELLM_API_KEY: 'test-litellm-key', + LITELLM_BASE_URL: 'http://localhost:4000', + ...env, + }, + base_urls: { litellm: 'http://localhost:4000' }, + }); +} + +function okResponse(dims: number, count: number = 1): Response { + const vec = Array(dims).fill(0).map((_, i) => 0.001 * i); + return new Response( + JSON.stringify({ data: Array.from({ length: count }, () => ({ embedding: vec })) }), + { status: 200, headers: { 'Content-Type': 'application/json' } }, + ); +} + +describe('embedMultimodal — openai-compat routing (#875)', () => { + test('LiteLLM recipe accepts a single image input and returns one embedding', async () => { + configureLitellm(); + let capturedUrl = ''; + let capturedBody: any = null; + let capturedAuth = ''; + fetchHandler = async (url, init) => { + capturedUrl = url; + capturedAuth = (init.headers as Record).Authorization ?? ''; + capturedBody = JSON.parse(init.body as string); + return okResponse(1024, 1); + }; + + const result = await embedMultimodal([ + { kind: 'image_base64', data: 'fake-base64-bytes', mime: 'image/png' }, + ]); + + expect(result.length).toBe(1); + expect(result[0].length).toBe(1024); + expect(capturedUrl).toBe('http://localhost:4000/embeddings'); + expect(capturedAuth).toBe('Bearer test-litellm-key'); + expect(capturedBody.model).toBe('gpt-4o-multimodal'); + expect(capturedBody.input[0].type).toBe('image_url'); + expect(capturedBody.input[0].image_url.url).toBe('data:image/png;base64,fake-base64-bytes'); + }); + + test('multiple inputs trigger sequential /embeddings calls', async () => { + configureLitellm(); + let calls = 0; + fetchHandler = async () => { + calls += 1; + return okResponse(1024, 1); + }; + + const result = await embedMultimodal([ + { kind: 'image_base64', data: 'img1', mime: 'image/jpeg' }, + { kind: 'image_base64', data: 'img2', mime: 'image/png' }, + { kind: 'image_base64', data: 'img3', mime: 'image/webp' }, + ]); + + expect(calls).toBe(3); + expect(result.length).toBe(3); + }); + + test('LiteLLM without LITELLM_API_KEY still works (proxy may run unauthenticated)', async () => { + configureGateway({ + embedding_model: 'litellm:multimodal-foo', + embedding_dimensions: 768, + env: { LITELLM_BASE_URL: 'http://localhost:4000' }, // no API key + base_urls: { litellm: 'http://localhost:4000' }, + }); + let capturedAuth: string | null | undefined; + fetchHandler = async (_url, init) => { + capturedAuth = (init.headers as Record).Authorization; + return okResponse(768, 1); + }; + const result = await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + expect(result.length).toBe(1); + // defaultResolveAuth sends 'Bearer unauthenticated' when no api key is + // configured — servers like Ollama / llama-server ignore the value but + // the SDK contract still requires SOME Authorization header. + expect(capturedAuth).toBe('Bearer unauthenticated'); + }); + + test('D12 — provider returns wrong-dim vector throws AIConfigError', async () => { + // Brain configured for 1024; provider returns 768. D12 catches the + // mismatch BEFORE the vector lands in the DB column. + configureLitellm({}, 1024); + fetchHandler = async () => okResponse(768, 1); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AIConfigError); + expect((caught as Error).message).toContain('768-dim vector'); + expect((caught as Error).message).toContain('expected 1024'); + expect((caught as Error).message).toContain('gpt-4o-multimodal'); + }); + + test('D12 — default embedding_dimensions (1536) applies when not explicitly set', async () => { + // configureGateway normalizes embedding_dimensions to 1536 when unset + // (the DEFAULT_EMBEDDING_DIMENSIONS). LiteLLM recipe's default_dims=0 + // so we fall back to the brain's configured value. This test pins the + // "always validate via the configured/default dim" contract — there + // is no skip-when-unset path in practice because configureGateway + // always populates it. + configureGateway({ + embedding_model: 'litellm:any-model', + // intentionally NO embedding_dimensions → falls back to 1536 + env: { LITELLM_BASE_URL: 'http://localhost:4000' }, + base_urls: { litellm: 'http://localhost:4000' }, + }); + fetchHandler = async () => okResponse(1536, 1); + const result = await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + expect(result.length).toBe(1); + expect(result[0].length).toBe(1536); + }); + + test('provider returns 401 → AIConfigError with model id in message', async () => { + configureLitellm(); + fetchHandler = async () => + new Response('invalid key', { status: 401, headers: { 'Content-Type': 'text/plain' } }); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AIConfigError); + expect((caught as Error).message).toContain('401'); + }); + + test('provider returns 400 (model does not support multimodal) → AITransientError surfaces body', async () => { + configureLitellm(); + fetchHandler = async () => + new Response('model does not support image inputs', { status: 400 }); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AITransientError); + expect((caught as Error).message).toContain('400'); + expect((caught as Error).message).toContain('model does not support image inputs'); + }); + + test('malformed JSON response → AITransientError', async () => { + configureLitellm(); + fetchHandler = async () => + new Response('not json', { status: 200, headers: { 'Content-Type': 'application/json' } }); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AITransientError); + expect((caught as Error).message).toContain('malformed JSON'); + }); + + test('non-array embedding payload → AITransientError', async () => { + configureLitellm(); + fetchHandler = async () => + new Response(JSON.stringify({ data: [{ embedding: 'not-array' }] }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AITransientError); + expect((caught as Error).message).toContain('non-array'); + }); + + test('empty data array → AITransientError', async () => { + configureLitellm(); + fetchHandler = async () => + new Response(JSON.stringify({ data: [] }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AITransientError); + }); + + test('Voyage recipe still routes to /multimodalembeddings (regression)', async () => { + // Ensure the new openai-compat route doesn't accidentally hijack Voyage. + configureGateway({ + embedding_model: 'voyage:voyage-multimodal-3', + embedding_dimensions: 1024, + env: { VOYAGE_API_KEY: 'voyage-key' }, + }); + let capturedUrl = ''; + fetchHandler = async (url) => { + capturedUrl = url; + return okResponse(1024, 1); + }; + + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + expect(capturedUrl).toContain('/multimodalembeddings'); + }); +}); diff --git a/test/serve-stdio-lifecycle.test.ts b/test/serve-stdio-lifecycle.test.ts index e548b01..e250ad2 100644 --- a/test/serve-stdio-lifecycle.test.ts +++ b/test/serve-stdio-lifecycle.test.ts @@ -81,6 +81,7 @@ function makeHarness(opts: { isTTY?: boolean; initialParentPid?: number; probeWatchdog?: boolean; + mcpStdio?: boolean; } = {}): Harness { const engine = new StubEngine(); const stdin = new EventEmitter() as EventEmitter & { isTTY?: boolean }; @@ -120,6 +121,7 @@ function makeHarness(opts: { setInterval: timers.setInterval, clearInterval: timers.clearInterval, probeWatchdog: () => probeWatchdogResult, + mcpStdio: opts.mcpStdio, }; return { @@ -439,4 +441,63 @@ describe('runServe stdio lifecycle', () => { expect(code).toBe(0); expect(h.logs.some(l => l.includes('cleanup error: synthetic disconnect failure'))).toBe(true); }); + + // v0.34.1 (#870): OpenClaw gateway / bundle-mcp wrappers pipe the + // JSON-RPC handshake on stdin then close their stdin half. Without + // MCP_STDIO=1 the server treats that as a permanent disconnect and + // exits before handling tools/call. The guard skips the stdin 'end' / + // 'close' hooks when MCP_STDIO=1; signals and parent watchdog still + // cover legitimate shutdown. + describe('MCP_STDIO=1 piped-stdin guard (#870)', () => { + test('stdin end with mcpStdio=true does NOT trigger shutdown', async () => { + const h = makeHarness({ mcpStdio: true }); + await startInBackground(h.engine, [], h.opts); + + // Without the guard this would shutdown; with the guard it must not. + h.stdin.emit('end'); + + // Give the event loop a microtask turn to catch any erroneous shutdown + // path. We assert NO exit was registered. + await new Promise((r) => setTimeout(r, 10)); + expect(h.engine.disconnectCalls).toBe(0); + + // Then trigger SIGTERM to drive the test to completion; signal handlers + // remain active even with mcpStdio=true (codex would catch if they didn't). + h.signals.emit('SIGTERM'); + const code = await h.exited; + expect(code).toBe(0); + expect(h.engine.disconnectCalls).toBe(1); + expect(h.logs.some(l => l.includes('graceful exit (SIGTERM)'))).toBe(true); + }); + + test('stdin close with mcpStdio=true does NOT trigger shutdown', async () => { + const h = makeHarness({ mcpStdio: true }); + await startInBackground(h.engine, [], h.opts); + + h.stdin.emit('close'); + + await new Promise((r) => setTimeout(r, 10)); + expect(h.engine.disconnectCalls).toBe(0); + + h.signals.emit('SIGINT'); + const code = await h.exited; + expect(code).toBe(0); + expect(h.engine.disconnectCalls).toBe(1); + }); + + test('mcpStdio=false (default) preserves stdin EOF shutdown', async () => { + // Regression guard: the guard must not over-trigger. With the env + // unset, stdin EOF must still drive shutdown so existing CLI usage + // (gbrain serve under launchd, claude-desktop's stdio MCP) is + // unchanged. + const h = makeHarness({ mcpStdio: false }); + await startInBackground(h.engine, [], h.opts); + + h.stdin.emit('end'); + const code = await h.exited; + expect(code).toBe(0); + expect(h.engine.disconnectCalls).toBe(1); + expect(h.logs.some(l => l.includes('graceful exit (stdin-end)'))).toBe(true); + }); + }); });