diff --git a/CHANGELOG.md b/CHANGELOG.md index 74257b4..644ad5e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,245 @@ All notable changes to GBrain will be documented in this file. +## [0.34.1.0] - 2026-05-14 + +**MCP hardening wave: stricter source isolation on the read path, PKCE +DCR works, loopback-by-default, and federated read scopes for shared +brains. Six community PRs land as one release.** + +The v0.34.1 wave consolidates six community PRs into a single ship. +Source-isolation tightening is the centerpiece — an authenticated OAuth +client scoped to one source no longer sees rows from neighboring sources +through the read path. The wave also seals smaller papercuts that have +been costing real users: gateway-managed stdio MCP servers no longer +exit on the post-handshake EOF, PKCE-only DCR clients can register +without inheriting a phantom secret, and `gbrain serve --http` defaults +to loopback so a personal-laptop brain isn't one accidental config away +from publishing itself to the LAN. Federated_read adds the read-scope +axis that shared-brain deployments need: a department client can write +to one source and read across a curated set without becoming a +super-reader. + +### What you can now do + +**Scope OAuth clients to a single source.** `gbrain auth register-client +my-agent --source dept-x` registers a client whose write authority is +`dept-x`. Read paths only return rows whose `source_id` matches. The +auth-layer thread is on `verifyAccessToken` so every per-request +dispatch starts with the scope in `AuthInfo.sourceId`; ops consume it +through the canonical `ctx.sourceId` pattern that v0.31.8 established. +Pre-v0.34 clients without an explicit source default to `default` on +upgrade — the v0.33 effective behavior is preserved verbatim. + +**Federate read scope independently.** `gbrain auth register-client +my-l3-dept --source dept-x --federated-read dept-x,wecare,shared` +registers a client that writes to `dept-x` but reads from the union of +`dept-x`, `wecare` parent canon, and `shared` org canon. The two scope +axes are orthogonal — `source_id` is the write authority, `federated_read` +is the read authority. Engine read paths apply +`WHERE source_id = ANY($1::text[])` at SQL when the array is set; scalar +single-source clients keep the v0.31.12 fast path. + +**Run gateway-piped stdio MCP without race-killing your server.** Set +`MCP_STDIO=1` in the env and the server skips the stdin EOF shutdown +hooks. Signal handlers (SIGTERM / SIGINT / SIGHUP) and parent-process +watchdog still cover legitimate disconnects. OpenClaw's bundle-mcp +gateway and similar wrappers pipe the handshake then close their stdin +half; pre-fix this killed the server before the first tool call landed. + +**Register PKCE-only public clients.** `POST /register` with +`token_endpoint_auth_method: "none"` (Claude Code, Cursor, every other +PKCE-first MCP client) now returns RFC 7591-compliant response shape: +no `client_secret` field on public clients, secret-bearing on default +`client_secret_post` clients. The `/token` exchange accepts the public +client through the SDK's clientAuth path because `getClient` correctly +normalizes a NULL `client_secret_hash` to JS undefined. + +**Bind the HTTP MCP server to loopback by default.** `gbrain serve +--http` now listens on `127.0.0.1` unless you pass `--bind 0.0.0.0` (or +a specific interface IP). Personal-laptop installs are no longer one +default-config away from publishing the brain to a LAN. Self-hosted +server operators who actually want remote access pass `--bind 0.0.0.0` +once; a stderr WARN fires when `--public-url` is set without `--bind` so +the operator doesn't silently bind loopback at startup. + +**Embed images through LiteLLM / openai-compatible multimodal models.** +The gateway's `embedMultimodal` no longer hardcodes Voyage; recipes with +`implementation: 'openai-compatible'` route through the standard +`/embeddings` endpoint with content arrays carrying `image_url` entries. +Runtime dimension validation throws a clear error pre-storage if the +provider returns a vector that doesn't match the brain's embedding +column width — no more cryptic `vector dimension mismatch` at INSERT +time. + +### The migration block + +`gbrain upgrade` applies migrations v60-v65 automatically. The migration +chain adds two columns to `oauth_clients` and an index, plus a FK flip +once federated_read is in place. + +### To take advantage of v0.34.1 + +`gbrain upgrade` should do this automatically. If it didn't, or if +`gbrain doctor` warns about a partial migration: + +1. **Run the orchestrator manually:** + ```bash + gbrain apply-migrations --yes + ``` +2. **Verify the schema landed:** + ```bash + gbrain doctor --json | jq '.checks.schema_version' + # version should be >= 60 + ``` +3. **Existing OAuth clients keep working.** Pre-v0.34 clients without an + explicit source scope are backfilled to `source_id='default'` so + their effective scope matches v0.33. To narrow a specific client's + scope, re-register it with `--source `: + ```bash + gbrain auth revoke-client + gbrain auth register-client --source --scopes read,write + ``` +4. **`gbrain serve --http` default changed.** Existing self-hosted + server deployments must add `--bind 0.0.0.0` to keep accepting remote + connections. Personal-laptop users see no behavior change (loopback + is now the default). +5. **If `gbrain apply-migrations` refuses with an `oauth_clients` stale + source_id error** (possible only if an operator hand-poked the column + before upgrading): the error message names the offending client IDs. + Either revoke + re-register them with a valid source, or re-run with + `GBRAIN_ACCEPT_SILENT_WIDEN=1` to NULL the stale values (widens those + clients to super-reader; re-scope via `gbrain auth register-client` + after). +6. **If any step fails or the numbers look wrong,** file an issue: + https://github.com/garrytan/gbrain/issues with `gbrain doctor --json` + output and the failing step. + +### Itemized changes + +**Source-isolation hardening:** +- `OperationContext.auth` now carries `AuthInfo.sourceId` (write scope) + and `AuthInfo.allowedSources` (federated read scope), threaded from + `oauth_clients` rows at token-verification time. +- New helper `sourceScopeOpts(ctx)` in `src/core/operations.ts` encodes + the precedence ladder: federated array wins over scalar over nothing. + Every read-side op handler routes through it so future ops can't + silently drift from the canonical thread. +- `src/core/search/hybrid.ts` inner `SearchOpts` rebuild now includes + `sourceId` + `sourceIds` fields — the structural fix that prevents + the explicit-pick footgun that motivated the wave. +- `src/core/types.ts` adds `sourceIds?: string[]` to `SearchOpts` and + `PageFilters` for the federated read axis. Both Postgres and PGLite + engines apply `WHERE source_id = ANY($N::text[])` when the array is + set; scalar fast path preserved when unset. +- Engine method signatures `traverseGraph(slug, depth, opts?)` and + `traversePaths(slug, opts?)` accept `opts.sourceId` / + `opts.sourceIds` so graph walks respect the caller's scope. +- `src/core/oauth-provider.ts:verifyAccessToken` JOINs + `oauth_clients.source_id` + `federated_read` and surfaces both on the + returned `AuthInfo`. Pre-v60 / pre-v61 brains degrade gracefully via + `isUndefinedColumnError` fallback. +- `src/commands/serve-http.ts` drops the `(authInfo as AuthInfo & + {sourceId?: string}).sourceId ?? env ?? 'default'` cast chain. The + typed field is the source of truth now. +- Legacy `src/mcp/http-transport.ts` (v0.22.7-style access_tokens path) + threads `sourceId: 'default'` through DispatchOpts so legacy tokens + stay source-scoped. + +**Migration chain v60-v65 (six new migrations on top of v54):** +- v60 (`oauth_clients_source_id_fk`) — ALTER TABLE ... ADD COLUMN + source_id TEXT, backfill NULL→'default', install FK with + ON DELETE SET NULL. +- v61 (`oauth_clients_federated_read_column`) — ALTER TABLE ... ADD + COLUMN federated_read TEXT[] NOT NULL DEFAULT '{}'. +- v62 (`oauth_clients_federated_read_backfill`) — explicit CASE backfill + so `source_id IS NULL` produces `'{}'` not an array-containing-NULL. +- v63 (`oauth_clients_federated_read_validate`) — fail-loud check that + every row's source_id is in its federated_read array post-backfill. +- v64 (`oauth_clients_source_id_fk_restrict`) — flip FK to + ON DELETE RESTRICT now that federated_read provides the alternative + scope-loss path. Source delete is refused if any client references it. +- v65 (`oauth_clients_federated_read_gin_index`) — GIN index for the + array-containment queries the read paths run. + +**OAuth + auth surface:** +- `auth.ts` CLI adds `--source ` and `--federated-read ` + flags to `register-client`. The output now prints the resolved + `Write source` and `Federated reads` for the registered client. +- DCR `/register` endpoint now writes `source_id='default'` and + `federated_read=['default']` on the inserted row so new public clients + start in a sane scope. +- `registerClient` honors `token_endpoint_auth_method: "none"` (RFC 7591 + §3.2.1): public clients store `client_secret_hash = NULL` and the + response payload omits `client_secret` entirely. Confidential clients + (default `client_secret_post` and explicit `client_secret_basic`) keep + their one-time-reveal shape. + +**MCP transports:** +- `src/mcp/server.ts` + `src/commands/serve.ts` skip stdin + `'end'`/`'close'` shutdown hooks when `process.env.MCP_STDIO === '1'`. + `ServeOptions` gains a `mcpStdio?: boolean` test seam so the runtime + guard is exercisable without process.env mutation. +- `src/commands/serve-http.ts` adds `--bind HOST` (default `127.0.0.1`). + Stderr WARN fires when `--public-url` is set without `--bind`. + Startup banner prints the resolved `Bind:` line. + +**Multimodal embedding:** +- `gateway.ts` adds `embedMultimodalOpenAICompat()` that POSTs to the + standard `/embeddings` endpoint with content arrays. Routes by + `recipe.implementation === 'openai-compatible'` so LiteLLM-fronted + providers (Anyscale, vLLM, Gemini multimodal via proxy) work alongside + Voyage's existing `/multimodalembeddings` path. +- `recipes/litellm-proxy.ts` declares `supports_multimodal: true` so the + recipe accepts multimodal calls without a model allow-list (LiteLLM is + a passthrough; user-owned model id selection). +- Runtime dimension validation: the returned vector length is checked + against the recipe's declared `default_dims` or the brain's + `embedding_dimensions` config. Mismatch throws `AIConfigError` with + model id + observed + expected before the vector reaches storage. + +**Tests:** +- `test/e2e/source-isolation-pglite.test.ts` — 14 cases pinning the + scope filter at the engine layer plus op-handler threading for both + `ctx.sourceId` and `ctx.auth.allowedSources` paths. +- `test/openai-compat-multimodal.test.ts` — 11 cases covering the + openai-compatible multimodal path: happy-path single + multi-input, + unauthenticated proxy, D12 dim mismatch + default-dim fallback, + 401 / 400 / malformed-JSON / non-array error paths, and a Voyage + regression test. +- `test/oauth.test.ts` — 5 new cases for PKCE DCR public-client gate + (no secret for public; secret unchanged for default; getClient + NULL→undefined normalization; full PKCE `/authorize` → `/token` + round-trip). +- `test/serve-stdio-lifecycle.test.ts` — 3 new cases for the + `MCP_STDIO=1` guard (stdin EOF does NOT trigger shutdown; SIGTERM + still does; unset env preserves CLI behavior). +- `test/book-mirror.test.ts` — `runCli` helper now uses a fresh + `GBRAIN_HOME` tempdir so the test isn't sensitive to the developer's + local `~/.gbrain/config.json`. Pre-fix this could silently inherit a + real Postgres connection and hang past the default 5s test timeout. + +**Test results: 6045 unit tests pass / 0 fail. typecheck clean. PGLite +initSchema runs the v60-v65 chain in ~786ms total.** + +### For contributors + +- The wave bundled six community PRs (#870, #909, #864, #861, #875, + #876) with cross-model plan review (codex outside-voice) before + cherry-pick. Codex caught three structural bugs the original PRs + missed: a 6th source-isolation leak surface (the `query` image path + in `operations.ts:1071-1082`), a 5th read-side op missing from #861's + thread (`find_experts`), and migration-numbering collisions on + v47-v53 that would have wedged on cherry-pick (the branch had grown + to v57 by ship time after master shipped its own v55-v57 search-lite + migrations). The wave's migration chain renumbers to v60-v65. +- The single PR carries `Co-Authored-By:` for the six external + contributors. PRs originated against earlier base branches and the + diffs were re-implemented on the collector branch rather than merged + directly (per CLAUDE.md's PR wave process). + +Contributed by @Hansen1018 (#870), @ding-modding (#909), @DukeDawg +(#864), @toilalesondev (#861 + #876), @yoelgal (#875). ## [0.34.0.0] - 2026-05-14 **Recursive code intelligence ships. Plan-mode subagents get one-call blast and flow.** diff --git a/CLAUDE.md b/CLAUDE.md index 5c6ec36..4269771 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -40,8 +40,8 @@ strict behavior when unset. ## Key files -- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (v0.23 glob matcher: `/*` matches recursive children; bare `` matches exact only). `OperationContext.remote` flags untrusted callers; `OperationContext.allowedSlugPrefixes` (v0.23) is the trusted-workspace allow-list set by the dream cycle. `put_page` enforces: when `viaSubagent` and `allowedSlugPrefixes` is set, slug must match the allow-list; else the legacy `wiki/agents//...` namespace check applies. Auto-link enabled for trusted-workspace writes (skipped only when `remote=true && !trustedWorkspace`). As of v0.26.0, every `Operation` also carries `scope?: 'read' | 'write' | 'admin'` + `localOnly?: boolean`. All ops are annotated; `sync_brain`, `file_upload`, `file_list`, and `file_url` are `admin + localOnly` (rejected over HTTP). `OperationContext.auth?: AuthInfo` is threaded through HTTP dispatch for scope enforcement in `serve-http.ts` before the op runs. **v0.26.9 (D12 + F7b):** `OperationContext.remote` is now a REQUIRED field in the TypeScript type — the compiler is the first defense against transports that forget to set it. Four trust-boundary call sites (`put_page` allowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) flipped from falsy-default (`!ctx.remote`) to fail-closed semantics (`ctx.remote === false` for "trusted-only" sites and `ctx.remote !== false` for "untrust unless explicit-false"). Anything that isn't strictly `false` is now treated as remote. Closed an HTTP MCP shell-job RCE: a `read+write`-scoped OAuth token could submit `shell` jobs because the HTTP request handler's literal context skipped `remote: true` and `submit_job`'s protected-name guard saw a falsy undefined. Stdio MCP set the field correctly via dispatch.ts; HTTP inlined a parallel context-builder for several releases and lost it. -- `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput` / `TimelineBatchInput` for the v0.12.1 bulk-insert API (`addLinksBatch` / `addTimelineEntriesBatch`). As of v0.13.1, `BrainEngine` has a `readonly kind: 'postgres' | 'pglite'` discriminator so migrations (`src/core/migrate.ts`) and other consumers can branch on engine without `instanceof` + dynamic imports. **v0.29:** four new methods — `batchLoadEmotionalInputs(slugs?)` (CTE-shaped read with per-table aggregates so a page × N tags × M takes never produces N×M rows), `setEmotionalWeightBatch(rows)` (`UPDATE FROM unnest($1::text[], $2::text[], $3::real[])` composite-keyed on `(slug, source_id)` for multi-source safety), `getRecentSalience(opts)`, `findAnomalies(opts)`. `PageFilters` extended with `sort?: 'updated_desc' | 'updated_asc' | 'created_desc' | 'slug'` + `PAGE_SORT_SQL` whitelist consumed by both engines (was hardcoded `ORDER BY updated_at DESC`). **v0.32.8 (PR #860):** new `listAllPageRefs(): Promise>` ordered by `(source_id, slug)`. Cheap cross-source enumeration for hot loops on large brains — replaces the `getAllSlugs()→getPage(slug)` N+1 pattern in extract-takes, extract, integrity, which silently defaulted to `source_id='default'` for non-default-source pages. Implementation parity across postgres-engine.ts + pglite-engine.ts. Pinned by `test/e2e/multi-source-bug-class.test.ts`. +- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (v0.23 glob matcher: `/*` matches recursive children; bare `` matches exact only). `OperationContext.remote` flags untrusted callers; `OperationContext.allowedSlugPrefixes` (v0.23) is the trusted-workspace allow-list set by the dream cycle. `put_page` enforces: when `viaSubagent` and `allowedSlugPrefixes` is set, slug must match the allow-list; else the legacy `wiki/agents//...` namespace check applies. Auto-link enabled for trusted-workspace writes (skipped only when `remote=true && !trustedWorkspace`). As of v0.26.0, every `Operation` also carries `scope?: 'read' | 'write' | 'admin'` + `localOnly?: boolean`. All ops are annotated; `sync_brain`, `file_upload`, `file_list`, and `file_url` are `admin + localOnly` (rejected over HTTP). `OperationContext.auth?: AuthInfo` is threaded through HTTP dispatch for scope enforcement in `serve-http.ts` before the op runs. **v0.26.9 (D12 + F7b):** `OperationContext.remote` is now a REQUIRED field in the TypeScript type — the compiler is the first defense against transports that forget to set it. Four trust-boundary call sites (`put_page` allowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) flipped from falsy-default (`!ctx.remote`) to fail-closed semantics (`ctx.remote === false` for "trusted-only" sites and `ctx.remote !== false` for "untrust unless explicit-false"). Anything that isn't strictly `false` is now treated as remote. Closed an HTTP MCP shell-job RCE: a `read+write`-scoped OAuth token could submit `shell` jobs because the HTTP request handler's literal context skipped `remote: true` and `submit_job`'s protected-name guard saw a falsy undefined. Stdio MCP set the field correctly via dispatch.ts; HTTP inlined a parallel context-builder for several releases and lost it. **v0.34.1.0 (#861 + #876):** new helper `sourceScopeOpts(ctx)` encodes the precedence ladder for source-scoped reads — federated array (`ctx.auth.allowedSources`) wins over scalar (`ctx.sourceId` / `ctx.auth.sourceId`) over nothing. Every read-side op handler routes through it so future ops can't silently drift from the canonical v0.31.8 thread. Closes the source-isolation leak on the read path: a `read+write`-scoped OAuth client bound to `--source dept-x` no longer sees rows from neighboring sources via `search` / `query` / `list_pages` / `get_page` / `find_experts` / `query`'s image path. +- `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput` / `TimelineBatchInput` for the v0.12.1 bulk-insert API (`addLinksBatch` / `addTimelineEntriesBatch`). As of v0.13.1, `BrainEngine` has a `readonly kind: 'postgres' | 'pglite'` discriminator so migrations (`src/core/migrate.ts`) and other consumers can branch on engine without `instanceof` + dynamic imports. **v0.29:** four new methods — `batchLoadEmotionalInputs(slugs?)` (CTE-shaped read with per-table aggregates so a page × N tags × M takes never produces N×M rows), `setEmotionalWeightBatch(rows)` (`UPDATE FROM unnest($1::text[], $2::text[], $3::real[])` composite-keyed on `(slug, source_id)` for multi-source safety), `getRecentSalience(opts)`, `findAnomalies(opts)`. `PageFilters` extended with `sort?: 'updated_desc' | 'updated_asc' | 'created_desc' | 'slug'` + `PAGE_SORT_SQL` whitelist consumed by both engines (was hardcoded `ORDER BY updated_at DESC`). **v0.32.8 (PR #860):** new `listAllPageRefs(): Promise>` ordered by `(source_id, slug)`. Cheap cross-source enumeration for hot loops on large brains — replaces the `getAllSlugs()→getPage(slug)` N+1 pattern in extract-takes, extract, integrity, which silently defaulted to `source_id='default'` for non-default-source pages. Implementation parity across postgres-engine.ts + pglite-engine.ts. Pinned by `test/e2e/multi-source-bug-class.test.ts`. **v0.34.1.0 (#861):** `SearchOpts` + `PageFilters` add `sourceIds?: string[]` for the federated read axis; both engines apply `WHERE source_id = ANY($N::text[])` when the array is set and preserve the scalar `sourceId` fast path when unset. `traverseGraph(slug, depth, opts?)` and `traversePaths(slug, opts?)` accept `opts.sourceId` / `opts.sourceIds` so graph walks respect the caller's scope. - `src/core/engine-factory.ts` — Engine factory with dynamic imports (`'pglite'` | `'postgres'`) - `src/core/pglite-engine.ts` — PGLite (embedded Postgres 17.5 via WASM) implementation, all 40 BrainEngine methods. `addLinksBatch` / `addTimelineEntriesBatch` use multi-row `unnest()` with manual `$N` placeholders. As of v0.13.1, `connect()` wraps `PGlite.create()` in a try/catch that emits an actionable error naming the macOS 26.3 WASM bug (#223) and pointing at `gbrain doctor`; the lock is released on failure so the next process can retry cleanly. v0.22.0: `searchKeyword` and `searchKeywordChunks` multiply `ts_rank` by the source-factor CASE expression at the chunk-grain level; `searchVector` becomes a two-stage CTE — inner CTE keeps `ORDER BY cc.embedding <=> vec` so HNSW stays usable, outer SELECT re-ranks by `raw_score * source_factor`. Inner LIMIT scales with offset to preserve pagination contract. As of v0.22.6.1, `initSchema()` calls `applyForwardReferenceBootstrap()` BEFORE replaying SCHEMA_SQL — probes for the specific forward-referenced state the embedded schema blob needs (`pages.source_id`, `links.link_source`, `links.origin_page_id`, `content_chunks.symbol_name`, `content_chunks.language`, `sources` FK target table) and adds only what's missing. Closes the upgrade-wedge bug class that bit users 10+ times across 6 schema versions over 2 years (#239/#243/#266/#357/#366/#374/#375/#378/#395/#396). No-op on fresh installs and modern brains. - `src/core/pglite-schema.ts` — PGLite-specific DDL (pgvector, pg_trgm, triggers) @@ -94,7 +94,7 @@ strict behavior when unset. - `src/core/embedding.ts` — OpenAI text-embedding-3-large, batch, retry, backoff. **v0.28.7:** `BATCH_SIZE` reverted 50→100 — the original Voyage safety guard halved OpenAI throughput on every page. Per-recipe pre-split + recursive halving + adaptive shrink-on-miss now live in the gateway, so the outer paginator goes back to its original purpose: progress-callback granularity, not batch protection. - `src/core/ai/dims.ts` (v0.33.1.1, PR #962 + #866) — per-provider `providerOptions` resolver for embed-time dimension passthrough. The single source of truth for "which provider needs which knob to produce `vector(N)`". Exports `dimsProviderOptions(implementation, modelId, dims)` (called by `embed()` in `gateway.ts`), `VOYAGE_OUTPUT_DIMENSION_MODELS` (private const — the 7 hosted Voyage models that accept `output_dimension`: `voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-3-large`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-code-3` — nano deliberately excluded), `VOYAGE_VALID_OUTPUT_DIMS = [256, 512, 1024, 2048] as const`, `supportsVoyageOutputDimension(modelId)`, and `isValidVoyageOutputDim(dims)`. **Voyage path uses the SDK-supported `dimensions` field** (`{ openaiCompatible: { dimensions: N } }`), NOT Voyage's `output_dimension` wire-key — the existing `voyageCompatFetch` shim in `gateway.ts:541` translates `dimensions → output_dimension` before the HTTP body is built. The reverse (sending `output_dimension` from here) was the v0.33.1.0 bug class: the AI SDK's openai-compatible adapter doesn't recognize the wire-key so it was silently dropped, Voyage returned its default 1024-dim, and the gateway dimension check threw on every embed call. Runtime guard: when a Voyage flexible-dim model is configured with `dims` outside `VOYAGE_VALID_OUTPUT_DIMS`, throws `AIConfigError` with a paste-ready `gbrain config set embedding_dimensions <256|512|1024|2048>` hint at the embed boundary — fail-loud instead of opaque Voyage HTTP 400. Most common trigger: `embedding_model: voyage:voyage-4-large` configured without `embedding_dimensions` (falls back to `DEFAULT_EMBEDDING_DIMENSIONS=1536`, an OpenAI default not a Voyage one). Eva (@100yenadmin) shipped the wire-key fix in #866; Codex P3 follow-up landed the validator + valid-dims allowlist in #962. - `src/core/ai/types.ts` — provider/recipe types. **v0.28.7 (#680):** `EmbeddingTouchpoint` extended with optional `chars_per_token` (default 4 chars/token, matching OpenAI tiktoken on English) and `safety_factor` (default 0.8, budget-utilization ceiling). Both consulted only when `max_batch_tokens` is also set. Voyage declares `chars_per_token=1` + `safety_factor=0.5` to handle dense payloads (CJK/JSON/base64) that overshoot tiktoken. The pre-split budget is `max_batch_tokens × safety_factor / chars_per_token`. **v0.28.11 (#719):** `EmbeddingTouchpoint.multimodal_models?: string[]` model-level allow-list for recipes that mix text-only + multimodal models under one touchpoint (Voyage's 12 models share `supports_multimodal: true` but only `voyage-multimodal-3` accepts `/multimodalembeddings`). When omitted, recipe-level `supports_multimodal` is sufficient. `AIGatewayConfig.embedding_multimodal_model?: string` lets `embedMultimodal()` route to a different model than `embedding_model` — brains using OpenAI for text can use Voyage for images without flipping the primary embedding pipeline. -- `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state. **v0.33.1.1 (#962, Codex P3 follow-up):** new exported `VoyageResponseTooLargeError` tagged class at the top of the file. `voyageCompatFetch`'s two OOM-defense caps (Layer 1 Content-Length check at `:595`, Layer 2 per-embedding base64 cap at `:619`) now throw `VoyageResponseTooLargeError` instead of a generic `Error`. The inbound response-rewriter's surrounding try/catch (which intentionally swallows parse failures so misshaped Voyage responses fall through to the SDK's JSON parser) checks `instanceof VoyageResponseTooLargeError` and rethrows. Pre-fix, the Layer 2 throw was silently swallowed and the oversized response returned to the AI SDK anyway — Layer 2 was theatrical. Source-shape regression assertion in `test/voyage-response-cap.test.ts` pins the `instanceof ⇒ throw err` line. +- `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state. **v0.33.1.1 (#962, Codex P3 follow-up):** new exported `VoyageResponseTooLargeError` tagged class at the top of the file. `voyageCompatFetch`'s two OOM-defense caps (Layer 1 Content-Length check at `:595`, Layer 2 per-embedding base64 cap at `:619`) now throw `VoyageResponseTooLargeError` instead of a generic `Error`. The inbound response-rewriter's surrounding try/catch (which intentionally swallows parse failures so misshaped Voyage responses fall through to the SDK's JSON parser) checks `instanceof VoyageResponseTooLargeError` and rethrows. Pre-fix, the Layer 2 throw was silently swallowed and the oversized response returned to the AI SDK anyway — Layer 2 was theatrical. Source-shape regression assertion in `test/voyage-response-cap.test.ts` pins the `instanceof ⇒ throw err` line. **v0.34.1.0 (#875):** new `embedMultimodalOpenAICompat()` routes recipes with `implementation: 'openai-compatible'` (LiteLLM, Anyscale, vLLM, Gemini multimodal via proxy) through the standard `/embeddings` endpoint with content arrays carrying `image_url` entries. The pre-existing Voyage `/multimodalembeddings` path is unchanged; the gateway selects by recipe `implementation` tag. Runtime dimension validation throws `AIConfigError` (with model id + observed + expected) before the vector reaches storage when the provider returns a width that doesn't match the recipe's `default_dims` or the brain's `embedding_dimensions` config — no more cryptic `vector dimension mismatch` at INSERT time. Pinned by 11 cases in `test/openai-compat-multimodal.test.ts`. - `src/core/ai/recipes/voyage.ts` — Voyage AI openai-compatible recipe. **v0.28.7 (#680):** declares `chars_per_token=1` + `safety_factor=0.5` so the gateway pre-splits Voyage batches at a 60K-character budget (50% of 120K-token cap with the dense-tokenizer ratio). Closes the v0.27 backfill loop where ~26% of the corpus stayed un-embedded because tiktoken-grounded budgeting silently undercounted Voyage's actual token usage. **v0.28.11 (#719):** declares `multimodal_models: ['voyage-multimodal-3']` so the gateway rejects text-only Voyage models pointed at the multimodal endpoint with a clear `AIConfigError` instead of waiting for Voyage's HTTP 400. **v0.33.1.1 (#962, fixup):** recipe docstring at `:7-16` tightened to name the seven hosted flexible-dim models that accept `output_dimension` explicitly (`voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-3-large`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-code-3`) and call out that `voyage-4-nano` is the open-weight variant listed separately by Voyage as fixed 1024-dim — does NOT accept the parameter. The "all v4 variants are flexible" misread is what caused the original PR to include nano in `VOYAGE_OUTPUT_DIMENSION_MODELS`; the negative regression assertion in `test/ai/gateway.test.ts` (`dimsProviderOptions` returns `undefined` for `voyage-4-nano`) pins the contract. - `src/core/ai/recipes/anthropic.ts` — Anthropic recipe (chat + expansion touchpoints). **v0.31.12:** chat and expansion `models:` lists drop the v0.31.6 phantom `claude-sonnet-4-6-20250929` date suffix — canonical id is `claude-sonnet-4-6`. The wrong-direction alias `claude-sonnet-4-6 → claude-sonnet-4-6-20250929` is removed; a reverse alias `claude-sonnet-4-6-20250929 → claude-sonnet-4-6` keeps stale user configs working (rescues `facts.extraction_model` and `models.dream.synthesize` set by v0.31.6 installs). Recipe-shape regression pinned by `test/anthropic-model-ids.test.ts` (6 cases, verbatim cherry-pick of PR #830 plus the reverse-alias rescue case). - `src/core/anthropic-pricing.ts` — Single source of truth for Anthropic model pricing (per-MTok input/output). **v0.31.12:** Opus 4.7 corrected from `$15/$75` to `$5/$25` (the old number was from Opus 4 generation, never refreshed when 4.7 shipped); Opus 4.6 also corrected. Consumed by `src/core/budget-meter.ts` and `src/core/cross-modal-eval/runner.ts` — the cross-modal estimator now reads `ANTHROPIC_PRICING` for Anthropic models instead of duplicating the table, killing the v0.31.6 drift bug class. @@ -153,15 +153,15 @@ strict behavior when unset. - `src/commands/jobs.ts` — `gbrain jobs` CLI subcommands + `gbrain jobs work` daemon. **v0.28.1:** `case 'work'` now wraps `worker.start()` in try/finally and owns engine lifecycle — calls `engine.disconnect()` on shutdown with loud error logging on failure. Replaces the prior call inside `MinionWorker.start()` (which violated engine ownership: the worker disconnected an engine it didn't own, and clobbered the module-level singleton on PostgresEngine via the now-fixed idempotency bug). Pool slots now free immediately on shutdown instead of waiting for TCP keepalive (~minutes). v0.13.1 surfaces the full `MinionJobInput` retry/backoff/timeout/idempotency surface as first-class CLI flags on `jobs submit`: `--max-stalled`, `--backoff-type fixed|exponential`, `--backoff-delay`, `--backoff-jitter`, `--timeout-ms`, `--idempotency-key`. `jobs smoke --sigkill-rescue` is the opt-in regression guard for #219. v0.16 wires `registerBuiltinHandlers` to always register `subagent` + `subagent_aggregator` (no env flag — `ANTHROPIC_API_KEY` is the natural cost gate, trust is via `PROTECTED_JOB_NAMES`) and loads `GBRAIN_PLUGIN_PATH` plugins at worker startup with a loud startup-line per plugin. `shell` handler still gated by `GBRAIN_ALLOW_SHELL_JOBS=1` (RCE surface, separate concern). v0.22.10 (#521): the `autopilot-cycle` handler now forwards `job.data.phases` to `runCycle` (was previously discarded — caller-supplied phase selection silently became a full cycle). Phases are validated against `ALL_PHASES` from `src/core/cycle.ts`; invalid names are filtered out and an empty/missing array falls back to the default 6-phase cycle. v0.22.13 (PR #490 CODEX-1+CODEX-4): `sync` handler now resolves `sourceId` at entry by looking up `sources.local_path` (mirrors `cycle.ts:480`'s autopilot fix from PR #475) so multi-source brains read the per-source `last_commit` anchor instead of the global config key. Concurrency routed through the shared `autoConcurrency()` policy in `src/core/sync-concurrency.ts` instead of the prior hardcoded `4`; PGLite stays serial. `noEmbed` default is `true` (embed is a separate job — submit `gbrain embed --stale` after sync, or rely on the autopilot cycle's embed phase). - `src/commands/features.ts` — `gbrain features --json --auto-fix`: usage scan + feature adoption salesman - `src/commands/autopilot.ts` — `gbrain autopilot --install`: self-maintaining brain daemon (sync+extract+embed). **v0.28.1:** consumes `detectTini()` from `src/core/minions/spawn-helpers.ts` and resolves it once at startup instead of per worker respawn (was paying an `execFileSync` cost on every restart). -- `src/mcp/server.ts` — MCP stdio server (generated from operations). v0.22.7: tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path. +- `src/mcp/server.ts` — MCP stdio server (generated from operations). v0.22.7: tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path. **v0.34.1.0 (#870):** stdin `'end'` / `'close'` shutdown hooks are skipped when `process.env.MCP_STDIO === '1'`. Gateway-piped stdio MCP wrappers (OpenClaw's `bundle-mcp`, similar) pipe the JSON-RPC handshake then close their stdin half; pre-fix this killed the server before the first tool call landed. Signal handlers (SIGTERM / SIGINT / SIGHUP) and the parent-process watchdog still cover legitimate disconnects. `src/commands/serve.ts` exposes `ServeOptions.mcpStdio?: boolean` as a test seam so the runtime guard is exercisable without process.env mutation. Pinned by `test/serve-stdio-lifecycle.test.ts`. - `src/mcp/dispatch.ts` (v0.22.7) — Shared tool-call dispatch consumed by both stdio (`server.ts`) and HTTP transports. Exports `dispatchToolCall(engine, name, params, opts)`, `buildOperationContext(engine, params, opts)`, and `validateParams(op, params)`. Single source of truth for `(ctx, params)` handler arg order and the 5-field `OperationContext` shape (engine + config + logger + dryRun + remote). Defaults to `remote: true` (untrusted); local CLI callers pass `remote: false`. Closed F1/F2/F3 drift bugs in the original v0.22.5 HTTP transport. **v0.26.9 (F8):** adds `summarizeMcpParams(opName, params)` — privacy-preserving redactor for `mcp_request_log` and the admin SSE feed. Returns `{redacted, kind, declared_keys, unknown_key_count, approx_bytes}`. Intersects submitted top-level keys against the operation's declared `params` allow-list (declared keys preserved as a sorted array for debug visibility; unknown keys counted but never named, closing the attacker-controlled-key-name leak). Byte counts bucketed up to nearest 1KB so an attacker can't binary-search secret-content sizes via repeated probes. Operators on a personal laptop who want raw payload visibility opt back in with `gbrain serve --http --log-full-params` (loud stderr warning at startup). Canonical helper — new logging code paths route through it rather than `JSON.stringify(params)`. - `src/mcp/rate-limit.ts` (v0.22.7) — Bounded-LRU token-bucket limiter. `buildDefaultLimiters()` returns the two-bucket pipeline: pre-auth IP (30/60s, fires BEFORE the DB lookup so brute-force load against `access_tokens` is actually capped) + post-auth token-id (60/60s). Tracks `lastTouchedMs` separately from `lastRefillMs` so an exhausted key can't be reset by hammering past the TTL. LRU cap bounds memory under attacker-controlled key growth. -- `src/commands/serve-http.ts` (v0.26.0) — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. Started via `gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--log-full-params]`. Supersedes the v0.22.7 `src/mcp/http-transport.ts` simple bearer-auth path. Combines MCP SDK's `mcpAuthRouter` (authorize / token / register / revoke endpoints), a custom `client_credentials` handler (SDK's token endpoint throws `UnsupportedGrantTypeError` for CC; the custom handler runs BEFORE the router and falls through for `auth_code` / `refresh_token`), `requireBearerAuth` middleware for `/mcp` with scope enforcement before op dispatch, `localOnly` rejection, and `express-rate-limit` at 50 req / 15 min on `/token`. Serves the built admin SPA from `admin/dist/` with SPA fallback. `/admin/events` SSE endpoint broadcasts every MCP request to connected admin browsers. `cookie-parser` middleware wired (Express 5 has no built-in). Startup logging prints port, engine, configured issuer URL (honors `--public-url`), registered-client count, DCR status, and admin bootstrap token. **v0.26.9 hardening pass:** F7 sets `remote: true` explicitly on the `/mcp` request handler's OperationContext literal (closes the HTTP shell-job RCE — without this, `submit_job`'s protected-name guard at `operations.ts:1391` saw a falsy undefined and skipped, letting a `read+write`-scoped OAuth token submit `shell` jobs). F8 wires `summarizeMcpParams` from `src/mcp/dispatch.ts` into both `mcp_request_log` writes and the admin SSE feed by default (raw payloads opt-in via `--log-full-params` with stderr warning). F9 sets cookie `Secure` flag when behind HTTPS or a public-URL proxy. F10 caps the magic-link nonce store with an LRU bound. F12 routes DCR disable through the `GBrainOAuthProvider` constructor's `dcrDisabled` option instead of the prior monkey-patch on the express router. F14 wraps `transport.handleRequest` in try/catch so SDK throws return a JSON-RPC 500 envelope instead of express's default HTML error page. F15 unifies OperationError + unexpected exceptions through `buildError` / `serializeError` so `/mcp` always returns the same envelope shape. **v0.28.1:** `/health` endpoint extracted into pure `probeHealth(engine)` async function with `HEALTH_TIMEOUT_MS = 3000` exported constant — drops the timeout from 5s to 3s so Fly.io's 5s health-check deadline gets 2s of headroom for TCP, response framing, and clock skew. Races `engine.getStats()` against the timeout via `Promise.race`; saturated pool returns 503 with `Health check timed out (database pool may be saturated)` instead of hanging. `clearTimeout` in finally block prevents pending-timer pile-up under high probe rates (race-leak fix from adversarial review). **v0.28.10:** `/health` is now liveness-only via the new `probeLiveness(sql, engineName, version, timeoutMs)` helper that races `sql\`SELECT 1\`` against `HEALTH_TIMEOUT_MS` and returns the same `ProbeHealthResult` tagged-union as `probeHealth` (single timer-cleanup site, single 503 envelope). Body shape: `{status, version, engine}` only — engine stats are no longer spread on the public route. Full stats moved to a new admin endpoint `/admin/api/full-stats` (sibling to `/admin/api/stats` and `/admin/api/health-indicators`) gated by the existing `requireAdmin` middleware; that route calls `probeHealth(engine, ...)` and returns the original spread-stats body. `?full=true` query param removed entirely. Closes the original DoS surface where `getStats()`'s 6× count(*) on 96K-page brains through PgBouncer exceeded `HEALTH_TIMEOUT_MS` and triggered orchestrator restart cascades (Fly.io / k8s seeing 503 → restart loop → advisory-lock pile-up on the migration lock). Outside-voice review (Codex) caught that `/admin/api/health-indicators` is NOT a full-stats endpoint (returns only `{expiring_soon, error_rate}`), and that an alternative loopback-IP gate would have depended on `app.set('trust proxy', 'loopback')` semantics holding under proxy/XFF misconfiguration; the shipped admin-cookie design avoids both. **v0.31.3 (#681):** every OAuth/admin/audit SQL call routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` so `gbrain serve --http` works against PGLite brains. The four `mcp_request_log.params` INSERT sites (success path, auth_failed path, scope_denied path, server-error path) all go through `executeRawJsonb(engine, ...)` so the JSONB column stores real objects, not JSON-encoded strings — closes the bug where `params->>'op'` returned the encoded string `"search"` (with quotes) instead of `search`. Migration v46 normalizes any pre-v0.31.3 string-shaped backlog rows on first start. +- `src/commands/serve-http.ts` (v0.26.0) — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. Started via `gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--log-full-params]`. Supersedes the v0.22.7 `src/mcp/http-transport.ts` simple bearer-auth path. Combines MCP SDK's `mcpAuthRouter` (authorize / token / register / revoke endpoints), a custom `client_credentials` handler (SDK's token endpoint throws `UnsupportedGrantTypeError` for CC; the custom handler runs BEFORE the router and falls through for `auth_code` / `refresh_token`), `requireBearerAuth` middleware for `/mcp` with scope enforcement before op dispatch, `localOnly` rejection, and `express-rate-limit` at 50 req / 15 min on `/token`. Serves the built admin SPA from `admin/dist/` with SPA fallback. `/admin/events` SSE endpoint broadcasts every MCP request to connected admin browsers. `cookie-parser` middleware wired (Express 5 has no built-in). Startup logging prints port, engine, configured issuer URL (honors `--public-url`), registered-client count, DCR status, and admin bootstrap token. **v0.26.9 hardening pass:** F7 sets `remote: true` explicitly on the `/mcp` request handler's OperationContext literal (closes the HTTP shell-job RCE — without this, `submit_job`'s protected-name guard at `operations.ts:1391` saw a falsy undefined and skipped, letting a `read+write`-scoped OAuth token submit `shell` jobs). F8 wires `summarizeMcpParams` from `src/mcp/dispatch.ts` into both `mcp_request_log` writes and the admin SSE feed by default (raw payloads opt-in via `--log-full-params` with stderr warning). F9 sets cookie `Secure` flag when behind HTTPS or a public-URL proxy. F10 caps the magic-link nonce store with an LRU bound. F12 routes DCR disable through the `GBrainOAuthProvider` constructor's `dcrDisabled` option instead of the prior monkey-patch on the express router. F14 wraps `transport.handleRequest` in try/catch so SDK throws return a JSON-RPC 500 envelope instead of express's default HTML error page. F15 unifies OperationError + unexpected exceptions through `buildError` / `serializeError` so `/mcp` always returns the same envelope shape. **v0.28.1:** `/health` endpoint extracted into pure `probeHealth(engine)` async function with `HEALTH_TIMEOUT_MS = 3000` exported constant — drops the timeout from 5s to 3s so Fly.io's 5s health-check deadline gets 2s of headroom for TCP, response framing, and clock skew. Races `engine.getStats()` against the timeout via `Promise.race`; saturated pool returns 503 with `Health check timed out (database pool may be saturated)` instead of hanging. `clearTimeout` in finally block prevents pending-timer pile-up under high probe rates (race-leak fix from adversarial review). **v0.28.10:** `/health` is now liveness-only via the new `probeLiveness(sql, engineName, version, timeoutMs)` helper that races `sql\`SELECT 1\`` against `HEALTH_TIMEOUT_MS` and returns the same `ProbeHealthResult` tagged-union as `probeHealth` (single timer-cleanup site, single 503 envelope). Body shape: `{status, version, engine}` only — engine stats are no longer spread on the public route. Full stats moved to a new admin endpoint `/admin/api/full-stats` (sibling to `/admin/api/stats` and `/admin/api/health-indicators`) gated by the existing `requireAdmin` middleware; that route calls `probeHealth(engine, ...)` and returns the original spread-stats body. `?full=true` query param removed entirely. Closes the original DoS surface where `getStats()`'s 6× count(*) on 96K-page brains through PgBouncer exceeded `HEALTH_TIMEOUT_MS` and triggered orchestrator restart cascades (Fly.io / k8s seeing 503 → restart loop → advisory-lock pile-up on the migration lock). Outside-voice review (Codex) caught that `/admin/api/health-indicators` is NOT a full-stats endpoint (returns only `{expiring_soon, error_rate}`), and that an alternative loopback-IP gate would have depended on `app.set('trust proxy', 'loopback')` semantics holding under proxy/XFF misconfiguration; the shipped admin-cookie design avoids both. **v0.31.3 (#681):** every OAuth/admin/audit SQL call routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` so `gbrain serve --http` works against PGLite brains. The four `mcp_request_log.params` INSERT sites (success path, auth_failed path, scope_denied path, server-error path) all go through `executeRawJsonb(engine, ...)` so the JSONB column stores real objects, not JSON-encoded strings — closes the bug where `params->>'op'` returned the encoded string `"search"` (with quotes) instead of `search`. Migration v46 normalizes any pre-v0.31.3 string-shaped backlog rows on first start. **v0.34.1.0 (#864):** new `--bind HOST` CLI flag with default `127.0.0.1`. Personal-laptop installs no longer publish the brain to the LAN by accident. Self-hosted operators pass `--bind 0.0.0.0` (or a specific interface IP) once to accept remote connections. A stderr WARN fires when `--public-url` is set without `--bind` so the operator sees the binding before the first request (common cause of "ngrok forwards to me but the agent can't reach the upstream" misconfigurations). The startup banner prints a `Bind:` line. **v0.34.1.0 (#861):** drops the `(authInfo as AuthInfo & {sourceId?: string}).sourceId ?? env ?? 'default'` cast chain — `AuthInfo.sourceId` and `AuthInfo.allowedSources` are now the typed source of truth, populated by `oauth-provider.ts:verifyAccessToken` from the `oauth_clients` row. - `src/core/sql-query.ts` (v0.31.3) — Engine-aware tagged-template SQL adapter for OAuth/admin/auth infrastructure. `sqlQueryForEngine(engine)` returns a `SqlQuery` (`(strings, ...values) => Promise`) that walks the template, builds `$N` positional SQL, asserts every value is a `SqlValue` (string | number | bigint | boolean | Date | null), and routes through `engine.executeRaw(sql, params)` so Postgres goes via postgres.js's `unsafe(sql, params)` path and PGLite via its embedded `db.query(sql, params)`. Deliberately narrower than postgres.js's `sql` tag: no nested fragments, no `sql.json()`, no `sql.unsafe()`, no `sql.begin()`, no array binding. The narrow surface is the feature — codex finding #7 from the v0.31 plan review argued the adapter should stay scalar-only or it drifts into a partial postgres.js clone. JSONB writes go through the separate `executeRawJsonb(engine, sql, scalarParams, jsonbParams)` helper that composes positional `$N::jsonb` casts and passes JS objects through; the v0.12.0 double-encode bug class doesn't apply because positional binding through `unsafe()` reaches the wire protocol with the correct type oid (verified by `test/sql-query.test.ts` on PGLite and `test/e2e/auth-permissions.test.ts:67` on Postgres). `scripts/check-jsonb-pattern.sh` doesn't fire because `executeRawJsonb(...)` is a method call, not the banned literal-template-tag interpolation pattern. Consumed by `src/commands/auth.ts`, `src/commands/serve-http.ts`, `src/core/oauth-provider.ts`, `src/commands/files.ts`, and `src/mcp/http-transport.ts` so all five sites work against PGLite and Postgres uniformly. Closes the bug where `gbrain auth` + `gbrain serve --http` were silently Postgres-only because they routed every SQL through the postgres.js singleton (community PR #681). - `src/commands/serve.ts` (v0.31.3) — `gbrain serve` stdio MCP entrypoint with idempotent shutdown across every parent-disconnect signal. Stdio EOF, SIGTERM, SIGINT, SIGHUP, and parent-process death (every reparent case — PID 1, launchd subreaper, systemd, tmux, or a parent shell with `PR_SET_CHILD_SUBREAPER`) all funnel into one `cleanup(reason)` path that releases the engine and the PGLite write-lock dir within 5 seconds. Pre-v0.31.3 the stdio MCP server held the lock indefinitely after Claude Desktop / Cursor / launchd-managed gateways disconnected, forcing a 5-minute stale-lock wait on the next start. Watchdog reparent check is `getParentPid() !== initialParentPid` (capturing the initial ppid once at install time and firing on any change); the previous `=== 1` check missed the subreaper case under launchd / systemd. Bun's `process.ppid` cache is stale across reparenting (see [oven-sh/bun#30305](https://github.com/oven-sh/bun/issues/30305)) so `getParentPid()` runs `spawnSync('ps', ['-o', 'ppid=', '-p', PID])` per tick to read the live kernel PPID. Startup probe verifies `ps` is on PATH; if not (stripped containers, busybox without procps), the watchdog skips installing AND emits a loud `[gbrain serve] watchdog disabled: ps unavailable, parent-death detection unavailable — child will rely on stdin EOF / signals only` stderr line so operators see the degraded mode at boot. Pinned by `test/serve-stdio-lifecycle.test.ts` (22 cases). Closes #413, #446. Credit @Aragorn2046 (origin features in #591) and @seungsu-kr (rebased submitter, Bun ppid workaround). -- `src/core/oauth-provider.ts` (v0.26.0) — `GBrainOAuthProvider` implementing the MCP SDK's `OAuthServerProvider` + `OAuthRegisteredClientsStore` interfaces. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1 spec: `authorize` + `exchangeAuthorizationCode` with PKCE (for ChatGPT), `client_credentials` (for Perplexity / Claude), `refresh_token` with rotation, `revokeToken`, `registerClient` (DCR path validates redirect_uri must be `https://` or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Auth codes single-use with 10-minute TTL via atomic `DELETE...RETURNING` (closes RFC 6749 §10.5 TOCTOU race). Refresh rotation also `DELETE...RETURNING` (closes §10.4 stolen-token detection bypass). `pgArray()` escapes commas/quotes/braces in elements so a comma-bearing redirect_uri can't smuggle a second array element. Legacy `access_tokens` fallback in `verifyAccessToken` grandfathers pre-v0.26 bearer tokens as `read+write+admin`. `sweepExpiredTokens()` runs on startup wrapped in try/catch. **v0.26.9 RFC 6749/7009 hardening pass:** F1+F2 fold `client_id` atomically into the `DELETE WHERE` clauses for both auth-code exchange and refresh rotation — pre-fix the post-hoc client compare burned the row on wrong-client paths so the legitimate client couldn't retry. F3 enforces refresh-scope-subset against the original grant on the row (RFC 6749 §6), not the client's currently-allowed scopes — fixes the case where revoking a scope from a client wouldn't shrink the agent's existing refresh tokens. F4 binds `client_id` on `revokeToken` so a client can only revoke its own tokens (RFC 7009 §2.1). F7c validates the `/token` request's `redirect_uri` against the value stored at `/authorize` (RFC 6749 §4.1.3) — empty-string treated as missing rather than wildcard match (adversarial-review fix). F5 swaps bare `catch {}` blocks in `verifyAccessToken` and `getClient` for `isUndefinedColumnError` from `src/core/utils.ts` — only SQLSTATE 42703 falls through to legacy fallback; lock timeouts and network blips throw and surface. F6 makes `sweepExpiredTokens()` actually return the count via `RETURNING 1` + array length, not a fire-and-forget zero. F12 adds `dcrDisabled` constructor option so `serve-http.ts` can disable the `/register` endpoint without monkey-patching the router. **v0.26.2:** module-private `coerceTimestamp()` boundary helper at the top of the file normalizes postgres-driver-as-string BIGINT columns to JS numbers at every read site (5 call sites: `getClient` L112+L113 for DCR `/register` RFC 7591 §3.2.1 numeric timestamps, `exchangeRefreshToken` L274 + `verifyAccessToken` L296+L303 for the SDK's `typeof === 'number'` bearerAuth check). Throws on non-finite input (NaN/Infinity) so corrupt rows fail loud at the boundary instead of riding through as `expiresAt: NaN`; returns undefined for SQL NULL so callers decide NULL semantics explicitly (refresh + access token paths treat NULL as expired). Helper intentionally NOT promoted to `src/core/utils.ts` — codex review flagged repo-wide BIGINT precision-loss risk for a generic helper. +- `src/core/oauth-provider.ts` (v0.26.0) — `GBrainOAuthProvider` implementing the MCP SDK's `OAuthServerProvider` + `OAuthRegisteredClientsStore` interfaces. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1 spec: `authorize` + `exchangeAuthorizationCode` with PKCE (for ChatGPT), `client_credentials` (for Perplexity / Claude), `refresh_token` with rotation, `revokeToken`, `registerClient` (DCR path validates redirect_uri must be `https://` or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Auth codes single-use with 10-minute TTL via atomic `DELETE...RETURNING` (closes RFC 6749 §10.5 TOCTOU race). Refresh rotation also `DELETE...RETURNING` (closes §10.4 stolen-token detection bypass). `pgArray()` escapes commas/quotes/braces in elements so a comma-bearing redirect_uri can't smuggle a second array element. Legacy `access_tokens` fallback in `verifyAccessToken` grandfathers pre-v0.26 bearer tokens as `read+write+admin`. `sweepExpiredTokens()` runs on startup wrapped in try/catch. **v0.26.9 RFC 6749/7009 hardening pass:** F1+F2 fold `client_id` atomically into the `DELETE WHERE` clauses for both auth-code exchange and refresh rotation — pre-fix the post-hoc client compare burned the row on wrong-client paths so the legitimate client couldn't retry. F3 enforces refresh-scope-subset against the original grant on the row (RFC 6749 §6), not the client's currently-allowed scopes — fixes the case where revoking a scope from a client wouldn't shrink the agent's existing refresh tokens. F4 binds `client_id` on `revokeToken` so a client can only revoke its own tokens (RFC 7009 §2.1). F7c validates the `/token` request's `redirect_uri` against the value stored at `/authorize` (RFC 6749 §4.1.3) — empty-string treated as missing rather than wildcard match (adversarial-review fix). F5 swaps bare `catch {}` blocks in `verifyAccessToken` and `getClient` for `isUndefinedColumnError` from `src/core/utils.ts` — only SQLSTATE 42703 falls through to legacy fallback; lock timeouts and network blips throw and surface. F6 makes `sweepExpiredTokens()` actually return the count via `RETURNING 1` + array length, not a fire-and-forget zero. F12 adds `dcrDisabled` constructor option so `serve-http.ts` can disable the `/register` endpoint without monkey-patching the router. **v0.26.2:** module-private `coerceTimestamp()` boundary helper at the top of the file normalizes postgres-driver-as-string BIGINT columns to JS numbers at every read site (5 call sites: `getClient` L112+L113 for DCR `/register` RFC 7591 §3.2.1 numeric timestamps, `exchangeRefreshToken` L274 + `verifyAccessToken` L296+L303 for the SDK's `typeof === 'number'` bearerAuth check). Throws on non-finite input (NaN/Infinity) so corrupt rows fail loud at the boundary instead of riding through as `expiresAt: NaN`; returns undefined for SQL NULL so callers decide NULL semantics explicitly (refresh + access token paths treat NULL as expired). Helper intentionally NOT promoted to `src/core/utils.ts` — codex review flagged repo-wide BIGINT precision-loss risk for a generic helper. **v0.34.1.0 (#909):** `registerClient` honors `token_endpoint_auth_method: "none"` (RFC 7591 §3.2.1) — public PKCE clients (Claude Code, Cursor, every other PKCE-first MCP client) store `client_secret_hash = NULL` and the response payload omits `client_secret` entirely. Confidential clients (default `client_secret_post` and explicit `client_secret_basic`) keep their one-time-reveal shape. `getClient` correctly normalizes a NULL `client_secret_hash` to JS `undefined` so the SDK's clientAuth path accepts the public client at `/token`. **v0.34.1.0 (#861 + #876):** `verifyAccessToken` JOINs `oauth_clients.source_id` (write scope, scalar) + `oauth_clients.federated_read` (read scope, TEXT[]) and surfaces both on the returned `AuthInfo`. Pre-v60 / pre-v61 brains degrade gracefully via `isUndefinedColumnError` fallback so the upgrade chain is non-blocking on legacy DBs. - `admin/` (v0.26.0) — React 19 + Vite + TypeScript admin SPA embedded in the binary via `admin/dist/` served by `serve-http.ts`. 7 screens: Login (bootstrap token → session cookie), Dashboard (metrics + SSE feed + token health), Agents (sortable table + sparklines + Register button), Register (modal with scope checkboxes + grant type selector), Credentials reveal (full-screen modal with Copy + Download JSON + yellow one-time-only warning), Request Log (filterable paginated), Agent Detail drawer (Details / Activity / Config Export tabs + Revoke). Design tokens: `#0a0a0f` bg, Inter for UI, JetBrains Mono for data, 4-32px spacing scale, rounded pill badges. HTTP-only SameSite=Strict cookie auth. 65KB gzip. Build: `cd admin && bun install && bun run build`; output at `admin/dist/` is committed for self-contained binaries. -- `src/commands/auth.ts` — Token management. `gbrain auth create/list/revoke/test` for legacy bearer tokens (v0.22.7 wired as a first-class CLI subcommand) plus `gbrain auth register-client` (v0.26.0) and `gbrain auth revoke-client ` (v0.26.2) for OAuth 2.1 client lifecycle. `revoke-client` runs an atomic `DELETE...RETURNING` on `oauth_clients`; FK `ON DELETE CASCADE` on `oauth_tokens.client_id` and `oauth_codes.client_id` purges every active token + authorization code in a single transaction. `process.exit(1)` on no-such-client (idempotent — re-running on the same id produces the same exit-1 message). Legacy tokens stored as SHA-256 hashes in `access_tokens`; OAuth clients in `oauth_clients`. As of v0.26.0, legacy tokens grandfather to `read+write+admin` scopes on the OAuth HTTP server, so pre-v0.26 deployments keep working with no migration. **v0.31.3 (#681):** every SQL site routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` (and `executeRawJsonb` for the takes-holders `permissions` JSONB column) so `gbrain auth` works against PGLite brains. Pre-fix, every call hit the postgres.js singleton via `getConn()` and silently failed (or wrote to the wrong DB) when the active engine was PGLite. The takes-holders write goes through `executeRawJsonb(engine, sql, [name, hash], [{takes_holders:[...]}])` which round-trips with `jsonb_typeof = 'object'` instead of the pre-v0.31.3 quoted-string shape. +- `src/commands/auth.ts` — Token management. `gbrain auth create/list/revoke/test` for legacy bearer tokens (v0.22.7 wired as a first-class CLI subcommand) plus `gbrain auth register-client` (v0.26.0) and `gbrain auth revoke-client ` (v0.26.2) for OAuth 2.1 client lifecycle. `revoke-client` runs an atomic `DELETE...RETURNING` on `oauth_clients`; FK `ON DELETE CASCADE` on `oauth_tokens.client_id` and `oauth_codes.client_id` purges every active token + authorization code in a single transaction. `process.exit(1)` on no-such-client (idempotent — re-running on the same id produces the same exit-1 message). Legacy tokens stored as SHA-256 hashes in `access_tokens`; OAuth clients in `oauth_clients`. As of v0.26.0, legacy tokens grandfather to `read+write+admin` scopes on the OAuth HTTP server, so pre-v0.26 deployments keep working with no migration. **v0.31.3 (#681):** every SQL site routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` (and `executeRawJsonb` for the takes-holders `permissions` JSONB column) so `gbrain auth` works against PGLite brains. Pre-fix, every call hit the postgres.js singleton via `getConn()` and silently failed (or wrote to the wrong DB) when the active engine was PGLite. The takes-holders write goes through `executeRawJsonb(engine, sql, [name, hash], [{takes_holders:[...]}])` which round-trips with `jsonb_typeof = 'object'` instead of the pre-v0.31.3 quoted-string shape. **v0.34.1.0 (#876):** `register-client` accepts `--source ` (write authority, scalar) and `--federated-read ` (read scope, array). The output prints the resolved `Write source` and `Federated reads` for the registered client. Pre-v0.34 clients backfill to `source_id='default'` via migration v60 so existing deployments keep their v0.33 effective behavior verbatim. - `src/commands/upgrade.ts` — Self-update CLI. `runPostUpgrade()` enumerates migrations from the TS registry (src/commands/migrations/index.ts) and tail-calls `runApplyMigrations(['--yes', '--non-interactive'])` so the mechanical side of every outstanding migration runs unconditionally. - `src/commands/migrations/` — TS migration registry (compiled into the binary; no filesystem walk of `skills/migrations/*.md` needed at runtime). `index.ts` lists migrations in semver order. `v0_11_0.ts` = Minions adoption orchestrator (8 phases). `v0_12_0.ts` = Knowledge Graph auto-wire orchestrator (5 phases: schema → config check → backfill links → backfill timeline → verify). `phaseASchema` has a 600s timeout (bumped from 60s in v0.12.1 for duplicate-heavy brains). `v0_12_2.ts` = JSONB double-encode repair orchestrator (4 phases: schema → repair-jsonb → verify → record). `v0_14_0.ts` = shell-jobs + autopilot cooperative (2 phases: schema ALTER minion_jobs.max_stalled SET DEFAULT 3 — superseded by v0.14.3's schema-level DEFAULT 5 + UPDATE backfill; pending-host-work ping for skills/migrations/v0.14.0.md). All orchestrators are idempotent and resumable from `partial` status. As of v0.14.2 (Bug 3), the RUNNER owns all ledger writes — orchestrators return `OrchestratorResult` and `apply-migrations.ts` persists a canonical `{version, status, phases}` shape after return. Orchestrators no longer call `appendCompletedMigration` directly. `statusForVersion` prefers `complete` over `partial` (never regresses). 3 consecutive partials → wedged → `--force-retry ` writes a `'retry'` reset marker. v0.14.3 (fix wave) ships schema-only migrations v14 (`pages_updated_at_index`) + v15 (`minion_jobs_max_stalled_default_5` with UPDATE backfill) via the `MIGRATIONS` array in `src/core/migrate.ts` — no orchestrator phases needed. - `src/commands/repair-jsonb.ts` — `gbrain repair-jsonb [--dry-run] [--json]`: rewrites `jsonb_typeof='string'` rows in place across 5 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter). Fixes v0.12.0 double-encode bug on Postgres; PGLite no-ops. Idempotent. @@ -174,7 +174,7 @@ strict behavior when unset. - `src/commands/transcripts.ts` (v0.29) — `gbrain transcripts recent [--days N] [--full] [--json]`: recent raw `.txt` transcripts from the dream-cycle corpus dirs. Imports `listRecentTranscripts` from `src/core/transcripts.ts` (the same library the gated `get_recent_transcripts` MCP op uses). Local-only by construction — the CLI always runs with `ctx.remote=false`. - `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s). Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable. **v0.32.8 (PR #860):** batch projection switched from `SELECT DISTINCT ON (slug)` to `SELECT ... ORDER BY source_id, slug` so multi-source brains scan each `(source, slug)` row independently (pre-fix the DISTINCT collapsed same-slug-different-source pages into one scan, the same bug class this PR fixes). Sequential and auto-repair loops use `listAllPageRefs()` to enumerate `(slug, source_id)` pairs and thread `sourceId` to `getPage`. Batch + sequential paths now report the same page count on multi-source brains. - `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count. v0.26.7 (#612): `rls_event_trigger` check (post-install drift detector for migration v35's auto-RLS event trigger). Lives outside the `// 5. RLS` slice that the structural doctor.test.ts guards anchor on, so the existing test guards stay intact. Healthy `evtenabled` set is `('O','A')` only — `R` is replica-only and would not fire in normal sessions; `D` is disabled. Fix hint is `gbrain apply-migrations --force-retry 35`. **v0.30.2:** `queue_health` gains a fourth subcheck — surfaces dead-lettered subagent jobs with `last_error` matching the `prompt_too_long` classifier within the last 24h. Fix hint points at `gbrain dream --phase synthesize --dry-run --json` to identify the offending transcript and `gbrain jobs prune --status dead --queue default` to clean up. Postgres-only. **v0.31.7:** `runDoctor` switches to `autoDetectSkillsDirReadOnly` (from `src/core/repo-root.ts`) so `bun install -g github:garrytan/gbrain && cd ~ && gbrain doctor` finds the bundled `skills/` via the install-path fallback instead of warning "Could not find skills directory" + docking the health score. `--fix` carries a D6 safety gate: when `detected.source === 'install_path'`, the command refuses auto-repair with a stderr message pointing at `$GBRAIN_SKILLS_DIR` / `$OPENCLAW_WORKSPACE` / `--skills-dir`, because `autoFixDryViolations` writes to SKILL.md files and would otherwise silently rewrite the install tree. The `graph_coverage` check now short-circuits to `ok: 'No entity pages — graph_coverage not applicable (markdown-only brain)'` when `SELECT COUNT(*) FROM pages WHERE type IN ('entity','person','company','organization')` returns 0 (closes #530); the entity count is woven into the warn message and the WARN hint switches from the long-deprecated `gbrain link-extract && gbrain timeline-extract` (gone since v0.16) to the canonical `gbrain extract all`. Pinned by an IRON-RULE regression assertion in `test/doctor.test.ts` that bans the stale verb names from the source string. **v0.32.4:** new `sync_freshness` check (exported `checkSyncFreshness` at the same file) added to both `runDoctor` (local) and `doctorReportRemote` (thin-client). Pure staleness probe — queries `sources.last_sync_at` only, no filesystem access. Warns at 24h, fails at 72h (or never-synced). Future-`last_sync_at` warns ("clock skew or corrupted timestamp") instead of silently falling through as ok — codex outside-voice caught the negative-ageMs bug pre-merge. Env-var overrides `GBRAIN_SYNC_FRESHNESS_WARN_HOURS` / `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS`; invalid values fall back to defaults with a once-per-process stderr warn (`_resolveSyncFreshnessHours`). Failure messages embed `source.id` (not `source.name`) so the printed fix command `gbrain sync --source ` matches what the user copy-pastes. Filesystem-vs-DB page drift detection was deliberately stripped from the v0.32.4 scope — `doctorReportRemote` runs in the HTTP MCP server (`src/commands/serve-http.ts`), and walking DB-supplied `local_path` from a remote-callable endpoint crosses a trust boundary (OAuth write scope could mutate `sources.local_path`). Drift detection will resurface in a separate PR routed through `multi_source_drift`'s existing guard infrastructure (`GBRAIN_DRIFT_LIMIT` / `GBRAIN_DRIFT_TIMEOUT_MS`) with slug normalization tests and a meta-file allow-list. Pinned by 12 cases in `test/doctor.test.ts` ("v0.32.4 — sync_freshness check" describe block): empty sources, never-synced fail, >72h fail, exact 72h boundary, 24h-72h warn, exact 24h boundary, <24h ok, future-timestamp warn, mixed sources (highest severity wins), `executeRaw` throws → outer-catch warn, `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS=6` override fires at 7h, source.id-in-message regression. -- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). **v40 (v0.29):** `pages_emotional_weight` adds `pages.emotional_weight REAL NOT NULL DEFAULT 0.0`. Column-only (no index). On Postgres 11+ and PGLite, `ADD COLUMN` with a constant DEFAULT is metadata-only — instant on tables of any size. v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs. **v35 (v0.26.7):** auto-RLS event trigger + one-time backfill. `auto_rls_on_create_table` fires on `ddl_command_end` for `WHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO')` and runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY` on every new `public.*` table — no FORCE (matches v24/v29/schema.sql posture so non-BYPASSRLS apps can still read their own tables). The same migration backfills RLS on every existing `public.*` base table whose comment doesn't match the doctor regex (`^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,}`). Per-table failure aborts the offending CREATE TABLE (event triggers fire inside the DDL transaction); no EXCEPTION wrap — that would convert loud rollback into silent permissive default. PGLite no-op via `sqlFor.pglite: ''`. Breaking change: operators with intentionally-RLS-off public tables must add the GBRAIN:RLS_EXEMPT comment BEFORE upgrade or the backfill will flip them on. **v46 (v0.31.3):** `mcp_request_log_params_jsonb_normalize` rewrites pre-v0.31.3 rows where `mcp_request_log.params` was stored as a JSON-encoded string (`jsonb_typeof = 'string'`) up to a real JSONB object via `UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string'`. Single statement, idempotent — second-run finds no string-shaped rows and is a no-op. Closes the bug where `/admin/api/requests` returned a quoted string instead of the parsed object. +- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). **v40 (v0.29):** `pages_emotional_weight` adds `pages.emotional_weight REAL NOT NULL DEFAULT 0.0`. Column-only (no index). On Postgres 11+ and PGLite, `ADD COLUMN` with a constant DEFAULT is metadata-only — instant on tables of any size. v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs. **v35 (v0.26.7):** auto-RLS event trigger + one-time backfill. `auto_rls_on_create_table` fires on `ddl_command_end` for `WHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO')` and runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY` on every new `public.*` table — no FORCE (matches v24/v29/schema.sql posture so non-BYPASSRLS apps can still read their own tables). The same migration backfills RLS on every existing `public.*` base table whose comment doesn't match the doctor regex (`^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,}`). Per-table failure aborts the offending CREATE TABLE (event triggers fire inside the DDL transaction); no EXCEPTION wrap — that would convert loud rollback into silent permissive default. PGLite no-op via `sqlFor.pglite: ''`. Breaking change: operators with intentionally-RLS-off public tables must add the GBRAIN:RLS_EXEMPT comment BEFORE upgrade or the backfill will flip them on. **v46 (v0.31.3):** `mcp_request_log_params_jsonb_normalize` rewrites pre-v0.31.3 rows where `mcp_request_log.params` was stored as a JSON-encoded string (`jsonb_typeof = 'string'`) up to a real JSONB object via `UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string'`. Single statement, idempotent — second-run finds no string-shaped rows and is a no-op. Closes the bug where `/admin/api/requests` returned a quoted string instead of the parsed object. **v0.34.1.0 (#861 + #876, v60-v65):** six-migration chain wires source-scoping into the OAuth client table. v60 (`oauth_clients_source_id_fk`) adds `oauth_clients.source_id TEXT` with NULL→`'default'` backfill and an FK to `sources(id) ON DELETE SET NULL`. v61 (`oauth_clients_federated_read_column`) adds `federated_read TEXT[] NOT NULL DEFAULT '{}'`. v62 (`oauth_clients_federated_read_backfill`) explicit-CASE backfills so `source_id IS NULL` produces `'{}'` not an array-containing-NULL. v63 (`oauth_clients_federated_read_validate`) is the fail-loud check that every row's source_id is in its federated_read array post-backfill. v64 (`oauth_clients_source_id_fk_restrict`) flips the FK to `ON DELETE RESTRICT` now that federated_read provides the alternative scope-loss path — source delete is refused if any client references it. v65 (`oauth_clients_federated_read_gin_index`) is the GIN index for the array-containment queries the read paths run. PGLite parity via `sqlFor.pglite` where needed. - `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2. - `src/core/cli-options.ts` — Global CLI flag parser. `parseGlobalFlags(argv)` returns `{cliOpts, rest}` with `--quiet` / `--progress-json` / `--progress-interval=` stripped. `getCliOptions()` / `setCliOptions()` expose a module-level singleton so commands reach the resolved flags without parameter threading. `cliOptsToProgressOptions()` maps to reporter options. `childGlobalFlags()` returns the flag suffix to append to `execSync('gbrain ...')` calls in migration orchestrators. `OperationContext.cliOpts` extends shared-op dispatch for MCP callers. - `src/core/db-lock.ts` (v0.22.13) — generic `tryAcquireDbLock(engine, lockId, ttlMinutes)` over the existing `gbrain_cycle_locks` table. Parameterized lock id so different scopes can nest cleanly: `gbrain-cycle` for the broad cycle (held by `cycle.ts`) and `gbrain-sync` (`SYNC_LOCK_ID` constant) for `performSync`'s narrower writer window. Same UPSERT-with-TTL semantics as the prior cycle-only helper, just generalized. Survives PgBouncer transaction pooling (unlike session-scoped `pg_try_advisory_lock`); crashed holders auto-release once their TTL expires. @@ -649,6 +649,10 @@ E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_U - `test/e2e/serve-http-oauth.test.ts` (v0.26.0, expanded v0.26.2, expanded v0.26.9) — real-Postgres E2E against `gbrain serve --http` with full OAuth 2.1. Spawns a subprocess server, registers a client via the CLI, mints `client_credentials` tokens, exercises the `/mcp` JSON-RPC pipeline. **v0.26.2 adds:** real DCR `/register` HTTP-level response-shape test (asserts `typeof body.client_id_issued_at === 'number'` over the wire — RFC 7591 §3.2.1 spec compliance, not just internal-store shape); real CLI subprocess test for `revoke-client` (registers → mints token → revokes via `execSync` → asserts token rejected at `/mcp` → asserts re-run exits 1); server fixture flips on `--enable-dcr` so `/register` is reachable. **bun execSync env-inheritance fix:** bun's `execSync` does NOT inherit env mutations done via `process.env.X = ...`, only OS-level env from before bun started. helpers.ts loads `.env.testing` and sets `DATABASE_URL` via `process.env` mutation, which is invisible to subprocesses unless `env: { ...process.env }` is passed explicitly — every subprocess call in this file passes `env: { ...process.env }` for that reason. Reference fix for the next maintainer hitting the same failure mode in sibling sync/cycle/dream/claw-test E2Es. `afterAll` cleanup is guarded on `clientId` (won't throw if `beforeAll` failed before registration); cleanup errors surface to stderr without throwing so real test failures aren't masked. Tracks DCR-registered clients alongside the manual one. **v0.26.9** adds 2 regressions for the F7 trust-boundary fix: an HTTP MCP `submit_job` for `name: "shell"` MUST reject with a permission error (proving the request handler now sets `remote: true` and `submit_job`'s protected-name guard fires), and the same guard rejects subagent submission. Closes the OAuth-token-to-RCE escalation path. Skips gracefully when `DATABASE_URL` is unset. - `test/e2e/sync-parallel.test.ts` (v0.22.13 PR #490) — DATABASE_URL-gated. T2: 60-file Postgres sync at concurrency=4 imports all + no connection leak (probes `pg_stat_activity` before/after to confirm worker engines disconnected). P4: 120-file serial-vs-parallel benchmark prints `SYNC_PARALLEL_BENCH N files | serial=Xms | parallel(4)=Yms | speedup=Zx` for CHANGELOG quoting. Asserts parallel ≤ serial × 1.5 (CI-noise tolerant; not a strict speedup gate). - `test/e2e/multi-source-bug-class.test.ts` (v0.32.8, PR #860) — 7-case PGLite in-memory regression suite pinning every bug site fixed in this PR: `listAllPageRefs` ordering by `(source_id, slug)` (F11), `getPage` with sourceId picks the right `(source, slug)` row (F2), `extract-takes` processes both overlapping `people/alice` rows independently, `listPages` filters correctly with `PageFilters.sourceId`, `addLinksBatch` with `from/to_source_id` targets the right rows (F4), `validateSourceId` rejects path traversal (F6), reverse-write disk layout uses `brainDir/.sources//.md` for non-default sources (F6). No DATABASE_URL needed. Wired into `scripts/e2e-test-map.ts` so changes to extract-takes / patterns / synthesize / embed / extract / migrate-engine auto-trigger this test. Companion: `test/e2e/integrity-batch.test.ts`'s "multi-source duplicate slugs scan once" case was pinning the pre-fix bug — assertion flipped in v0.32.8 to expect both batch + sequential paths report 2. +- `test/e2e/source-isolation-pglite.test.ts` (v0.34.1.0, #861) — 14-case PGLite in-memory regression suite pinning the source-isolation P0 seal at two layers. Engine layer: `searchKeyword` / `searchVector` / `searchKeywordChunks` / `listPages` / `getPage` / `traverseGraph` / `traversePaths` apply `sourceId` (scalar fast path) and `sourceIds` (array path) correctly across both engines. Op-handler layer: routes through `sourceScopeOpts(ctx)` so a `read+write`-scoped OAuth client bound to `--source dept-x` cannot see rows from neighboring sources via `search`, `query`, `list_pages`, `get_page`, or `find_experts`. Covers both `ctx.sourceId` (single-source clients) and `ctx.auth.allowedSources` (federated_read clients) precedence; federated array wins over scalar wins over nothing. No DATABASE_URL needed. +- `test/openai-compat-multimodal.test.ts` (v0.34.1.0, #875) — 11-case unit suite for the gateway's openai-compatible multimodal path: happy-path single + multi-input embedding, unauthenticated proxy mode, dimension-mismatch guard (D12; throws `AIConfigError` with model id + observed + expected pre-storage), default-dim fallback when recipe declares `default_dims`, HTTP 401 / 400 / malformed-JSON / non-array error paths, plus a regression test that the existing Voyage `/multimodalembeddings` recipe still routes through its dedicated path (not the openai-compatible one). Hermetic via the `__setEmbedTransportForTests` seam. +- `test/serve-stdio-lifecycle.test.ts` (extended v0.34.1.0, #870) — adds 3 new cases for the `MCP_STDIO=1` env guard: stdin EOF does NOT trigger shutdown when the env is set, SIGTERM still does (guard scope is correct), unset env preserves the pre-v0.34 CLI lifecycle. Exercises the `ServeOptions.mcpStdio?: boolean` test seam directly so tests don't mutate `process.env`. +- `test/oauth.test.ts` (extended v0.34.1.0, #909) — 5 new cases for the PKCE DCR public-client gate: `registerClient` with `token_endpoint_auth_method: "none"` returns no `client_secret` field on the public client, default `client_secret_post` clients still get the one-time-reveal secret, `getClient` NULL→undefined normalization so the SDK's clientAuth path accepts public clients, full PKCE `/authorize` → `/token` round-trip against a public client (no client_secret presented), and a regression test that the public-vs-confidential branch doesn't break confidential client `client_secret_post` exchange. - Tier 2 (`skills.test.ts`) requires OpenClaw + API keys, runs nightly in CI - If `.env.testing` doesn't exist in this directory, check sibling worktrees for one: `find ../ -maxdepth 2 -name .env.testing -print -quit` and copy it here if found. diff --git a/README.md b/README.md index d86cc71..482c9bd 100644 --- a/README.md +++ b/README.md @@ -139,7 +139,9 @@ pick scopes, save the credentials shown once in the reveal modal. Programmatic registration via `oauthProvider.registerClientManual(...)` and the `gbrain auth register-client` CLI are also available. -- **OAuth 2.1 via the MCP SDK** — client credentials (machine-to-machine: Perplexity, Claude), authorization code + PKCE (browser-based: ChatGPT), refresh token rotation, revocation, protected resource metadata. Optional Dynamic Client Registration behind `--enable-dcr` (DCR redirect_uris must be `https://` or loopback per RFC 6749 §3.1.2.1). +- **OAuth 2.1 via the MCP SDK** — client credentials (machine-to-machine: Perplexity, Claude), authorization code + PKCE (browser-based: ChatGPT), refresh token rotation, revocation, protected resource metadata. PKCE-only public clients (`token_endpoint_auth_method: "none"`) register without a secret per RFC 7591 §3.2.1 (v0.34). Optional Dynamic Client Registration behind `--enable-dcr` (DCR redirect_uris must be `https://` or loopback per RFC 6749 §3.1.2.1). +- **Source-scoped OAuth clients (v0.34)** — `gbrain auth register-client my-agent --source dept-x` ties the client's write authority to one source; read paths only return rows matching that source. `--federated-read S1,S2,S3` adds an orthogonal read-scope axis for shared brains (departments writing to one canon while reading the union). Pre-v0.34 clients are backfilled to `source_id='default'` on upgrade. +- **Loopback default for `serve --http` (v0.34)** — listens on `127.0.0.1` unless `--bind 0.0.0.0` (or a specific interface IP). Personal-laptop installs no longer publish the brain to the LAN by accident. A stderr WARN fires when `--public-url` is set without `--bind` so the operator sees the binding before the first request. - **Scoped operations** — 30 operations tagged `read | write | admin`. `sync_brain` and `file_upload` are `localOnly`, rejected over HTTP. - **React admin dashboard** — 7 screens baked into the binary (~65KB gzip). Live SSE activity feed, agents table, credential reveal, filterable request log, per-client config export. - **Legacy bearer tokens still work** — pre-v0.26 `gbrain auth create` tokens continue to authenticate as `read+write+admin`. v0.22.7's simpler `src/mcp/http-transport.ts` path stays compiled in for backward compat callers; v0.26+ deployments use the OAuth-aware `serve-http.ts`. @@ -788,12 +790,16 @@ ADMIN [--skip=] [--json] gbrain serve MCP server (stdio) gbrain serve --http [--port 3131] HTTP MCP server with OAuth 2.1 + admin dashboard + [--bind HOST] (v0.34: default 127.0.0.1; pass + --bind 0.0.0.0 for LAN/remote access) [--token-ttl 3600] [--enable-dcr] [--public-url URL] [--log-full-params] gbrain auth create|list|revoke|test Legacy bearer token management gbrain auth register-client Register an OAuth 2.1 client --grant-types client_credentials,authorization_code --scopes "read write admin" + --source v0.34: write authority for source-scoped clients + --federated-read v0.34: read scope across multiple sources gbrain auth revoke-client Revoke an OAuth 2.1 client (cascade purges active tokens + auth codes via FK CASCADE) # OAuth 2.1 clients can also be registered from the /admin dashboard or diff --git a/SECURITY.md b/SECURITY.md index a271c63..7d89947 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -68,6 +68,16 @@ The built-in HTTP transport ships with several layers of hardening on by default. All env vars below are optional; the defaults are intentionally conservative. +### Bind address (v0.34: loopback by default) + +`gbrain serve --http` listens on `127.0.0.1` by default. Personal-laptop +installs cannot accidentally publish the brain to the LAN. Self-hosted +deployments that need remote access pass `--bind 0.0.0.0` (all +interfaces) or `--bind ` (specific NIC). A stderr WARN +fires when `--public-url` is set without `--bind` so the operator sees +the binding before the first request — common cause of "ngrok forwards +to me but the agent can't reach the upstream" misconfigurations. + ### Postgres-only `gbrain serve --http` requires a Postgres engine. PGLite is local-only by @@ -125,9 +135,11 @@ GBRAIN_HTTP_TRUST_PROXY=1 gbrain serve --http --port 8787 **both** of these are true: 1. gbrain is reachable only via a trusted reverse proxy (not directly - exposed to the internet on the configured port). The simplest - guarantee is to bind gbrain to `127.0.0.1` or a private interface - and have the proxy forward to it. + exposed to the internet on the configured port). As of v0.34 + `gbrain serve --http` binds `127.0.0.1` by default, so the + reverse-proxy-only posture is the out-of-the-box shape; only + override with `--bind 0.0.0.0` (or a specific interface IP) when + gbrain itself needs to accept remote connections directly. 2. The proxy strips any client-supplied `X-Forwarded-For` and `X-Real-IP` headers, then sets them itself. (nginx with `proxy_set_header X-Forwarded-For $remote_addr` does this; Cloudflare and most cloud diff --git a/TODOS.md b/TODOS.md index 46e5950..786ccf5 100644 --- a/TODOS.md +++ b/TODOS.md @@ -1,5 +1,21 @@ # TODOS +## MCP fix wave follow-ups (v0.34.1) + +- [ ] **v0.34.x: Source-scope `takes_*` ops (pre-existing leak surfaced during v0.34.1 adversarial review).** `takes_list`, `takes_search`, `takes_scorecard`, `takes_calibration` in `src/core/operations.ts:1248-1335` thread `ctx.takesHoldersAllowList` but never `ctx.sourceId`. An auth'd OAuth client scoped to `source_id='canon-a'` can call `takes_list --page_slug=foo` (slug in `canon-b`) and read takes attached to foreign-source pages. Pre-existing, not introduced by v0.34.1, but the wave was framed as "P0 source-isolation seal on the read path" and `takes_*` surfaces were missed. Fix: extend `TakesListOpts` in `src/core/engine.ts:186` with `sourceId?: string` + `sourceIds?: string[]`; thread `sourceScopeOpts(ctx)` at each op handler; engine `listTakes`/`searchTakes` filter via the `pages` JOIN. + +- [ ] **v0.34.x: Extend `sourceScopeOpts(ctx)` to the 14 read-side ops PR #861 didn't touch.** `get_page`, `get_tags`, `get_links`, `get_backlinks`, `get_timeline`, `list_files`, `get_file`, and the four `takes_*` ops (above) still use the v0.31.8-era `const sourceOpts = ctx.sourceId ? { sourceId: ctx.sourceId } : {}` pattern. NOT a leak (scalar `ctx.sourceId` IS threaded), but federated_read (#876, `ctx.auth?.allowedSources`) is silently dropped. A "WeCare L3 dept" client gets correct federated results from `search`/`query`/`list_pages`/`traverse_graph`/`find_experts` but only sees its scalar `source_id` for `get_page`/`get_tags`/etc. Fix: route all 14 sites through `sourceScopeOpts(ctx)`. + +- [ ] **v0.34.x: Migration v60 idempotency guard against `--force-retry` race with v64.** `gbrain apply-migrations --force-retry 58` after v64 has already run will re-install the FK with `ON DELETE SET NULL`, silently downgrading the v64 RESTRICT posture. Probability low (operator has to explicitly force-retry 58) but failure mode is invisible. Fix: v60 should probe `pg_constraint.confdeltype` before re-adding and refuse to clobber `'r'` (RESTRICT) with `'n'` (SET NULL). + +- [ ] **v0.34.x: `embedMultimodalOpenAICompat` batching + partial-failure handling.** `src/core/ai/gateway.ts:1180-1255` sends one HTTP request per input. Multi-input callers (10 images) get 10 sequential round-trips with no parallelism; a 401 on input #5 throws and discards inputs #1-#4's already-computed embeddings (wasted spend, no surfacing of the partial array). Voyage's existing path batches. Fix: batch via the provider's `input: [...]` array shape; on partial failure, return successful embeddings + failed-index array. + +- [ ] **v0.34.x: Doctor check `oauth_orphan_source_id`** — surfaces OAuth clients whose source_id was nulled by the v60 D10 silent-widen path (`GBRAIN_ACCEPT_SILENT_WIDEN=1`). Closes the observability gap from v0.34.1's D4 decision. Sibling to the `rls_event_trigger` check pattern in `src/commands/doctor.ts`. + +- [ ] **v0.34.x: `gbrain sources purge` FK error UX.** Post-v0.34, deleting a source is refused if any oauth_client references it (v64 ON DELETE RESTRICT). The CLI currently surfaces the raw Postgres FK violation. Fix: pre-check via `SELECT client_id, client_name FROM oauth_clients WHERE source_id = $1`, print "N OAuth clients reference this source: ... Revoke first via `gbrain auth revoke-client `." Mirrors `assessDestructiveImpact` in destructive-guard.ts (v0.26.5). + +- [ ] **v0.34.x: `hybrid.ts:223` explicit-pick refactor.** The SearchOpts rebuild manually picks fields from HybridSearchOpts. This is the bug shape that caused the original v0.34.1 P0 leak — a new SearchOpts field is silently dropped if not manually added here. The wave added `sourceId` + `sourceIds` to the pick; future fields will keep hitting this footgun. Fix: refactor to spread + TypeScript `Pick<>` helper that narrows HybridSearchOpts → SearchOpts type-safely. + ## functional-area-resolver follow-ups (v0.32.3.0) - [ ] **v0.33.x: Dogfood `functional-area-resolver` on gbrain's own `skills/RESOLVER.md`** when it crosses ~12KB (currently 8KB). Apply the pattern to the Operational section first (largest). Filed during v0.32.3.0 CEO review. diff --git a/VERSION b/VERSION index cedba06..a68a9c7 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.34.0.0 +0.34.1.0 \ No newline at end of file diff --git a/docs/architecture/topologies.md b/docs/architecture/topologies.md index 8e4e0dd..8782eea 100644 --- a/docs/architecture/topologies.md +++ b/docs/architecture/topologies.md @@ -120,10 +120,19 @@ at the remote host. `gbrain doctor` runs a dedicated thin-client check set ```bash gbrain init --supabase # or --pglite, doesn't matter -gbrain serve --http --port 3001 # exposes /mcp + OAuth +gbrain serve --http --port 3001 --bind 0.0.0.0 # v0.34: bind explicitly for remote access + # (defaults to 127.0.0.1 since v0.34) gbrain auth register-client neuromancer \ --grant-types client_credentials \ --scopes read,write,admin # admin needed for ping/doctor + +# v0.34: source-scoped client (write to one source, federate reads across +# multiple sources). Omit both flags for a v0.33-compatible super-client. +gbrain auth register-client neuromancer-dept \ + --grant-types client_credentials \ + --scopes read,write \ + --source dept-x \ + --federated-read dept-x,shared,parent-canon ``` The `register-client` command prints a `client_id` and `client_secret`. diff --git a/docs/mcp/DEPLOY.md b/docs/mcp/DEPLOY.md index 4f508bf..deec780 100644 --- a/docs/mcp/DEPLOY.md +++ b/docs/mcp/DEPLOY.md @@ -117,6 +117,24 @@ gbrain auth register-client perplexity \ --scopes "read write" ``` +**v0.34 — source-scoped clients.** Multi-source brains can scope a client's +write authority to one source and its read scope to a curated set with the +new `--source` and `--federated-read` flags: + +```bash +gbrain auth register-client dept-x-agent \ + --grant-types client_credentials \ + --scopes "read write" \ + --source dept-x \ + --federated-read dept-x,shared,parent-canon +``` + +`--source` controls the write authority — `put_page` / `add_link` / etc only +land in `dept-x`. `--federated-read` controls the read axis independently; +queries return rows from any of the listed sources. Omit both flags for the +v0.33-compatible super-client shape. Pre-v0.34 clients are backfilled to +`source_id='default'` on `gbrain upgrade`. + Host-repo wrappers can register programmatically: ```ts @@ -133,6 +151,18 @@ start the server with `--enable-dcr`. DCR is off by default. ### 3. Expose the server +**v0.34 — bind explicitly.** `gbrain serve --http` defaults to `127.0.0.1`. +To accept connections from the ngrok tunnel (or any non-loopback source), +restart with `--bind`: + +```bash +gbrain serve --http --port 3131 --bind 0.0.0.0 --public-url https://your-brain.ngrok.app +``` + +When `--public-url` is set without `--bind`, a stderr WARN fires at +startup so the misconfiguration ("the tunnel is up but my agent gets +ECONNREFUSED") is loud. + ```bash brew install ngrok ngrok config add-authtoken YOUR_TOKEN diff --git a/llms-full.txt b/llms-full.txt index 2cbcf1c..eae6f6b 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -148,8 +148,8 @@ strict behavior when unset. ## Key files -- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (v0.23 glob matcher: `/*` matches recursive children; bare `` matches exact only). `OperationContext.remote` flags untrusted callers; `OperationContext.allowedSlugPrefixes` (v0.23) is the trusted-workspace allow-list set by the dream cycle. `put_page` enforces: when `viaSubagent` and `allowedSlugPrefixes` is set, slug must match the allow-list; else the legacy `wiki/agents//...` namespace check applies. Auto-link enabled for trusted-workspace writes (skipped only when `remote=true && !trustedWorkspace`). As of v0.26.0, every `Operation` also carries `scope?: 'read' | 'write' | 'admin'` + `localOnly?: boolean`. All ops are annotated; `sync_brain`, `file_upload`, `file_list`, and `file_url` are `admin + localOnly` (rejected over HTTP). `OperationContext.auth?: AuthInfo` is threaded through HTTP dispatch for scope enforcement in `serve-http.ts` before the op runs. **v0.26.9 (D12 + F7b):** `OperationContext.remote` is now a REQUIRED field in the TypeScript type — the compiler is the first defense against transports that forget to set it. Four trust-boundary call sites (`put_page` allowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) flipped from falsy-default (`!ctx.remote`) to fail-closed semantics (`ctx.remote === false` for "trusted-only" sites and `ctx.remote !== false` for "untrust unless explicit-false"). Anything that isn't strictly `false` is now treated as remote. Closed an HTTP MCP shell-job RCE: a `read+write`-scoped OAuth token could submit `shell` jobs because the HTTP request handler's literal context skipped `remote: true` and `submit_job`'s protected-name guard saw a falsy undefined. Stdio MCP set the field correctly via dispatch.ts; HTTP inlined a parallel context-builder for several releases and lost it. -- `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput` / `TimelineBatchInput` for the v0.12.1 bulk-insert API (`addLinksBatch` / `addTimelineEntriesBatch`). As of v0.13.1, `BrainEngine` has a `readonly kind: 'postgres' | 'pglite'` discriminator so migrations (`src/core/migrate.ts`) and other consumers can branch on engine without `instanceof` + dynamic imports. **v0.29:** four new methods — `batchLoadEmotionalInputs(slugs?)` (CTE-shaped read with per-table aggregates so a page × N tags × M takes never produces N×M rows), `setEmotionalWeightBatch(rows)` (`UPDATE FROM unnest($1::text[], $2::text[], $3::real[])` composite-keyed on `(slug, source_id)` for multi-source safety), `getRecentSalience(opts)`, `findAnomalies(opts)`. `PageFilters` extended with `sort?: 'updated_desc' | 'updated_asc' | 'created_desc' | 'slug'` + `PAGE_SORT_SQL` whitelist consumed by both engines (was hardcoded `ORDER BY updated_at DESC`). **v0.32.8 (PR #860):** new `listAllPageRefs(): Promise>` ordered by `(source_id, slug)`. Cheap cross-source enumeration for hot loops on large brains — replaces the `getAllSlugs()→getPage(slug)` N+1 pattern in extract-takes, extract, integrity, which silently defaulted to `source_id='default'` for non-default-source pages. Implementation parity across postgres-engine.ts + pglite-engine.ts. Pinned by `test/e2e/multi-source-bug-class.test.ts`. +- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (v0.23 glob matcher: `/*` matches recursive children; bare `` matches exact only). `OperationContext.remote` flags untrusted callers; `OperationContext.allowedSlugPrefixes` (v0.23) is the trusted-workspace allow-list set by the dream cycle. `put_page` enforces: when `viaSubagent` and `allowedSlugPrefixes` is set, slug must match the allow-list; else the legacy `wiki/agents//...` namespace check applies. Auto-link enabled for trusted-workspace writes (skipped only when `remote=true && !trustedWorkspace`). As of v0.26.0, every `Operation` also carries `scope?: 'read' | 'write' | 'admin'` + `localOnly?: boolean`. All ops are annotated; `sync_brain`, `file_upload`, `file_list`, and `file_url` are `admin + localOnly` (rejected over HTTP). `OperationContext.auth?: AuthInfo` is threaded through HTTP dispatch for scope enforcement in `serve-http.ts` before the op runs. **v0.26.9 (D12 + F7b):** `OperationContext.remote` is now a REQUIRED field in the TypeScript type — the compiler is the first defense against transports that forget to set it. Four trust-boundary call sites (`put_page` allowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) flipped from falsy-default (`!ctx.remote`) to fail-closed semantics (`ctx.remote === false` for "trusted-only" sites and `ctx.remote !== false` for "untrust unless explicit-false"). Anything that isn't strictly `false` is now treated as remote. Closed an HTTP MCP shell-job RCE: a `read+write`-scoped OAuth token could submit `shell` jobs because the HTTP request handler's literal context skipped `remote: true` and `submit_job`'s protected-name guard saw a falsy undefined. Stdio MCP set the field correctly via dispatch.ts; HTTP inlined a parallel context-builder for several releases and lost it. **v0.34.1.0 (#861 + #876):** new helper `sourceScopeOpts(ctx)` encodes the precedence ladder for source-scoped reads — federated array (`ctx.auth.allowedSources`) wins over scalar (`ctx.sourceId` / `ctx.auth.sourceId`) over nothing. Every read-side op handler routes through it so future ops can't silently drift from the canonical v0.31.8 thread. Closes the source-isolation leak on the read path: a `read+write`-scoped OAuth client bound to `--source dept-x` no longer sees rows from neighboring sources via `search` / `query` / `list_pages` / `get_page` / `find_experts` / `query`'s image path. +- `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput` / `TimelineBatchInput` for the v0.12.1 bulk-insert API (`addLinksBatch` / `addTimelineEntriesBatch`). As of v0.13.1, `BrainEngine` has a `readonly kind: 'postgres' | 'pglite'` discriminator so migrations (`src/core/migrate.ts`) and other consumers can branch on engine without `instanceof` + dynamic imports. **v0.29:** four new methods — `batchLoadEmotionalInputs(slugs?)` (CTE-shaped read with per-table aggregates so a page × N tags × M takes never produces N×M rows), `setEmotionalWeightBatch(rows)` (`UPDATE FROM unnest($1::text[], $2::text[], $3::real[])` composite-keyed on `(slug, source_id)` for multi-source safety), `getRecentSalience(opts)`, `findAnomalies(opts)`. `PageFilters` extended with `sort?: 'updated_desc' | 'updated_asc' | 'created_desc' | 'slug'` + `PAGE_SORT_SQL` whitelist consumed by both engines (was hardcoded `ORDER BY updated_at DESC`). **v0.32.8 (PR #860):** new `listAllPageRefs(): Promise>` ordered by `(source_id, slug)`. Cheap cross-source enumeration for hot loops on large brains — replaces the `getAllSlugs()→getPage(slug)` N+1 pattern in extract-takes, extract, integrity, which silently defaulted to `source_id='default'` for non-default-source pages. Implementation parity across postgres-engine.ts + pglite-engine.ts. Pinned by `test/e2e/multi-source-bug-class.test.ts`. **v0.34.1.0 (#861):** `SearchOpts` + `PageFilters` add `sourceIds?: string[]` for the federated read axis; both engines apply `WHERE source_id = ANY($N::text[])` when the array is set and preserve the scalar `sourceId` fast path when unset. `traverseGraph(slug, depth, opts?)` and `traversePaths(slug, opts?)` accept `opts.sourceId` / `opts.sourceIds` so graph walks respect the caller's scope. - `src/core/engine-factory.ts` — Engine factory with dynamic imports (`'pglite'` | `'postgres'`) - `src/core/pglite-engine.ts` — PGLite (embedded Postgres 17.5 via WASM) implementation, all 40 BrainEngine methods. `addLinksBatch` / `addTimelineEntriesBatch` use multi-row `unnest()` with manual `$N` placeholders. As of v0.13.1, `connect()` wraps `PGlite.create()` in a try/catch that emits an actionable error naming the macOS 26.3 WASM bug (#223) and pointing at `gbrain doctor`; the lock is released on failure so the next process can retry cleanly. v0.22.0: `searchKeyword` and `searchKeywordChunks` multiply `ts_rank` by the source-factor CASE expression at the chunk-grain level; `searchVector` becomes a two-stage CTE — inner CTE keeps `ORDER BY cc.embedding <=> vec` so HNSW stays usable, outer SELECT re-ranks by `raw_score * source_factor`. Inner LIMIT scales with offset to preserve pagination contract. As of v0.22.6.1, `initSchema()` calls `applyForwardReferenceBootstrap()` BEFORE replaying SCHEMA_SQL — probes for the specific forward-referenced state the embedded schema blob needs (`pages.source_id`, `links.link_source`, `links.origin_page_id`, `content_chunks.symbol_name`, `content_chunks.language`, `sources` FK target table) and adds only what's missing. Closes the upgrade-wedge bug class that bit users 10+ times across 6 schema versions over 2 years (#239/#243/#266/#357/#366/#374/#375/#378/#395/#396). No-op on fresh installs and modern brains. - `src/core/pglite-schema.ts` — PGLite-specific DDL (pgvector, pg_trgm, triggers) @@ -202,7 +202,7 @@ strict behavior when unset. - `src/core/embedding.ts` — OpenAI text-embedding-3-large, batch, retry, backoff. **v0.28.7:** `BATCH_SIZE` reverted 50→100 — the original Voyage safety guard halved OpenAI throughput on every page. Per-recipe pre-split + recursive halving + adaptive shrink-on-miss now live in the gateway, so the outer paginator goes back to its original purpose: progress-callback granularity, not batch protection. - `src/core/ai/dims.ts` (v0.33.1.1, PR #962 + #866) — per-provider `providerOptions` resolver for embed-time dimension passthrough. The single source of truth for "which provider needs which knob to produce `vector(N)`". Exports `dimsProviderOptions(implementation, modelId, dims)` (called by `embed()` in `gateway.ts`), `VOYAGE_OUTPUT_DIMENSION_MODELS` (private const — the 7 hosted Voyage models that accept `output_dimension`: `voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-3-large`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-code-3` — nano deliberately excluded), `VOYAGE_VALID_OUTPUT_DIMS = [256, 512, 1024, 2048] as const`, `supportsVoyageOutputDimension(modelId)`, and `isValidVoyageOutputDim(dims)`. **Voyage path uses the SDK-supported `dimensions` field** (`{ openaiCompatible: { dimensions: N } }`), NOT Voyage's `output_dimension` wire-key — the existing `voyageCompatFetch` shim in `gateway.ts:541` translates `dimensions → output_dimension` before the HTTP body is built. The reverse (sending `output_dimension` from here) was the v0.33.1.0 bug class: the AI SDK's openai-compatible adapter doesn't recognize the wire-key so it was silently dropped, Voyage returned its default 1024-dim, and the gateway dimension check threw on every embed call. Runtime guard: when a Voyage flexible-dim model is configured with `dims` outside `VOYAGE_VALID_OUTPUT_DIMS`, throws `AIConfigError` with a paste-ready `gbrain config set embedding_dimensions <256|512|1024|2048>` hint at the embed boundary — fail-loud instead of opaque Voyage HTTP 400. Most common trigger: `embedding_model: voyage:voyage-4-large` configured without `embedding_dimensions` (falls back to `DEFAULT_EMBEDDING_DIMENSIONS=1536`, an OpenAI default not a Voyage one). Eva (@100yenadmin) shipped the wire-key fix in #866; Codex P3 follow-up landed the validator + valid-dims allowlist in #962. - `src/core/ai/types.ts` — provider/recipe types. **v0.28.7 (#680):** `EmbeddingTouchpoint` extended with optional `chars_per_token` (default 4 chars/token, matching OpenAI tiktoken on English) and `safety_factor` (default 0.8, budget-utilization ceiling). Both consulted only when `max_batch_tokens` is also set. Voyage declares `chars_per_token=1` + `safety_factor=0.5` to handle dense payloads (CJK/JSON/base64) that overshoot tiktoken. The pre-split budget is `max_batch_tokens × safety_factor / chars_per_token`. **v0.28.11 (#719):** `EmbeddingTouchpoint.multimodal_models?: string[]` model-level allow-list for recipes that mix text-only + multimodal models under one touchpoint (Voyage's 12 models share `supports_multimodal: true` but only `voyage-multimodal-3` accepts `/multimodalembeddings`). When omitted, recipe-level `supports_multimodal` is sufficient. `AIGatewayConfig.embedding_multimodal_model?: string` lets `embedMultimodal()` route to a different model than `embedding_model` — brains using OpenAI for text can use Voyage for images without flipping the primary embedding pipeline. -- `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state. **v0.33.1.1 (#962, Codex P3 follow-up):** new exported `VoyageResponseTooLargeError` tagged class at the top of the file. `voyageCompatFetch`'s two OOM-defense caps (Layer 1 Content-Length check at `:595`, Layer 2 per-embedding base64 cap at `:619`) now throw `VoyageResponseTooLargeError` instead of a generic `Error`. The inbound response-rewriter's surrounding try/catch (which intentionally swallows parse failures so misshaped Voyage responses fall through to the SDK's JSON parser) checks `instanceof VoyageResponseTooLargeError` and rethrows. Pre-fix, the Layer 2 throw was silently swallowed and the oversized response returned to the AI SDK anyway — Layer 2 was theatrical. Source-shape regression assertion in `test/voyage-response-cap.test.ts` pins the `instanceof ⇒ throw err` line. +- `src/core/ai/gateway.ts` — unified seam for every AI call. **v0.28.7 (#680):** module-scoped `_embedTransport` defaulting to AI SDK `embedMany`, with `__setEmbedTransportForTests(fn)` test seam so tests drive the public `embed()` function with a stubbed transport instead of probing private helpers. `splitByTokenBudget` and `isTokenLimitError` are now exported `@internal` — pure functions reused directly by the test file. Module-level `_shrinkState: Map` halves the recipe's effective `safety_factor` on token-limit miss (floor 0.05) and heals back ×1.5 toward the ceiling after `SHRINK_HEAL_AFTER=10` consecutive successes. `configureGateway()` walks every registered recipe at construction time and emits a once-per-process stderr warning for any embedding touchpoint missing `max_batch_tokens` (excluding the canonical OpenAI fast-path recipe). `resetGateway()` clears `_shrinkState`, the warned-set, and restores the real transport. ASCII flow diagram embedded in the `embed()` JSDoc covers the routing decision, recursion + halving, and shrinkState lifecycle. **v0.28.11 (#719):** `embedMultimodal()` reads `cfg.embedding_multimodal_model` first (falls back to `cfg.embedding_model` for single-model setups). After the existing recipe-level `supports_multimodal` fast-fail, validates the resolved model against `touchpoint.multimodal_models` when declared — closes the Voyage-text-only-model-into-multimodal-endpoint footgun before any HTTP call (Codex F1 from PR review). New `getMultimodalModel()` accessor mirrors `getEmbeddingModel` / `getChatModel` so doctor and integration tests can read the gateway state. **v0.33.1.1 (#962, Codex P3 follow-up):** new exported `VoyageResponseTooLargeError` tagged class at the top of the file. `voyageCompatFetch`'s two OOM-defense caps (Layer 1 Content-Length check at `:595`, Layer 2 per-embedding base64 cap at `:619`) now throw `VoyageResponseTooLargeError` instead of a generic `Error`. The inbound response-rewriter's surrounding try/catch (which intentionally swallows parse failures so misshaped Voyage responses fall through to the SDK's JSON parser) checks `instanceof VoyageResponseTooLargeError` and rethrows. Pre-fix, the Layer 2 throw was silently swallowed and the oversized response returned to the AI SDK anyway — Layer 2 was theatrical. Source-shape regression assertion in `test/voyage-response-cap.test.ts` pins the `instanceof ⇒ throw err` line. **v0.34.1.0 (#875):** new `embedMultimodalOpenAICompat()` routes recipes with `implementation: 'openai-compatible'` (LiteLLM, Anyscale, vLLM, Gemini multimodal via proxy) through the standard `/embeddings` endpoint with content arrays carrying `image_url` entries. The pre-existing Voyage `/multimodalembeddings` path is unchanged; the gateway selects by recipe `implementation` tag. Runtime dimension validation throws `AIConfigError` (with model id + observed + expected) before the vector reaches storage when the provider returns a width that doesn't match the recipe's `default_dims` or the brain's `embedding_dimensions` config — no more cryptic `vector dimension mismatch` at INSERT time. Pinned by 11 cases in `test/openai-compat-multimodal.test.ts`. - `src/core/ai/recipes/voyage.ts` — Voyage AI openai-compatible recipe. **v0.28.7 (#680):** declares `chars_per_token=1` + `safety_factor=0.5` so the gateway pre-splits Voyage batches at a 60K-character budget (50% of 120K-token cap with the dense-tokenizer ratio). Closes the v0.27 backfill loop where ~26% of the corpus stayed un-embedded because tiktoken-grounded budgeting silently undercounted Voyage's actual token usage. **v0.28.11 (#719):** declares `multimodal_models: ['voyage-multimodal-3']` so the gateway rejects text-only Voyage models pointed at the multimodal endpoint with a clear `AIConfigError` instead of waiting for Voyage's HTTP 400. **v0.33.1.1 (#962, fixup):** recipe docstring at `:7-16` tightened to name the seven hosted flexible-dim models that accept `output_dimension` explicitly (`voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-3-large`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-code-3`) and call out that `voyage-4-nano` is the open-weight variant listed separately by Voyage as fixed 1024-dim — does NOT accept the parameter. The "all v4 variants are flexible" misread is what caused the original PR to include nano in `VOYAGE_OUTPUT_DIMENSION_MODELS`; the negative regression assertion in `test/ai/gateway.test.ts` (`dimsProviderOptions` returns `undefined` for `voyage-4-nano`) pins the contract. - `src/core/ai/recipes/anthropic.ts` — Anthropic recipe (chat + expansion touchpoints). **v0.31.12:** chat and expansion `models:` lists drop the v0.31.6 phantom `claude-sonnet-4-6-20250929` date suffix — canonical id is `claude-sonnet-4-6`. The wrong-direction alias `claude-sonnet-4-6 → claude-sonnet-4-6-20250929` is removed; a reverse alias `claude-sonnet-4-6-20250929 → claude-sonnet-4-6` keeps stale user configs working (rescues `facts.extraction_model` and `models.dream.synthesize` set by v0.31.6 installs). Recipe-shape regression pinned by `test/anthropic-model-ids.test.ts` (6 cases, verbatim cherry-pick of PR #830 plus the reverse-alias rescue case). - `src/core/anthropic-pricing.ts` — Single source of truth for Anthropic model pricing (per-MTok input/output). **v0.31.12:** Opus 4.7 corrected from `$15/$75` to `$5/$25` (the old number was from Opus 4 generation, never refreshed when 4.7 shipped); Opus 4.6 also corrected. Consumed by `src/core/budget-meter.ts` and `src/core/cross-modal-eval/runner.ts` — the cross-modal estimator now reads `ANTHROPIC_PRICING` for Anthropic models instead of duplicating the table, killing the v0.31.6 drift bug class. @@ -261,15 +261,15 @@ strict behavior when unset. - `src/commands/jobs.ts` — `gbrain jobs` CLI subcommands + `gbrain jobs work` daemon. **v0.28.1:** `case 'work'` now wraps `worker.start()` in try/finally and owns engine lifecycle — calls `engine.disconnect()` on shutdown with loud error logging on failure. Replaces the prior call inside `MinionWorker.start()` (which violated engine ownership: the worker disconnected an engine it didn't own, and clobbered the module-level singleton on PostgresEngine via the now-fixed idempotency bug). Pool slots now free immediately on shutdown instead of waiting for TCP keepalive (~minutes). v0.13.1 surfaces the full `MinionJobInput` retry/backoff/timeout/idempotency surface as first-class CLI flags on `jobs submit`: `--max-stalled`, `--backoff-type fixed|exponential`, `--backoff-delay`, `--backoff-jitter`, `--timeout-ms`, `--idempotency-key`. `jobs smoke --sigkill-rescue` is the opt-in regression guard for #219. v0.16 wires `registerBuiltinHandlers` to always register `subagent` + `subagent_aggregator` (no env flag — `ANTHROPIC_API_KEY` is the natural cost gate, trust is via `PROTECTED_JOB_NAMES`) and loads `GBRAIN_PLUGIN_PATH` plugins at worker startup with a loud startup-line per plugin. `shell` handler still gated by `GBRAIN_ALLOW_SHELL_JOBS=1` (RCE surface, separate concern). v0.22.10 (#521): the `autopilot-cycle` handler now forwards `job.data.phases` to `runCycle` (was previously discarded — caller-supplied phase selection silently became a full cycle). Phases are validated against `ALL_PHASES` from `src/core/cycle.ts`; invalid names are filtered out and an empty/missing array falls back to the default 6-phase cycle. v0.22.13 (PR #490 CODEX-1+CODEX-4): `sync` handler now resolves `sourceId` at entry by looking up `sources.local_path` (mirrors `cycle.ts:480`'s autopilot fix from PR #475) so multi-source brains read the per-source `last_commit` anchor instead of the global config key. Concurrency routed through the shared `autoConcurrency()` policy in `src/core/sync-concurrency.ts` instead of the prior hardcoded `4`; PGLite stays serial. `noEmbed` default is `true` (embed is a separate job — submit `gbrain embed --stale` after sync, or rely on the autopilot cycle's embed phase). - `src/commands/features.ts` — `gbrain features --json --auto-fix`: usage scan + feature adoption salesman - `src/commands/autopilot.ts` — `gbrain autopilot --install`: self-maintaining brain daemon (sync+extract+embed). **v0.28.1:** consumes `detectTini()` from `src/core/minions/spawn-helpers.ts` and resolves it once at startup instead of per worker respawn (was paying an `execFileSync` cost on every restart). -- `src/mcp/server.ts` — MCP stdio server (generated from operations). v0.22.7: tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path. +- `src/mcp/server.ts` — MCP stdio server (generated from operations). v0.22.7: tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path. **v0.34.1.0 (#870):** stdin `'end'` / `'close'` shutdown hooks are skipped when `process.env.MCP_STDIO === '1'`. Gateway-piped stdio MCP wrappers (OpenClaw's `bundle-mcp`, similar) pipe the JSON-RPC handshake then close their stdin half; pre-fix this killed the server before the first tool call landed. Signal handlers (SIGTERM / SIGINT / SIGHUP) and the parent-process watchdog still cover legitimate disconnects. `src/commands/serve.ts` exposes `ServeOptions.mcpStdio?: boolean` as a test seam so the runtime guard is exercisable without process.env mutation. Pinned by `test/serve-stdio-lifecycle.test.ts`. - `src/mcp/dispatch.ts` (v0.22.7) — Shared tool-call dispatch consumed by both stdio (`server.ts`) and HTTP transports. Exports `dispatchToolCall(engine, name, params, opts)`, `buildOperationContext(engine, params, opts)`, and `validateParams(op, params)`. Single source of truth for `(ctx, params)` handler arg order and the 5-field `OperationContext` shape (engine + config + logger + dryRun + remote). Defaults to `remote: true` (untrusted); local CLI callers pass `remote: false`. Closed F1/F2/F3 drift bugs in the original v0.22.5 HTTP transport. **v0.26.9 (F8):** adds `summarizeMcpParams(opName, params)` — privacy-preserving redactor for `mcp_request_log` and the admin SSE feed. Returns `{redacted, kind, declared_keys, unknown_key_count, approx_bytes}`. Intersects submitted top-level keys against the operation's declared `params` allow-list (declared keys preserved as a sorted array for debug visibility; unknown keys counted but never named, closing the attacker-controlled-key-name leak). Byte counts bucketed up to nearest 1KB so an attacker can't binary-search secret-content sizes via repeated probes. Operators on a personal laptop who want raw payload visibility opt back in with `gbrain serve --http --log-full-params` (loud stderr warning at startup). Canonical helper — new logging code paths route through it rather than `JSON.stringify(params)`. - `src/mcp/rate-limit.ts` (v0.22.7) — Bounded-LRU token-bucket limiter. `buildDefaultLimiters()` returns the two-bucket pipeline: pre-auth IP (30/60s, fires BEFORE the DB lookup so brute-force load against `access_tokens` is actually capped) + post-auth token-id (60/60s). Tracks `lastTouchedMs` separately from `lastRefillMs` so an exhausted key can't be reset by hammering past the TTL. LRU cap bounds memory under attacker-controlled key growth. -- `src/commands/serve-http.ts` (v0.26.0) — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. Started via `gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--log-full-params]`. Supersedes the v0.22.7 `src/mcp/http-transport.ts` simple bearer-auth path. Combines MCP SDK's `mcpAuthRouter` (authorize / token / register / revoke endpoints), a custom `client_credentials` handler (SDK's token endpoint throws `UnsupportedGrantTypeError` for CC; the custom handler runs BEFORE the router and falls through for `auth_code` / `refresh_token`), `requireBearerAuth` middleware for `/mcp` with scope enforcement before op dispatch, `localOnly` rejection, and `express-rate-limit` at 50 req / 15 min on `/token`. Serves the built admin SPA from `admin/dist/` with SPA fallback. `/admin/events` SSE endpoint broadcasts every MCP request to connected admin browsers. `cookie-parser` middleware wired (Express 5 has no built-in). Startup logging prints port, engine, configured issuer URL (honors `--public-url`), registered-client count, DCR status, and admin bootstrap token. **v0.26.9 hardening pass:** F7 sets `remote: true` explicitly on the `/mcp` request handler's OperationContext literal (closes the HTTP shell-job RCE — without this, `submit_job`'s protected-name guard at `operations.ts:1391` saw a falsy undefined and skipped, letting a `read+write`-scoped OAuth token submit `shell` jobs). F8 wires `summarizeMcpParams` from `src/mcp/dispatch.ts` into both `mcp_request_log` writes and the admin SSE feed by default (raw payloads opt-in via `--log-full-params` with stderr warning). F9 sets cookie `Secure` flag when behind HTTPS or a public-URL proxy. F10 caps the magic-link nonce store with an LRU bound. F12 routes DCR disable through the `GBrainOAuthProvider` constructor's `dcrDisabled` option instead of the prior monkey-patch on the express router. F14 wraps `transport.handleRequest` in try/catch so SDK throws return a JSON-RPC 500 envelope instead of express's default HTML error page. F15 unifies OperationError + unexpected exceptions through `buildError` / `serializeError` so `/mcp` always returns the same envelope shape. **v0.28.1:** `/health` endpoint extracted into pure `probeHealth(engine)` async function with `HEALTH_TIMEOUT_MS = 3000` exported constant — drops the timeout from 5s to 3s so Fly.io's 5s health-check deadline gets 2s of headroom for TCP, response framing, and clock skew. Races `engine.getStats()` against the timeout via `Promise.race`; saturated pool returns 503 with `Health check timed out (database pool may be saturated)` instead of hanging. `clearTimeout` in finally block prevents pending-timer pile-up under high probe rates (race-leak fix from adversarial review). **v0.28.10:** `/health` is now liveness-only via the new `probeLiveness(sql, engineName, version, timeoutMs)` helper that races `sql\`SELECT 1\`` against `HEALTH_TIMEOUT_MS` and returns the same `ProbeHealthResult` tagged-union as `probeHealth` (single timer-cleanup site, single 503 envelope). Body shape: `{status, version, engine}` only — engine stats are no longer spread on the public route. Full stats moved to a new admin endpoint `/admin/api/full-stats` (sibling to `/admin/api/stats` and `/admin/api/health-indicators`) gated by the existing `requireAdmin` middleware; that route calls `probeHealth(engine, ...)` and returns the original spread-stats body. `?full=true` query param removed entirely. Closes the original DoS surface where `getStats()`'s 6× count(*) on 96K-page brains through PgBouncer exceeded `HEALTH_TIMEOUT_MS` and triggered orchestrator restart cascades (Fly.io / k8s seeing 503 → restart loop → advisory-lock pile-up on the migration lock). Outside-voice review (Codex) caught that `/admin/api/health-indicators` is NOT a full-stats endpoint (returns only `{expiring_soon, error_rate}`), and that an alternative loopback-IP gate would have depended on `app.set('trust proxy', 'loopback')` semantics holding under proxy/XFF misconfiguration; the shipped admin-cookie design avoids both. **v0.31.3 (#681):** every OAuth/admin/audit SQL call routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` so `gbrain serve --http` works against PGLite brains. The four `mcp_request_log.params` INSERT sites (success path, auth_failed path, scope_denied path, server-error path) all go through `executeRawJsonb(engine, ...)` so the JSONB column stores real objects, not JSON-encoded strings — closes the bug where `params->>'op'` returned the encoded string `"search"` (with quotes) instead of `search`. Migration v46 normalizes any pre-v0.31.3 string-shaped backlog rows on first start. +- `src/commands/serve-http.ts` (v0.26.0) — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. Started via `gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--log-full-params]`. Supersedes the v0.22.7 `src/mcp/http-transport.ts` simple bearer-auth path. Combines MCP SDK's `mcpAuthRouter` (authorize / token / register / revoke endpoints), a custom `client_credentials` handler (SDK's token endpoint throws `UnsupportedGrantTypeError` for CC; the custom handler runs BEFORE the router and falls through for `auth_code` / `refresh_token`), `requireBearerAuth` middleware for `/mcp` with scope enforcement before op dispatch, `localOnly` rejection, and `express-rate-limit` at 50 req / 15 min on `/token`. Serves the built admin SPA from `admin/dist/` with SPA fallback. `/admin/events` SSE endpoint broadcasts every MCP request to connected admin browsers. `cookie-parser` middleware wired (Express 5 has no built-in). Startup logging prints port, engine, configured issuer URL (honors `--public-url`), registered-client count, DCR status, and admin bootstrap token. **v0.26.9 hardening pass:** F7 sets `remote: true` explicitly on the `/mcp` request handler's OperationContext literal (closes the HTTP shell-job RCE — without this, `submit_job`'s protected-name guard at `operations.ts:1391` saw a falsy undefined and skipped, letting a `read+write`-scoped OAuth token submit `shell` jobs). F8 wires `summarizeMcpParams` from `src/mcp/dispatch.ts` into both `mcp_request_log` writes and the admin SSE feed by default (raw payloads opt-in via `--log-full-params` with stderr warning). F9 sets cookie `Secure` flag when behind HTTPS or a public-URL proxy. F10 caps the magic-link nonce store with an LRU bound. F12 routes DCR disable through the `GBrainOAuthProvider` constructor's `dcrDisabled` option instead of the prior monkey-patch on the express router. F14 wraps `transport.handleRequest` in try/catch so SDK throws return a JSON-RPC 500 envelope instead of express's default HTML error page. F15 unifies OperationError + unexpected exceptions through `buildError` / `serializeError` so `/mcp` always returns the same envelope shape. **v0.28.1:** `/health` endpoint extracted into pure `probeHealth(engine)` async function with `HEALTH_TIMEOUT_MS = 3000` exported constant — drops the timeout from 5s to 3s so Fly.io's 5s health-check deadline gets 2s of headroom for TCP, response framing, and clock skew. Races `engine.getStats()` against the timeout via `Promise.race`; saturated pool returns 503 with `Health check timed out (database pool may be saturated)` instead of hanging. `clearTimeout` in finally block prevents pending-timer pile-up under high probe rates (race-leak fix from adversarial review). **v0.28.10:** `/health` is now liveness-only via the new `probeLiveness(sql, engineName, version, timeoutMs)` helper that races `sql\`SELECT 1\`` against `HEALTH_TIMEOUT_MS` and returns the same `ProbeHealthResult` tagged-union as `probeHealth` (single timer-cleanup site, single 503 envelope). Body shape: `{status, version, engine}` only — engine stats are no longer spread on the public route. Full stats moved to a new admin endpoint `/admin/api/full-stats` (sibling to `/admin/api/stats` and `/admin/api/health-indicators`) gated by the existing `requireAdmin` middleware; that route calls `probeHealth(engine, ...)` and returns the original spread-stats body. `?full=true` query param removed entirely. Closes the original DoS surface where `getStats()`'s 6× count(*) on 96K-page brains through PgBouncer exceeded `HEALTH_TIMEOUT_MS` and triggered orchestrator restart cascades (Fly.io / k8s seeing 503 → restart loop → advisory-lock pile-up on the migration lock). Outside-voice review (Codex) caught that `/admin/api/health-indicators` is NOT a full-stats endpoint (returns only `{expiring_soon, error_rate}`), and that an alternative loopback-IP gate would have depended on `app.set('trust proxy', 'loopback')` semantics holding under proxy/XFF misconfiguration; the shipped admin-cookie design avoids both. **v0.31.3 (#681):** every OAuth/admin/audit SQL call routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` so `gbrain serve --http` works against PGLite brains. The four `mcp_request_log.params` INSERT sites (success path, auth_failed path, scope_denied path, server-error path) all go through `executeRawJsonb(engine, ...)` so the JSONB column stores real objects, not JSON-encoded strings — closes the bug where `params->>'op'` returned the encoded string `"search"` (with quotes) instead of `search`. Migration v46 normalizes any pre-v0.31.3 string-shaped backlog rows on first start. **v0.34.1.0 (#864):** new `--bind HOST` CLI flag with default `127.0.0.1`. Personal-laptop installs no longer publish the brain to the LAN by accident. Self-hosted operators pass `--bind 0.0.0.0` (or a specific interface IP) once to accept remote connections. A stderr WARN fires when `--public-url` is set without `--bind` so the operator sees the binding before the first request (common cause of "ngrok forwards to me but the agent can't reach the upstream" misconfigurations). The startup banner prints a `Bind:` line. **v0.34.1.0 (#861):** drops the `(authInfo as AuthInfo & {sourceId?: string}).sourceId ?? env ?? 'default'` cast chain — `AuthInfo.sourceId` and `AuthInfo.allowedSources` are now the typed source of truth, populated by `oauth-provider.ts:verifyAccessToken` from the `oauth_clients` row. - `src/core/sql-query.ts` (v0.31.3) — Engine-aware tagged-template SQL adapter for OAuth/admin/auth infrastructure. `sqlQueryForEngine(engine)` returns a `SqlQuery` (`(strings, ...values) => Promise`) that walks the template, builds `$N` positional SQL, asserts every value is a `SqlValue` (string | number | bigint | boolean | Date | null), and routes through `engine.executeRaw(sql, params)` so Postgres goes via postgres.js's `unsafe(sql, params)` path and PGLite via its embedded `db.query(sql, params)`. Deliberately narrower than postgres.js's `sql` tag: no nested fragments, no `sql.json()`, no `sql.unsafe()`, no `sql.begin()`, no array binding. The narrow surface is the feature — codex finding #7 from the v0.31 plan review argued the adapter should stay scalar-only or it drifts into a partial postgres.js clone. JSONB writes go through the separate `executeRawJsonb(engine, sql, scalarParams, jsonbParams)` helper that composes positional `$N::jsonb` casts and passes JS objects through; the v0.12.0 double-encode bug class doesn't apply because positional binding through `unsafe()` reaches the wire protocol with the correct type oid (verified by `test/sql-query.test.ts` on PGLite and `test/e2e/auth-permissions.test.ts:67` on Postgres). `scripts/check-jsonb-pattern.sh` doesn't fire because `executeRawJsonb(...)` is a method call, not the banned literal-template-tag interpolation pattern. Consumed by `src/commands/auth.ts`, `src/commands/serve-http.ts`, `src/core/oauth-provider.ts`, `src/commands/files.ts`, and `src/mcp/http-transport.ts` so all five sites work against PGLite and Postgres uniformly. Closes the bug where `gbrain auth` + `gbrain serve --http` were silently Postgres-only because they routed every SQL through the postgres.js singleton (community PR #681). - `src/commands/serve.ts` (v0.31.3) — `gbrain serve` stdio MCP entrypoint with idempotent shutdown across every parent-disconnect signal. Stdio EOF, SIGTERM, SIGINT, SIGHUP, and parent-process death (every reparent case — PID 1, launchd subreaper, systemd, tmux, or a parent shell with `PR_SET_CHILD_SUBREAPER`) all funnel into one `cleanup(reason)` path that releases the engine and the PGLite write-lock dir within 5 seconds. Pre-v0.31.3 the stdio MCP server held the lock indefinitely after Claude Desktop / Cursor / launchd-managed gateways disconnected, forcing a 5-minute stale-lock wait on the next start. Watchdog reparent check is `getParentPid() !== initialParentPid` (capturing the initial ppid once at install time and firing on any change); the previous `=== 1` check missed the subreaper case under launchd / systemd. Bun's `process.ppid` cache is stale across reparenting (see [oven-sh/bun#30305](https://github.com/oven-sh/bun/issues/30305)) so `getParentPid()` runs `spawnSync('ps', ['-o', 'ppid=', '-p', PID])` per tick to read the live kernel PPID. Startup probe verifies `ps` is on PATH; if not (stripped containers, busybox without procps), the watchdog skips installing AND emits a loud `[gbrain serve] watchdog disabled: ps unavailable, parent-death detection unavailable — child will rely on stdin EOF / signals only` stderr line so operators see the degraded mode at boot. Pinned by `test/serve-stdio-lifecycle.test.ts` (22 cases). Closes #413, #446. Credit @Aragorn2046 (origin features in #591) and @seungsu-kr (rebased submitter, Bun ppid workaround). -- `src/core/oauth-provider.ts` (v0.26.0) — `GBrainOAuthProvider` implementing the MCP SDK's `OAuthServerProvider` + `OAuthRegisteredClientsStore` interfaces. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1 spec: `authorize` + `exchangeAuthorizationCode` with PKCE (for ChatGPT), `client_credentials` (for Perplexity / Claude), `refresh_token` with rotation, `revokeToken`, `registerClient` (DCR path validates redirect_uri must be `https://` or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Auth codes single-use with 10-minute TTL via atomic `DELETE...RETURNING` (closes RFC 6749 §10.5 TOCTOU race). Refresh rotation also `DELETE...RETURNING` (closes §10.4 stolen-token detection bypass). `pgArray()` escapes commas/quotes/braces in elements so a comma-bearing redirect_uri can't smuggle a second array element. Legacy `access_tokens` fallback in `verifyAccessToken` grandfathers pre-v0.26 bearer tokens as `read+write+admin`. `sweepExpiredTokens()` runs on startup wrapped in try/catch. **v0.26.9 RFC 6749/7009 hardening pass:** F1+F2 fold `client_id` atomically into the `DELETE WHERE` clauses for both auth-code exchange and refresh rotation — pre-fix the post-hoc client compare burned the row on wrong-client paths so the legitimate client couldn't retry. F3 enforces refresh-scope-subset against the original grant on the row (RFC 6749 §6), not the client's currently-allowed scopes — fixes the case where revoking a scope from a client wouldn't shrink the agent's existing refresh tokens. F4 binds `client_id` on `revokeToken` so a client can only revoke its own tokens (RFC 7009 §2.1). F7c validates the `/token` request's `redirect_uri` against the value stored at `/authorize` (RFC 6749 §4.1.3) — empty-string treated as missing rather than wildcard match (adversarial-review fix). F5 swaps bare `catch {}` blocks in `verifyAccessToken` and `getClient` for `isUndefinedColumnError` from `src/core/utils.ts` — only SQLSTATE 42703 falls through to legacy fallback; lock timeouts and network blips throw and surface. F6 makes `sweepExpiredTokens()` actually return the count via `RETURNING 1` + array length, not a fire-and-forget zero. F12 adds `dcrDisabled` constructor option so `serve-http.ts` can disable the `/register` endpoint without monkey-patching the router. **v0.26.2:** module-private `coerceTimestamp()` boundary helper at the top of the file normalizes postgres-driver-as-string BIGINT columns to JS numbers at every read site (5 call sites: `getClient` L112+L113 for DCR `/register` RFC 7591 §3.2.1 numeric timestamps, `exchangeRefreshToken` L274 + `verifyAccessToken` L296+L303 for the SDK's `typeof === 'number'` bearerAuth check). Throws on non-finite input (NaN/Infinity) so corrupt rows fail loud at the boundary instead of riding through as `expiresAt: NaN`; returns undefined for SQL NULL so callers decide NULL semantics explicitly (refresh + access token paths treat NULL as expired). Helper intentionally NOT promoted to `src/core/utils.ts` — codex review flagged repo-wide BIGINT precision-loss risk for a generic helper. +- `src/core/oauth-provider.ts` (v0.26.0) — `GBrainOAuthProvider` implementing the MCP SDK's `OAuthServerProvider` + `OAuthRegisteredClientsStore` interfaces. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1 spec: `authorize` + `exchangeAuthorizationCode` with PKCE (for ChatGPT), `client_credentials` (for Perplexity / Claude), `refresh_token` with rotation, `revokeToken`, `registerClient` (DCR path validates redirect_uri must be `https://` or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Auth codes single-use with 10-minute TTL via atomic `DELETE...RETURNING` (closes RFC 6749 §10.5 TOCTOU race). Refresh rotation also `DELETE...RETURNING` (closes §10.4 stolen-token detection bypass). `pgArray()` escapes commas/quotes/braces in elements so a comma-bearing redirect_uri can't smuggle a second array element. Legacy `access_tokens` fallback in `verifyAccessToken` grandfathers pre-v0.26 bearer tokens as `read+write+admin`. `sweepExpiredTokens()` runs on startup wrapped in try/catch. **v0.26.9 RFC 6749/7009 hardening pass:** F1+F2 fold `client_id` atomically into the `DELETE WHERE` clauses for both auth-code exchange and refresh rotation — pre-fix the post-hoc client compare burned the row on wrong-client paths so the legitimate client couldn't retry. F3 enforces refresh-scope-subset against the original grant on the row (RFC 6749 §6), not the client's currently-allowed scopes — fixes the case where revoking a scope from a client wouldn't shrink the agent's existing refresh tokens. F4 binds `client_id` on `revokeToken` so a client can only revoke its own tokens (RFC 7009 §2.1). F7c validates the `/token` request's `redirect_uri` against the value stored at `/authorize` (RFC 6749 §4.1.3) — empty-string treated as missing rather than wildcard match (adversarial-review fix). F5 swaps bare `catch {}` blocks in `verifyAccessToken` and `getClient` for `isUndefinedColumnError` from `src/core/utils.ts` — only SQLSTATE 42703 falls through to legacy fallback; lock timeouts and network blips throw and surface. F6 makes `sweepExpiredTokens()` actually return the count via `RETURNING 1` + array length, not a fire-and-forget zero. F12 adds `dcrDisabled` constructor option so `serve-http.ts` can disable the `/register` endpoint without monkey-patching the router. **v0.26.2:** module-private `coerceTimestamp()` boundary helper at the top of the file normalizes postgres-driver-as-string BIGINT columns to JS numbers at every read site (5 call sites: `getClient` L112+L113 for DCR `/register` RFC 7591 §3.2.1 numeric timestamps, `exchangeRefreshToken` L274 + `verifyAccessToken` L296+L303 for the SDK's `typeof === 'number'` bearerAuth check). Throws on non-finite input (NaN/Infinity) so corrupt rows fail loud at the boundary instead of riding through as `expiresAt: NaN`; returns undefined for SQL NULL so callers decide NULL semantics explicitly (refresh + access token paths treat NULL as expired). Helper intentionally NOT promoted to `src/core/utils.ts` — codex review flagged repo-wide BIGINT precision-loss risk for a generic helper. **v0.34.1.0 (#909):** `registerClient` honors `token_endpoint_auth_method: "none"` (RFC 7591 §3.2.1) — public PKCE clients (Claude Code, Cursor, every other PKCE-first MCP client) store `client_secret_hash = NULL` and the response payload omits `client_secret` entirely. Confidential clients (default `client_secret_post` and explicit `client_secret_basic`) keep their one-time-reveal shape. `getClient` correctly normalizes a NULL `client_secret_hash` to JS `undefined` so the SDK's clientAuth path accepts the public client at `/token`. **v0.34.1.0 (#861 + #876):** `verifyAccessToken` JOINs `oauth_clients.source_id` (write scope, scalar) + `oauth_clients.federated_read` (read scope, TEXT[]) and surfaces both on the returned `AuthInfo`. Pre-v60 / pre-v61 brains degrade gracefully via `isUndefinedColumnError` fallback so the upgrade chain is non-blocking on legacy DBs. - `admin/` (v0.26.0) — React 19 + Vite + TypeScript admin SPA embedded in the binary via `admin/dist/` served by `serve-http.ts`. 7 screens: Login (bootstrap token → session cookie), Dashboard (metrics + SSE feed + token health), Agents (sortable table + sparklines + Register button), Register (modal with scope checkboxes + grant type selector), Credentials reveal (full-screen modal with Copy + Download JSON + yellow one-time-only warning), Request Log (filterable paginated), Agent Detail drawer (Details / Activity / Config Export tabs + Revoke). Design tokens: `#0a0a0f` bg, Inter for UI, JetBrains Mono for data, 4-32px spacing scale, rounded pill badges. HTTP-only SameSite=Strict cookie auth. 65KB gzip. Build: `cd admin && bun install && bun run build`; output at `admin/dist/` is committed for self-contained binaries. -- `src/commands/auth.ts` — Token management. `gbrain auth create/list/revoke/test` for legacy bearer tokens (v0.22.7 wired as a first-class CLI subcommand) plus `gbrain auth register-client` (v0.26.0) and `gbrain auth revoke-client ` (v0.26.2) for OAuth 2.1 client lifecycle. `revoke-client` runs an atomic `DELETE...RETURNING` on `oauth_clients`; FK `ON DELETE CASCADE` on `oauth_tokens.client_id` and `oauth_codes.client_id` purges every active token + authorization code in a single transaction. `process.exit(1)` on no-such-client (idempotent — re-running on the same id produces the same exit-1 message). Legacy tokens stored as SHA-256 hashes in `access_tokens`; OAuth clients in `oauth_clients`. As of v0.26.0, legacy tokens grandfather to `read+write+admin` scopes on the OAuth HTTP server, so pre-v0.26 deployments keep working with no migration. **v0.31.3 (#681):** every SQL site routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` (and `executeRawJsonb` for the takes-holders `permissions` JSONB column) so `gbrain auth` works against PGLite brains. Pre-fix, every call hit the postgres.js singleton via `getConn()` and silently failed (or wrote to the wrong DB) when the active engine was PGLite. The takes-holders write goes through `executeRawJsonb(engine, sql, [name, hash], [{takes_holders:[...]}])` which round-trips with `jsonb_typeof = 'object'` instead of the pre-v0.31.3 quoted-string shape. +- `src/commands/auth.ts` — Token management. `gbrain auth create/list/revoke/test` for legacy bearer tokens (v0.22.7 wired as a first-class CLI subcommand) plus `gbrain auth register-client` (v0.26.0) and `gbrain auth revoke-client ` (v0.26.2) for OAuth 2.1 client lifecycle. `revoke-client` runs an atomic `DELETE...RETURNING` on `oauth_clients`; FK `ON DELETE CASCADE` on `oauth_tokens.client_id` and `oauth_codes.client_id` purges every active token + authorization code in a single transaction. `process.exit(1)` on no-such-client (idempotent — re-running on the same id produces the same exit-1 message). Legacy tokens stored as SHA-256 hashes in `access_tokens`; OAuth clients in `oauth_clients`. As of v0.26.0, legacy tokens grandfather to `read+write+admin` scopes on the OAuth HTTP server, so pre-v0.26 deployments keep working with no migration. **v0.31.3 (#681):** every SQL site routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` (and `executeRawJsonb` for the takes-holders `permissions` JSONB column) so `gbrain auth` works against PGLite brains. Pre-fix, every call hit the postgres.js singleton via `getConn()` and silently failed (or wrote to the wrong DB) when the active engine was PGLite. The takes-holders write goes through `executeRawJsonb(engine, sql, [name, hash], [{takes_holders:[...]}])` which round-trips with `jsonb_typeof = 'object'` instead of the pre-v0.31.3 quoted-string shape. **v0.34.1.0 (#876):** `register-client` accepts `--source ` (write authority, scalar) and `--federated-read ` (read scope, array). The output prints the resolved `Write source` and `Federated reads` for the registered client. Pre-v0.34 clients backfill to `source_id='default'` via migration v60 so existing deployments keep their v0.33 effective behavior verbatim. - `src/commands/upgrade.ts` — Self-update CLI. `runPostUpgrade()` enumerates migrations from the TS registry (src/commands/migrations/index.ts) and tail-calls `runApplyMigrations(['--yes', '--non-interactive'])` so the mechanical side of every outstanding migration runs unconditionally. - `src/commands/migrations/` — TS migration registry (compiled into the binary; no filesystem walk of `skills/migrations/*.md` needed at runtime). `index.ts` lists migrations in semver order. `v0_11_0.ts` = Minions adoption orchestrator (8 phases). `v0_12_0.ts` = Knowledge Graph auto-wire orchestrator (5 phases: schema → config check → backfill links → backfill timeline → verify). `phaseASchema` has a 600s timeout (bumped from 60s in v0.12.1 for duplicate-heavy brains). `v0_12_2.ts` = JSONB double-encode repair orchestrator (4 phases: schema → repair-jsonb → verify → record). `v0_14_0.ts` = shell-jobs + autopilot cooperative (2 phases: schema ALTER minion_jobs.max_stalled SET DEFAULT 3 — superseded by v0.14.3's schema-level DEFAULT 5 + UPDATE backfill; pending-host-work ping for skills/migrations/v0.14.0.md). All orchestrators are idempotent and resumable from `partial` status. As of v0.14.2 (Bug 3), the RUNNER owns all ledger writes — orchestrators return `OrchestratorResult` and `apply-migrations.ts` persists a canonical `{version, status, phases}` shape after return. Orchestrators no longer call `appendCompletedMigration` directly. `statusForVersion` prefers `complete` over `partial` (never regresses). 3 consecutive partials → wedged → `--force-retry ` writes a `'retry'` reset marker. v0.14.3 (fix wave) ships schema-only migrations v14 (`pages_updated_at_index`) + v15 (`minion_jobs_max_stalled_default_5` with UPDATE backfill) via the `MIGRATIONS` array in `src/core/migrate.ts` — no orchestrator phases needed. - `src/commands/repair-jsonb.ts` — `gbrain repair-jsonb [--dry-run] [--json]`: rewrites `jsonb_typeof='string'` rows in place across 5 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter). Fixes v0.12.0 double-encode bug on Postgres; PGLite no-ops. Idempotent. @@ -282,7 +282,7 @@ strict behavior when unset. - `src/commands/transcripts.ts` (v0.29) — `gbrain transcripts recent [--days N] [--full] [--json]`: recent raw `.txt` transcripts from the dream-cycle corpus dirs. Imports `listRecentTranscripts` from `src/core/transcripts.ts` (the same library the gated `get_recent_transcripts` MCP op uses). Local-only by construction — the CLI always runs with `ctx.remote=false`. - `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s). Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable. **v0.32.8 (PR #860):** batch projection switched from `SELECT DISTINCT ON (slug)` to `SELECT ... ORDER BY source_id, slug` so multi-source brains scan each `(source, slug)` row independently (pre-fix the DISTINCT collapsed same-slug-different-source pages into one scan, the same bug class this PR fixes). Sequential and auto-repair loops use `listAllPageRefs()` to enumerate `(slug, source_id)` pairs and thread `sourceId` to `getPage`. Batch + sequential paths now report the same page count on multi-source brains. - `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count. v0.26.7 (#612): `rls_event_trigger` check (post-install drift detector for migration v35's auto-RLS event trigger). Lives outside the `// 5. RLS` slice that the structural doctor.test.ts guards anchor on, so the existing test guards stay intact. Healthy `evtenabled` set is `('O','A')` only — `R` is replica-only and would not fire in normal sessions; `D` is disabled. Fix hint is `gbrain apply-migrations --force-retry 35`. **v0.30.2:** `queue_health` gains a fourth subcheck — surfaces dead-lettered subagent jobs with `last_error` matching the `prompt_too_long` classifier within the last 24h. Fix hint points at `gbrain dream --phase synthesize --dry-run --json` to identify the offending transcript and `gbrain jobs prune --status dead --queue default` to clean up. Postgres-only. **v0.31.7:** `runDoctor` switches to `autoDetectSkillsDirReadOnly` (from `src/core/repo-root.ts`) so `bun install -g github:garrytan/gbrain && cd ~ && gbrain doctor` finds the bundled `skills/` via the install-path fallback instead of warning "Could not find skills directory" + docking the health score. `--fix` carries a D6 safety gate: when `detected.source === 'install_path'`, the command refuses auto-repair with a stderr message pointing at `$GBRAIN_SKILLS_DIR` / `$OPENCLAW_WORKSPACE` / `--skills-dir`, because `autoFixDryViolations` writes to SKILL.md files and would otherwise silently rewrite the install tree. The `graph_coverage` check now short-circuits to `ok: 'No entity pages — graph_coverage not applicable (markdown-only brain)'` when `SELECT COUNT(*) FROM pages WHERE type IN ('entity','person','company','organization')` returns 0 (closes #530); the entity count is woven into the warn message and the WARN hint switches from the long-deprecated `gbrain link-extract && gbrain timeline-extract` (gone since v0.16) to the canonical `gbrain extract all`. Pinned by an IRON-RULE regression assertion in `test/doctor.test.ts` that bans the stale verb names from the source string. **v0.32.4:** new `sync_freshness` check (exported `checkSyncFreshness` at the same file) added to both `runDoctor` (local) and `doctorReportRemote` (thin-client). Pure staleness probe — queries `sources.last_sync_at` only, no filesystem access. Warns at 24h, fails at 72h (or never-synced). Future-`last_sync_at` warns ("clock skew or corrupted timestamp") instead of silently falling through as ok — codex outside-voice caught the negative-ageMs bug pre-merge. Env-var overrides `GBRAIN_SYNC_FRESHNESS_WARN_HOURS` / `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS`; invalid values fall back to defaults with a once-per-process stderr warn (`_resolveSyncFreshnessHours`). Failure messages embed `source.id` (not `source.name`) so the printed fix command `gbrain sync --source ` matches what the user copy-pastes. Filesystem-vs-DB page drift detection was deliberately stripped from the v0.32.4 scope — `doctorReportRemote` runs in the HTTP MCP server (`src/commands/serve-http.ts`), and walking DB-supplied `local_path` from a remote-callable endpoint crosses a trust boundary (OAuth write scope could mutate `sources.local_path`). Drift detection will resurface in a separate PR routed through `multi_source_drift`'s existing guard infrastructure (`GBRAIN_DRIFT_LIMIT` / `GBRAIN_DRIFT_TIMEOUT_MS`) with slug normalization tests and a meta-file allow-list. Pinned by 12 cases in `test/doctor.test.ts` ("v0.32.4 — sync_freshness check" describe block): empty sources, never-synced fail, >72h fail, exact 72h boundary, 24h-72h warn, exact 24h boundary, <24h ok, future-timestamp warn, mixed sources (highest severity wins), `executeRaw` throws → outer-catch warn, `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS=6` override fires at 7h, source.id-in-message regression. -- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). **v40 (v0.29):** `pages_emotional_weight` adds `pages.emotional_weight REAL NOT NULL DEFAULT 0.0`. Column-only (no index). On Postgres 11+ and PGLite, `ADD COLUMN` with a constant DEFAULT is metadata-only — instant on tables of any size. v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs. **v35 (v0.26.7):** auto-RLS event trigger + one-time backfill. `auto_rls_on_create_table` fires on `ddl_command_end` for `WHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO')` and runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY` on every new `public.*` table — no FORCE (matches v24/v29/schema.sql posture so non-BYPASSRLS apps can still read their own tables). The same migration backfills RLS on every existing `public.*` base table whose comment doesn't match the doctor regex (`^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,}`). Per-table failure aborts the offending CREATE TABLE (event triggers fire inside the DDL transaction); no EXCEPTION wrap — that would convert loud rollback into silent permissive default. PGLite no-op via `sqlFor.pglite: ''`. Breaking change: operators with intentionally-RLS-off public tables must add the GBRAIN:RLS_EXEMPT comment BEFORE upgrade or the backfill will flip them on. **v46 (v0.31.3):** `mcp_request_log_params_jsonb_normalize` rewrites pre-v0.31.3 rows where `mcp_request_log.params` was stored as a JSON-encoded string (`jsonb_typeof = 'string'`) up to a real JSONB object via `UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string'`. Single statement, idempotent — second-run finds no string-shaped rows and is a no-op. Closes the bug where `/admin/api/requests` returned a quoted string instead of the parsed object. +- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). **v40 (v0.29):** `pages_emotional_weight` adds `pages.emotional_weight REAL NOT NULL DEFAULT 0.0`. Column-only (no index). On Postgres 11+ and PGLite, `ADD COLUMN` with a constant DEFAULT is metadata-only — instant on tables of any size. v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs. **v35 (v0.26.7):** auto-RLS event trigger + one-time backfill. `auto_rls_on_create_table` fires on `ddl_command_end` for `WHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO')` and runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY` on every new `public.*` table — no FORCE (matches v24/v29/schema.sql posture so non-BYPASSRLS apps can still read their own tables). The same migration backfills RLS on every existing `public.*` base table whose comment doesn't match the doctor regex (`^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,}`). Per-table failure aborts the offending CREATE TABLE (event triggers fire inside the DDL transaction); no EXCEPTION wrap — that would convert loud rollback into silent permissive default. PGLite no-op via `sqlFor.pglite: ''`. Breaking change: operators with intentionally-RLS-off public tables must add the GBRAIN:RLS_EXEMPT comment BEFORE upgrade or the backfill will flip them on. **v46 (v0.31.3):** `mcp_request_log_params_jsonb_normalize` rewrites pre-v0.31.3 rows where `mcp_request_log.params` was stored as a JSON-encoded string (`jsonb_typeof = 'string'`) up to a real JSONB object via `UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string'`. Single statement, idempotent — second-run finds no string-shaped rows and is a no-op. Closes the bug where `/admin/api/requests` returned a quoted string instead of the parsed object. **v0.34.1.0 (#861 + #876, v60-v65):** six-migration chain wires source-scoping into the OAuth client table. v60 (`oauth_clients_source_id_fk`) adds `oauth_clients.source_id TEXT` with NULL→`'default'` backfill and an FK to `sources(id) ON DELETE SET NULL`. v61 (`oauth_clients_federated_read_column`) adds `federated_read TEXT[] NOT NULL DEFAULT '{}'`. v62 (`oauth_clients_federated_read_backfill`) explicit-CASE backfills so `source_id IS NULL` produces `'{}'` not an array-containing-NULL. v63 (`oauth_clients_federated_read_validate`) is the fail-loud check that every row's source_id is in its federated_read array post-backfill. v64 (`oauth_clients_source_id_fk_restrict`) flips the FK to `ON DELETE RESTRICT` now that federated_read provides the alternative scope-loss path — source delete is refused if any client references it. v65 (`oauth_clients_federated_read_gin_index`) is the GIN index for the array-containment queries the read paths run. PGLite parity via `sqlFor.pglite` where needed. - `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2. - `src/core/cli-options.ts` — Global CLI flag parser. `parseGlobalFlags(argv)` returns `{cliOpts, rest}` with `--quiet` / `--progress-json` / `--progress-interval=` stripped. `getCliOptions()` / `setCliOptions()` expose a module-level singleton so commands reach the resolved flags without parameter threading. `cliOptsToProgressOptions()` maps to reporter options. `childGlobalFlags()` returns the flag suffix to append to `execSync('gbrain ...')` calls in migration orchestrators. `OperationContext.cliOpts` extends shared-op dispatch for MCP callers. - `src/core/db-lock.ts` (v0.22.13) — generic `tryAcquireDbLock(engine, lockId, ttlMinutes)` over the existing `gbrain_cycle_locks` table. Parameterized lock id so different scopes can nest cleanly: `gbrain-cycle` for the broad cycle (held by `cycle.ts`) and `gbrain-sync` (`SYNC_LOCK_ID` constant) for `performSync`'s narrower writer window. Same UPSERT-with-TTL semantics as the prior cycle-only helper, just generalized. Survives PgBouncer transaction pooling (unlike session-scoped `pg_try_advisory_lock`); crashed holders auto-release once their TTL expires. @@ -757,6 +757,10 @@ E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_U - `test/e2e/serve-http-oauth.test.ts` (v0.26.0, expanded v0.26.2, expanded v0.26.9) — real-Postgres E2E against `gbrain serve --http` with full OAuth 2.1. Spawns a subprocess server, registers a client via the CLI, mints `client_credentials` tokens, exercises the `/mcp` JSON-RPC pipeline. **v0.26.2 adds:** real DCR `/register` HTTP-level response-shape test (asserts `typeof body.client_id_issued_at === 'number'` over the wire — RFC 7591 §3.2.1 spec compliance, not just internal-store shape); real CLI subprocess test for `revoke-client` (registers → mints token → revokes via `execSync` → asserts token rejected at `/mcp` → asserts re-run exits 1); server fixture flips on `--enable-dcr` so `/register` is reachable. **bun execSync env-inheritance fix:** bun's `execSync` does NOT inherit env mutations done via `process.env.X = ...`, only OS-level env from before bun started. helpers.ts loads `.env.testing` and sets `DATABASE_URL` via `process.env` mutation, which is invisible to subprocesses unless `env: { ...process.env }` is passed explicitly — every subprocess call in this file passes `env: { ...process.env }` for that reason. Reference fix for the next maintainer hitting the same failure mode in sibling sync/cycle/dream/claw-test E2Es. `afterAll` cleanup is guarded on `clientId` (won't throw if `beforeAll` failed before registration); cleanup errors surface to stderr without throwing so real test failures aren't masked. Tracks DCR-registered clients alongside the manual one. **v0.26.9** adds 2 regressions for the F7 trust-boundary fix: an HTTP MCP `submit_job` for `name: "shell"` MUST reject with a permission error (proving the request handler now sets `remote: true` and `submit_job`'s protected-name guard fires), and the same guard rejects subagent submission. Closes the OAuth-token-to-RCE escalation path. Skips gracefully when `DATABASE_URL` is unset. - `test/e2e/sync-parallel.test.ts` (v0.22.13 PR #490) — DATABASE_URL-gated. T2: 60-file Postgres sync at concurrency=4 imports all + no connection leak (probes `pg_stat_activity` before/after to confirm worker engines disconnected). P4: 120-file serial-vs-parallel benchmark prints `SYNC_PARALLEL_BENCH N files | serial=Xms | parallel(4)=Yms | speedup=Zx` for CHANGELOG quoting. Asserts parallel ≤ serial × 1.5 (CI-noise tolerant; not a strict speedup gate). - `test/e2e/multi-source-bug-class.test.ts` (v0.32.8, PR #860) — 7-case PGLite in-memory regression suite pinning every bug site fixed in this PR: `listAllPageRefs` ordering by `(source_id, slug)` (F11), `getPage` with sourceId picks the right `(source, slug)` row (F2), `extract-takes` processes both overlapping `people/alice` rows independently, `listPages` filters correctly with `PageFilters.sourceId`, `addLinksBatch` with `from/to_source_id` targets the right rows (F4), `validateSourceId` rejects path traversal (F6), reverse-write disk layout uses `brainDir/.sources//.md` for non-default sources (F6). No DATABASE_URL needed. Wired into `scripts/e2e-test-map.ts` so changes to extract-takes / patterns / synthesize / embed / extract / migrate-engine auto-trigger this test. Companion: `test/e2e/integrity-batch.test.ts`'s "multi-source duplicate slugs scan once" case was pinning the pre-fix bug — assertion flipped in v0.32.8 to expect both batch + sequential paths report 2. +- `test/e2e/source-isolation-pglite.test.ts` (v0.34.1.0, #861) — 14-case PGLite in-memory regression suite pinning the source-isolation P0 seal at two layers. Engine layer: `searchKeyword` / `searchVector` / `searchKeywordChunks` / `listPages` / `getPage` / `traverseGraph` / `traversePaths` apply `sourceId` (scalar fast path) and `sourceIds` (array path) correctly across both engines. Op-handler layer: routes through `sourceScopeOpts(ctx)` so a `read+write`-scoped OAuth client bound to `--source dept-x` cannot see rows from neighboring sources via `search`, `query`, `list_pages`, `get_page`, or `find_experts`. Covers both `ctx.sourceId` (single-source clients) and `ctx.auth.allowedSources` (federated_read clients) precedence; federated array wins over scalar wins over nothing. No DATABASE_URL needed. +- `test/openai-compat-multimodal.test.ts` (v0.34.1.0, #875) — 11-case unit suite for the gateway's openai-compatible multimodal path: happy-path single + multi-input embedding, unauthenticated proxy mode, dimension-mismatch guard (D12; throws `AIConfigError` with model id + observed + expected pre-storage), default-dim fallback when recipe declares `default_dims`, HTTP 401 / 400 / malformed-JSON / non-array error paths, plus a regression test that the existing Voyage `/multimodalembeddings` recipe still routes through its dedicated path (not the openai-compatible one). Hermetic via the `__setEmbedTransportForTests` seam. +- `test/serve-stdio-lifecycle.test.ts` (extended v0.34.1.0, #870) — adds 3 new cases for the `MCP_STDIO=1` env guard: stdin EOF does NOT trigger shutdown when the env is set, SIGTERM still does (guard scope is correct), unset env preserves the pre-v0.34 CLI lifecycle. Exercises the `ServeOptions.mcpStdio?: boolean` test seam directly so tests don't mutate `process.env`. +- `test/oauth.test.ts` (extended v0.34.1.0, #909) — 5 new cases for the PKCE DCR public-client gate: `registerClient` with `token_endpoint_auth_method: "none"` returns no `client_secret` field on the public client, default `client_secret_post` clients still get the one-time-reveal secret, `getClient` NULL→undefined normalization so the SDK's clientAuth path accepts public clients, full PKCE `/authorize` → `/token` round-trip against a public client (no client_secret presented), and a regression test that the public-vs-confidential branch doesn't break confidential client `client_secret_post` exchange. - Tier 2 (`skills.test.ts`) requires OpenClaw + API keys, runs nightly in CI - If `.env.testing` doesn't exist in this directory, check sibling worktrees for one: `find ../ -maxdepth 2 -name .env.testing -print -quit` and copy it here if found. @@ -2210,7 +2214,9 @@ pick scopes, save the credentials shown once in the reveal modal. Programmatic registration via `oauthProvider.registerClientManual(...)` and the `gbrain auth register-client` CLI are also available. -- **OAuth 2.1 via the MCP SDK** — client credentials (machine-to-machine: Perplexity, Claude), authorization code + PKCE (browser-based: ChatGPT), refresh token rotation, revocation, protected resource metadata. Optional Dynamic Client Registration behind `--enable-dcr` (DCR redirect_uris must be `https://` or loopback per RFC 6749 §3.1.2.1). +- **OAuth 2.1 via the MCP SDK** — client credentials (machine-to-machine: Perplexity, Claude), authorization code + PKCE (browser-based: ChatGPT), refresh token rotation, revocation, protected resource metadata. PKCE-only public clients (`token_endpoint_auth_method: "none"`) register without a secret per RFC 7591 §3.2.1 (v0.34). Optional Dynamic Client Registration behind `--enable-dcr` (DCR redirect_uris must be `https://` or loopback per RFC 6749 §3.1.2.1). +- **Source-scoped OAuth clients (v0.34)** — `gbrain auth register-client my-agent --source dept-x` ties the client's write authority to one source; read paths only return rows matching that source. `--federated-read S1,S2,S3` adds an orthogonal read-scope axis for shared brains (departments writing to one canon while reading the union). Pre-v0.34 clients are backfilled to `source_id='default'` on upgrade. +- **Loopback default for `serve --http` (v0.34)** — listens on `127.0.0.1` unless `--bind 0.0.0.0` (or a specific interface IP). Personal-laptop installs no longer publish the brain to the LAN by accident. A stderr WARN fires when `--public-url` is set without `--bind` so the operator sees the binding before the first request. - **Scoped operations** — 30 operations tagged `read | write | admin`. `sync_brain` and `file_upload` are `localOnly`, rejected over HTTP. - **React admin dashboard** — 7 screens baked into the binary (~65KB gzip). Live SSE activity feed, agents table, credential reveal, filterable request log, per-client config export. - **Legacy bearer tokens still work** — pre-v0.26 `gbrain auth create` tokens continue to authenticate as `read+write+admin`. v0.22.7's simpler `src/mcp/http-transport.ts` path stays compiled in for backward compat callers; v0.26+ deployments use the OAuth-aware `serve-http.ts`. @@ -2859,12 +2865,16 @@ ADMIN [--skip=] [--json] gbrain serve MCP server (stdio) gbrain serve --http [--port 3131] HTTP MCP server with OAuth 2.1 + admin dashboard + [--bind HOST] (v0.34: default 127.0.0.1; pass + --bind 0.0.0.0 for LAN/remote access) [--token-ttl 3600] [--enable-dcr] [--public-url URL] [--log-full-params] gbrain auth create|list|revoke|test Legacy bearer token management gbrain auth register-client Register an OAuth 2.1 client --grant-types client_credentials,authorization_code --scopes "read write admin" + --source v0.34: write authority for source-scoped clients + --federated-read v0.34: read scope across multiple sources gbrain auth revoke-client Revoke an OAuth 2.1 client (cascade purges active tokens + auth codes via FK CASCADE) # OAuth 2.1 clients can also be registered from the /admin dashboard or @@ -5192,6 +5202,24 @@ gbrain auth register-client perplexity \ --scopes "read write" ``` +**v0.34 — source-scoped clients.** Multi-source brains can scope a client's +write authority to one source and its read scope to a curated set with the +new `--source` and `--federated-read` flags: + +```bash +gbrain auth register-client dept-x-agent \ + --grant-types client_credentials \ + --scopes "read write" \ + --source dept-x \ + --federated-read dept-x,shared,parent-canon +``` + +`--source` controls the write authority — `put_page` / `add_link` / etc only +land in `dept-x`. `--federated-read` controls the read axis independently; +queries return rows from any of the listed sources. Omit both flags for the +v0.33-compatible super-client shape. Pre-v0.34 clients are backfilled to +`source_id='default'` on `gbrain upgrade`. + Host-repo wrappers can register programmatically: ```ts @@ -5208,6 +5236,18 @@ start the server with `--enable-dcr`. DCR is off by default. ### 3. Expose the server +**v0.34 — bind explicitly.** `gbrain serve --http` defaults to `127.0.0.1`. +To accept connections from the ngrok tunnel (or any non-loopback source), +restart with `--bind`: + +```bash +gbrain serve --http --port 3131 --bind 0.0.0.0 --public-url https://your-brain.ngrok.app +``` + +When `--public-url` is set without `--bind`, a stderr WARN fires at +startup so the misconfiguration ("the tunnel is up but my agent gets +ECONNREFUSED") is loud. + ```bash brew install ngrok ngrok config add-authtoken YOUR_TOKEN diff --git a/package.json b/package.json index 5e7a7f5..5491452 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gbrain", - "version": "0.34.0.0", + "version": "0.34.1.0", "description": "Postgres-native personal knowledge brain with hybrid RAG search", "type": "module", "main": "src/core/index.ts", diff --git a/src/commands/auth.ts b/src/commands/auth.ts index 4aaca2c..f9ae0cb 100644 --- a/src/commands/auth.ts +++ b/src/commands/auth.ts @@ -329,26 +329,44 @@ async function revokeClient(clientId: string) { } async function registerClient(name: string, args: string[]) { - if (!name) { console.error('Usage: auth register-client [--grant-types G] [--scopes S]'); process.exit(1); } + if (!name) { + console.error('Usage: auth register-client [--grant-types G] [--scopes S] [--source SOURCE] [--federated-read SRC1,SRC2,...]'); + process.exit(1); + } const grantsIdx = args.indexOf('--grant-types'); const scopesIdx = args.indexOf('--scopes'); + const sourceIdx = args.indexOf('--source'); + const federatedIdx = args.indexOf('--federated-read'); const grantTypes = grantsIdx >= 0 && args[grantsIdx + 1] ? args[grantsIdx + 1].split(',').map(s => s.trim()).filter(Boolean) : ['client_credentials']; const scopes = scopesIdx >= 0 && args[scopesIdx + 1] ? args[scopesIdx + 1] : 'read'; + // v0.34.1 (#861, D2): --source flag scopes the OAuth client to a single + // source. Defaults to 'default' to match migration v60's backfill so + // operators upgrading without changing flags see no behavior change. + const sourceId = sourceIdx >= 0 && args[sourceIdx + 1] ? args[sourceIdx + 1] : 'default'; + // v0.34.1 (#876): --federated-read accepts a comma-separated source list + // for federated read scope. When omitted, federated_read defaults to + // [sourceId] (read scope == write scope, the v0.33 default). + const federatedRead = federatedIdx >= 0 && args[federatedIdx + 1] + ? args[federatedIdx + 1].split(',').map(s => s.trim()).filter(Boolean) + : undefined; try { await withConfiguredSql(async (sql) => { const { GBrainOAuthProvider } = await import('../core/oauth-provider.ts'); const provider = new GBrainOAuthProvider({ sql }); const { clientId, clientSecret } = await provider.registerClientManual( - name, grantTypes, scopes, [], + name, grantTypes, scopes, [], sourceId, federatedRead, ); + const effectiveFederated = federatedRead && federatedRead.length > 0 ? federatedRead : [sourceId]; console.log(`OAuth client registered: "${name}"\n`); - console.log(` Client ID: ${clientId}`); - console.log(` Client Secret: ${clientSecret}\n`); - console.log(` Grant types: ${grantTypes.join(', ')}`); - console.log(` Scopes: ${scopes}\n`); + console.log(` Client ID: ${clientId}`); + console.log(` Client Secret: ${clientSecret}\n`); + console.log(` Grant types: ${grantTypes.join(', ')}`); + console.log(` Scopes: ${scopes}`); + console.log(` Write source: ${sourceId}`); + console.log(` Federated reads: ${effectiveFederated.join(', ')}\n`); console.log('Save the client secret — it will not be shown again.'); console.log(`Revoke with: gbrain auth revoke-client "${clientId}"`); }); diff --git a/src/commands/serve-http.ts b/src/commands/serve-http.ts index b8da946..752324b 100644 --- a/src/commands/serve-http.ts +++ b/src/commands/serve-http.ts @@ -162,10 +162,31 @@ interface ServeHttpOptions { * at startup so the privacy posture change is visible. */ logFullParams?: boolean; + /** + * Network interface(s) to bind. Defaults to `127.0.0.1` (loopback only) in + * v0.34.1+ — gbrain's primary use case is a personal-knowledge brain on a + * laptop, and the pre-v0.34 default of `0.0.0.0` made it one accidental + * `--http` invocation away from publishing the brain to a LAN. + * + * Server operators who DO want to accept remote connections pass + * `--bind 0.0.0.0` (or a specific interface IP). When `--public-url` is + * set but `--bind` is unset, a stderr WARN fires at startup recommending + * the explicit flag — defaulting to loopback while declaring a public URL + * is almost always a misconfiguration. + */ + bind?: string; } export async function runServeHttp(engine: BrainEngine, options: ServeHttpOptions) { const { port, tokenTtl, enableDcr, publicUrl, logFullParams } = options; + // v0.34.1 (#864, D11): default bind flipped from 0.0.0.0 to 127.0.0.1. + // gbrain's primary use case is a personal-knowledge brain on a laptop; + // the pre-v0.34 default exposed brains on every interface. Server + // operators who need remote access pass `--bind 0.0.0.0` (or a specific + // interface). Declaring `--public-url` without `--bind` is almost always + // a misconfiguration; we WARN to stderr at startup in that case rather + // than silently binding loopback only. + const bind = options.bind ?? '127.0.0.1'; const config = loadConfig() || { engine: 'pglite' as const }; if (logFullParams) { @@ -174,6 +195,12 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption ); } + if (publicUrl && options.bind === undefined) { + console.error( + '[serve-http] WARNING: --public-url is set but --bind is not. Default bind changed to 127.0.0.1 in v0.34.1; remote clients reaching the public URL will be refused. Pass --bind 0.0.0.0 to accept all interfaces.', + ); + } + // Engine-aware SQL adapter. Routes through engine.executeRaw on both // Postgres and PGLite — the OAuth/admin/auth surface no longer requires // a postgres.js singleton, so `gbrain serve --http` works against PGLite @@ -926,9 +953,14 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption // ToolResult and we read isError + _meta to pick the right branch. const tokenAllowList = (authInfo as AuthInfo & { takesHoldersAllowList?: string[] }).takesHoldersAllowList ?? ['world']; - const tokenSourceId = (authInfo as AuthInfo & { sourceId?: string }).sourceId - ?? process.env.GBRAIN_SOURCE - ?? 'default'; + // v0.34.1 (#861, D13): AuthInfo.sourceId is now a real typed field + // populated from oauth_clients.source_id (migration v60 backfilled + // NULL → 'default'). Pre-fix this site cast through AuthInfo and + // fell back to GBRAIN_SOURCE env / 'default' — the silent-fallback + // path codex flagged in plan review. Post-v60, every OAuth client + // has source_id set; legacy bearer tokens default to 'default' in + // verifyAccessToken. The env-fallback is gone. + const tokenSourceId = authInfo.sourceId ?? 'default'; let toolResult: Awaited>; try { @@ -1058,12 +1090,13 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption // --------------------------------------------------------------------------- const clientCount = await sql`SELECT count(*)::int as count FROM oauth_clients`; - app.listen(port, () => { + app.listen(port, bind, () => { console.error(` ╔══════════════════════════════════════════════════════╗ ║ GBrain MCP Server v${VERSION.padEnd(37)}║ ╠══════════════════════════════════════════════════════╣ ║ Port: ${String(port).padEnd(40)}║ +║ Bind: ${bind.padEnd(40)}║ ║ Engine: ${(config.engine || 'pglite').padEnd(40)}║ ║ Issuer: ${issuerUrl.origin.padEnd(40)}║ ║ Clients: ${String((clientCount[0] as any).count).padEnd(40)}║ diff --git a/src/commands/serve.ts b/src/commands/serve.ts index 1b1651a..3fa1362 100644 --- a/src/commands/serve.ts +++ b/src/commands/serve.ts @@ -56,6 +56,13 @@ export interface ServeOptions { // tick fell through to the cached `process.ppid` and the watchdog // never fired, while still claiming to be installed. probeWatchdog?: () => boolean; + // v0.34.1 (#870): test seam for the MCP_STDIO=1 piped-stdin guard. + // When true, runServe skips the stdin 'end'/'close' shutdown hooks + // because the wrapping gateway (OpenClaw bundle-mcp, others) pipes the + // JSON-RPC handshake and closes stdin immediately. Signal handlers and + // transport.onclose still cover legitimate shutdown. + // Defaults to `process.env.MCP_STDIO === '1'` when omitted. + mcpStdio?: boolean; } export async function runServe( @@ -90,8 +97,18 @@ export async function runServe( // when set so the posture change is visible in stderr. const logFullParams = args.includes('--log-full-params'); + // v0.34.1 (#864, D11): `--bind HOST` lets operators choose the network + // interface to listen on. When unset, runServeHttp defaults to 127.0.0.1 + // (loopback) — server operators who need remote access pass + // `--bind 0.0.0.0` (or a specific interface IP). `bind` is intentionally + // left undefined here when the flag is absent so the WARN-on-public-url + // path in serve-http can distinguish "operator chose loopback explicitly" + // from "operator didn't set the flag at all." + const bindIdx = args.indexOf('--bind'); + const bind = bindIdx >= 0 ? args[bindIdx + 1] : undefined; + const { runServeHttp } = await import('./serve-http.ts'); - await runServeHttp(engine, { port, tokenTtl, enableDcr, publicUrl, logFullParams }); + await runServeHttp(engine, { port, tokenTtl, enableDcr, publicUrl, logFullParams, bind }); return; } @@ -201,7 +218,15 @@ function installStdioLifecycle( // Skip when stdin is a TTY: interactive `gbrain serve` use shouldn't // terminate just because the user hasn't typed anything. Signal / // watchdog paths still cover that case if needed. - if (!deps.stdin.isTTY) { + // v0.34.1 (#870): when MCP_STDIO=1, the wrapping gateway pipes the + // JSON-RPC handshake then closes its stdin half. Treating that as a + // permanent disconnect kills the server before the first tool call. + // Signal handlers (SIGTERM/SIGINT/SIGHUP), transport.onclose, and the + // parent-process watchdog below still cover legitimate shutdown paths. + // `mcpStdio` is the injectable form; default reads the env once at + // install time so tests stay isolated (no process.env mutation). + const mcpStdioMode = opts.mcpStdio ?? (process.env.MCP_STDIO === '1'); + if (!deps.stdin.isTTY && !mcpStdioMode) { deps.stdin.once('end', () => beginShutdown('stdin-end')); deps.stdin.once('close', () => beginShutdown('stdin-close')); } diff --git a/src/commands/whoknows.ts b/src/commands/whoknows.ts index a422568..8460057 100644 --- a/src/commands/whoknows.ts +++ b/src/commands/whoknows.ts @@ -52,6 +52,21 @@ export interface WhoknowsOpts { * ranking function without redefining the type filter. */ types?: PageType[]; + /** + * v0.34.1 (#861, D3 — P0 leak seal): scope expert candidates to a + * single source. The op-handler at operations.ts:find_experts threads + * `ctx.sourceId` here so an authenticated MCP client scoped to src-A + * cannot surface people pages from src-B in the rankings. Pre-fix, the + * whoknows op was authored against v0.33 after PR #861 was drafted and + * the source-scope thread was missing entirely. + */ + sourceId?: string; + /** + * v0.34.1 (#876, D9): federated read — scope candidates to ANY of these + * source ids. Threaded from `ctx.auth?.allowedSources` via + * `sourceScopeOpts` in operations.ts. Array wins over scalar `sourceId`. + */ + sourceIds?: string[]; } export interface WhoknowsResult { @@ -165,11 +180,15 @@ export async function findExperts( // 1. Hybrid search with SQL-level types filter (v0.33 typeFilter parameter). // Disable salience + recency boosts in hybridSearch — we apply our own // locked formula on top of the raw relevance score. + // v0.34.1 (#861, D3): thread source-scope so an authenticated MCP + // client only ranks experts within its accessible sources. const results: SearchResult[] = await hybridSearch(engine, opts.topic, { types, limit: innerLimit, salience: 'off', recency: 'off', + sourceId: opts.sourceId, + sourceIds: opts.sourceIds, }); if (results.length === 0) return []; diff --git a/src/core/ai/gateway.ts b/src/core/ai/gateway.ts index f6deb2e..eef0ee2 100644 --- a/src/core/ai/gateway.ts +++ b/src/core/ai/gateway.ts @@ -1016,11 +1016,18 @@ export async function embedMultimodal(inputs: MultimodalInput[]): Promise { + // Auth resolution via the gateway's canonical helper so LiteLLM-style + // optional-auth recipes (Authorization: Bearer LITELLM_API_KEY) and + // hard-required-auth recipes (OpenAI Authorization: Bearer + // OPENAI_API_KEY) both work via the same code path. Throws AIConfigError + // when required env is missing. + const authResult = recipe.resolveAuth + ? recipe.resolveAuth(cfg.env) + : defaultResolveAuth(recipe, cfg.env, 'embedding'); + const baseUrl = cfg.base_urls?.[recipe.id] ?? recipe.base_url_default; + if (!baseUrl) { + throw new AIConfigError( + `${recipe.name} requires a base URL for multimodal embedding.`, + recipe.setup_hint, + ); + } + + // D12 — dim validation. Prefer recipe's declared default_dims when set; + // fall back to the brain's configured embedding_dimensions. If neither + // is known (LiteLLM recipe with default_dims=0 and no config override), + // we skip the dim check rather than fabricate an expected value — the + // engine's vector(N) column will reject mismatched rows at INSERT time + // with a clearer error than anything we could throw here. + const recipeDims = recipe.touchpoints.embedding?.default_dims ?? 0; + const expectedDims = recipeDims > 0 + ? recipeDims + : (cfg.embedding_dimensions ?? 0); + + // Send each input as one /embeddings request. Most providers cap the + // number of inputs per call at the text-embedding batch limit, but the + // multimodal content array varies per provider. Single-input requests + // are the safe lowest common denominator; LiteLLM's proxy backend + // batches internally if it can. + const allEmbeddings: Float32Array[] = []; + for (const input of inputs) { + const body = { + model: modelId, + input: [ + { + // OpenAI's documented multimodal content shape. The data-URL + // form embeds the image bytes inline so the proxy doesn't need + // network access to fetch the image. + type: 'image_url', + image_url: { url: `data:${input.mime};base64,${input.data}` }, + }, + ], + }; + + let res: Response; + try { + res = await fetch(`${baseUrl}/embeddings`, { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + [authResult.headerName]: authResult.token, + }, + body: JSON.stringify(body), + }); + } catch (err) { + throw normalizeAIError(err, `embedMultimodal(${recipe.id}:${modelId})`); + } + + if (!res.ok) { + const text = await res.text().catch(() => ''); + if (res.status === 401 || res.status === 403) { + const requiredKey = recipe.auth_env?.required[0]; + throw new AIConfigError( + `${recipe.name} multimodal returned ${res.status}: ${text || 'auth failed'}.`, + requiredKey + ? `Re-export ${requiredKey} or rotate the key at ${recipe.auth_env?.setup_url ?? recipe.setup_hint}.` + : recipe.setup_hint, + ); + } + // Surface the upstream error verbatim — 400s here usually mean the + // proxied model doesn't support multimodal input. The error text is + // the user's best signal for picking a different model id. + throw new AITransientError( + `${recipe.name} multimodal returned ${res.status}: ${text || 'transient error'}.`, + ); + } + + let parsedBody: { data?: Array<{ embedding: number[] }> }; + try { + parsedBody = (await res.json()) as { data?: Array<{ embedding: number[] }> }; + } catch (err) { + throw new AITransientError( + `${recipe.name} multimodal returned malformed JSON: ${err instanceof Error ? err.message : String(err)}.`, + ); + } + if (!parsedBody.data || !Array.isArray(parsedBody.data) || parsedBody.data.length < 1) { + throw new AITransientError( + `${recipe.name} multimodal returned no embeddings (expected 1).`, + ); + } + + const row = parsedBody.data[0]; + if (!Array.isArray(row.embedding)) { + throw new AITransientError( + `${recipe.name} multimodal returned non-array embedding payload.`, + ); + } + // D12 — dim validation. Throw EmbedDimensionMismatchError-shape error + // (AIConfigError with model id + observed + expected so the operator + // can diagnose and pick a compatible model OR adjust the brain's + // embedding_dimensions config). Skip the check when expectedDims=0 + // (no recipe declaration AND no config override). + if (expectedDims > 0 && row.embedding.length !== expectedDims) { + throw new AIConfigError( + `${recipe.id}:${modelId} returned ${row.embedding.length}-dim vector; expected ${expectedDims}.`, + `The brain's embedding column is fixed at ${expectedDims} dims; this model is incompatible. ` + + `Either pick a model that returns ${expectedDims} dims, OR set --embedding-dimensions ${row.embedding.length} ` + + `and reinitialize the embedding column at the new width.`, + ); + } + allEmbeddings.push(new Float32Array(row.embedding)); + } + + return allEmbeddings; +} + // ---- Expansion ---- async function resolveExpansionProvider(modelStr: string): Promise<{ model: any; recipe: Recipe; modelId: string }> { diff --git a/src/core/ai/recipes/litellm-proxy.ts b/src/core/ai/recipes/litellm-proxy.ts index d7ee4e9..bc82b63 100644 --- a/src/core/ai/recipes/litellm-proxy.ts +++ b/src/core/ai/recipes/litellm-proxy.ts @@ -30,6 +30,15 @@ export const litellmProxy: Recipe = { // LiteLLM's batch capacity is determined by the backend it proxies; // no static cap to declare here. v0.32 (#779). no_batch_cap: true, + // v0.34.1 (#875): LiteLLM can forward to multimodal providers (OpenAI, + // Gemini, Voyage etc.). embedMultimodal routes openai-compatible + // recipes through embedMultimodalOpenAICompat() — same /embeddings + // endpoint as text, with content arrays carrying image_base64 + // entries. No multimodal_models allow-list: the user knows which of + // their proxied models support multimodal; we trust the model id and + // surface the provider's rejection (D12 dim-validation catches + // mismatched-dim responses pre-storage). + supports_multimodal: true, }, }, setup_hint: 'Run LiteLLM (https://docs.litellm.ai) in front of any provider; set LITELLM_BASE_URL + pass --embedding-model litellm: and --embedding-dimensions .', diff --git a/src/core/engine.ts b/src/core/engine.ts index 0f64a08..24b4703 100644 --- a/src/core/engine.ts +++ b/src/core/engine.ts @@ -629,18 +629,31 @@ export interface BrainEngine { dirPrefix?: string, minSimilarity?: number, ): Promise<{ slug: string; similarity: number } | null>; - traverseGraph(slug: string, depth?: number): Promise; + /** + * v0.34.1 (#861 — P0 leak seal): `opts.sourceId` / `opts.sourceIds` + * constrain visited nodes to a single source or array of sources. + * Pre-fix, the walk ignored source scope and an authenticated MCP + * client could enumerate cross-source topology + page metadata via + * the graph op. MCP-bound callers MUST pass the auth'd scope; local + * CLI callers omit it for the historical unscoped behavior. + */ + traverseGraph( + slug: string, + depth?: number, + opts?: { sourceId?: string; sourceIds?: string[] }, + ): Promise; /** * Edge-based graph traversal with optional type and direction filters. * Returns a list of edges (GraphPath[]) instead of nodes. Supports: * - linkType: per-edge filter, only follows matching edges (per-edge semantics) * - direction: 'in' (follow to->from), 'out' (follow from->to), 'both' * - depth: max depth from root (default 5) + * - sourceId/sourceIds: v0.34.1 source-isolation filter, see traverseGraph * Uses cycle prevention (visited array in recursive CTE). */ traversePaths( slug: string, - opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both' }, + opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] }, ): Promise; /** * For a list of slugs, return how many inbound links each has. diff --git a/src/core/migrate.ts b/src/core/migrate.ts index 54fe792..9077237 100644 --- a/src/core/migrate.ts +++ b/src/core/migrate.ts @@ -2969,6 +2969,187 @@ export const MIGRATIONS: Migration[] = [ ON search_telemetry (date DESC); `, }, + { + version: 60, + name: 'oauth_clients_source_id_fk', + // v0.34.1 (#861 + D4 + D10 + D13 — P0 source-isolation leak seal). + // + // Adds oauth_clients.source_id, validates ALL existing rows can map to a + // real source row, backfills NULL → 'default', and installs the FK with + // ON DELETE SET NULL. PR #861's original migration claimed v47-v51; we + // re-number to v60 because the branch already shipped through v54. + // + // D10 (codex outside-voice push-back): fail loud when stale source_id + // rows exist instead of silently widening to NULL. Pre-fix this column + // didn't exist; the only way a row has source_id IS a manual SQL poke, + // so the stale-row branch fires only on operator-modified brains. The + // GBRAIN_ACCEPT_SILENT_WIDEN=1 env var is the explicit opt-in for + // operators who'd rather upgrade than psql-fix. Doctor surfaces orphan + // rows post-clean via the v0.34.x follow-up TODO. + // + // D13: backfill NULL → 'default' BEFORE the FK ADD preserves the v0.33 + // effective behavior (legacy unscoped clients silently fell back to + // 'default' via serve-http.ts:929 cast). Verify 'default' exists in + // sources first — fresh brains have it from sources schema's default + // seed; brains that scripted it out would otherwise wedge here. + // + // PGLite parity via the same DO blocks. PGLite supports DO/EXCEPTION + // since 0.3; no engine branch needed. + idempotent: true, + sql: ` + -- v0.34.1 (#861 + D2 + D13 — P0 source-isolation leak seal). + -- + -- This migration is intentionally lean: oauth_clients.source_id did + -- NOT exist pre-v60, so the only state we inherit from upgrade is + -- "rows with NULL source_id." Backfill those to 'default' (D13: + -- preserves the pre-v0.34 effective fallback behavior verbatim) and + -- install the FK with ON DELETE SET NULL. + -- + -- D10 pre-clean is NOT NEEDED here: codex flagged the silent-widen + -- footgun assuming source_id was an existing column with possibly-stale + -- values. Since the column is brand new in this migration, the only + -- post-backfill values are 'default' (which we just verified exists + -- via the FK contract) plus any NULL the backfill left untouched + -- because of WHERE-clause filtering — none possible. The + -- GBRAIN_ACCEPT_SILENT_WIDEN env-flag stays in the runner for future + -- migrations that need it; this one doesn't. + + -- 1. Add the column. NULL for every existing row. + ALTER TABLE oauth_clients ADD COLUMN IF NOT EXISTS source_id TEXT; + + -- 2. Backfill NULL → 'default'. Pre-v0.34 legacy clients then map + -- to the same source the serve-http fallback chain used to put + -- them in implicitly. No-op on fresh installs (no rows yet). + UPDATE oauth_clients SET source_id = 'default' WHERE source_id IS NULL; + + -- 3. Install FK if not already present. The PGLite + Postgres fresh- + -- install schemas (src/core/pglite-schema.ts, src/schema.sql) now + -- include the FK inline on the CREATE TABLE, so this DO block + -- skips on fresh installs and only fires on upgrade brains where + -- oauth_clients was created pre-v60 without the FK. ON DELETE SET + -- NULL matches the original PR #861 posture; #876 later flips to + -- RESTRICT once federated_read provides the alternative + -- scope-recovery path. + DO $$ + BEGIN + IF NOT EXISTS ( + SELECT 1 FROM pg_constraint + WHERE conname = 'oauth_clients_source_id_fkey' + ) THEN + ALTER TABLE oauth_clients + ADD CONSTRAINT oauth_clients_source_id_fkey + FOREIGN KEY (source_id) REFERENCES sources(id) ON DELETE SET NULL; + END IF; + END $$; + + -- 4. Index for token-verification lookups (verifyAccessToken's JOIN + -- on oauth_clients.client_id → c.source_id). oauth_clients stays + -- small so plain CREATE INDEX (no CONCURRENTLY) is fine. + CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id + ON oauth_clients(source_id) WHERE source_id IS NOT NULL; + `, + }, + { + version: 61, + name: 'oauth_clients_federated_read_column', + // v0.34.1 (#876): add federated_read TEXT[] for the read-side + // federation feature. source_id (v60) is the WRITE-authority axis; + // federated_read is the READ-scope axis. A client can write to ONE + // source while reading from N (a "WeCare L3 dept" client writes to + // dept-x and reads dept-x + parent canon + shared canon). + // + // Default '{}' (empty array) on column add — pre-existing rows get + // backfilled in v62 with an explicit CASE so the array reflects the + // client's current scope rather than the column default. + idempotent: true, + sql: ` + ALTER TABLE oauth_clients ADD COLUMN IF NOT EXISTS federated_read TEXT[] NOT NULL DEFAULT '{}'; + `, + }, + { + version: 62, + name: 'oauth_clients_federated_read_backfill', + // v0.34.1 (#876, F5 — codex outside-voice fix). Backfill federated_read + // with explicit CASE so source_id IS NULL doesn't produce an ambiguous + // array containing NULL. Three cases: + // - source_id IS NULL → '{}' (empty read scope; legacy unscoped + // clients lost their implicit fallback in v60 backfill to 'default', + // so this branch fires only when an operator explicitly NULL'd + // source_id after migration). + // - source_id IS NOT NULL → ARRAY[source_id] (read scope matches + // write scope, the pre-federation default). + // Only fires on rows where federated_read is still the column default + // ({}). Operators who hand-set federated_read keep their config. + idempotent: true, + sql: ` + UPDATE oauth_clients + SET federated_read = CASE + WHEN source_id IS NULL THEN '{}'::text[] + ELSE ARRAY[source_id] + END + WHERE federated_read = '{}'::text[]; + `, + }, + { + version: 63, + name: 'oauth_clients_federated_read_validate', + // v0.34.1 (#876): post-backfill validation. Every client with a + // non-NULL source_id should now have its source_id reflected in + // federated_read. Fail loud if backfill missed a row — points at a + // logic bug in v62's WHERE clause. + idempotent: true, + sql: ` + DO $$ + DECLARE + bad_count INT; + BEGIN + SELECT count(*) INTO bad_count FROM oauth_clients + WHERE source_id IS NOT NULL + AND NOT (source_id = ANY(federated_read)); + IF bad_count > 0 THEN + RAISE EXCEPTION 'oauth_clients has % rows where source_id is not in federated_read after v62 backfill. This is a bug in v62 — re-run gbrain apply-migrations --force-retry 62.', bad_count; + END IF; + END $$; + `, + }, + { + version: 64, + name: 'oauth_clients_source_id_fk_restrict', + // v0.34.1 (#876): flip the source_id FK from ON DELETE SET NULL (v60 + // posture) to ON DELETE RESTRICT now that federated_read provides + // the alternative scope-loss path. Pre-fix, deleting a source could + // silently widen any oauth_client to super-reader (source_id → NULL). + // Post-flip, source delete is refused if any client references it; + // the operator's path is "revoke or re-scope the clients first." + idempotent: true, + sql: ` + DO $$ + BEGIN + IF EXISTS ( + SELECT 1 FROM pg_constraint + WHERE conname = 'oauth_clients_source_id_fkey' + ) THEN + ALTER TABLE oauth_clients DROP CONSTRAINT oauth_clients_source_id_fkey; + END IF; + ALTER TABLE oauth_clients + ADD CONSTRAINT oauth_clients_source_id_fkey + FOREIGN KEY (source_id) REFERENCES sources(id) ON DELETE RESTRICT; + END $$; + `, + }, + { + version: 65, + name: 'oauth_clients_federated_read_gin_index', + // v0.34.1 (#876): GIN index for array-containment lookups + // (`WHERE p.source_id = ANY(federated_read)` and similar). The five + // read-side ops fall back to scalar sourceId when no auth is set, so + // this index only matters under load on federated-scoped clients. + idempotent: true, + sql: ` + CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read + ON oauth_clients USING GIN (federated_read); + `, + }, ]; export const LATEST_VERSION = MIGRATIONS.length > 0 diff --git a/src/core/oauth-provider.ts b/src/core/oauth-provider.ts index c31188d..501fad2 100644 --- a/src/core/oauth-provider.ts +++ b/src/core/oauth-provider.ts @@ -139,9 +139,15 @@ class GBrainClientsStore implements OAuthRegisteredClientsStore { `; if (rows.length === 0) return undefined; const r = rows[0]; + // v0.34.1 (#909): public clients (token_endpoint_auth_method='none') + // have client_secret_hash = NULL. Normalize SQL NULL to JS undefined + // so SDK middleware that checks `client.client_secret === undefined` + // (not `=== null`) correctly identifies the client as public and + // skips the secret-comparison branch on /token. + const rawSecret = r.client_secret_hash; return { client_id: r.client_id as string, - client_secret: r.client_secret_hash as string | undefined, + client_secret: rawSecret == null ? undefined : (rawSecret as string), client_name: r.client_name as string, redirect_uris: (r.redirect_uris as string[]) || [], grant_types: (r.grant_types as string[]) || ['client_credentials'], @@ -170,27 +176,97 @@ class GBrainClientsStore implements OAuthRegisteredClientsStore { assertAllowedScopes(parseScopeString(client.scope)); const clientId = generateToken('gbrain_cl_'); - const clientSecret = generateToken('gbrain_cs_'); - const secretHash = hashToken(clientSecret); + // v0.34.1 (#909): RFC 7591 §2 — clients that authenticate at the token + // endpoint via PKCE alone declare `token_endpoint_auth_method: "none"`. + // For those clients the authorization server MUST NOT issue a client + // secret. Pre-fix, unconditional secret generation made the MCP SDK's + // clientAuth middleware check `client.client_secret` on every request, + // rejecting valid public-client (Claude Code, Cursor) flows. + // + // We persist secret_hash = NULL for public clients so `getClient` and + // the SDK's clientAuth path can detect them via `client_secret_hash IS + // NULL` and skip the secret comparison. Confidential clients (default + // `client_secret_post` and explicit `client_secret_basic`) still mint + // a secret as before. + const authMethod = client.token_endpoint_auth_method || 'client_secret_post'; + const isPublicClient = authMethod === 'none'; + const clientSecret = isPublicClient ? undefined : generateToken('gbrain_cs_'); + const secretHash = clientSecret ? hashToken(clientSecret) : null; const now = Math.floor(Date.now() / 1000); - await this.sql` - INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, - grant_types, scope, token_endpoint_auth_method, - client_id_issued_at) - VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'}, - ${pgArray((client.redirect_uris || []).map(String))}, - ${pgArray(client.grant_types || ['client_credentials'])}, - ${client.scope || ''}, ${client.token_endpoint_auth_method || 'client_secret_post'}, - ${now}) - `; + // v0.34.1 (#861, D2 + D13 + #876): DCR clients get source_id='default' + // (matches legacy fallback) and federated_read=['default'] (read scope + // == write scope). Operators who need narrower / wider scope rescope + // via the CLI later. Pre-v60/v61 brain falls through to the legacy + // projection (no source_id / federated_read column yet). + try { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, token_endpoint_auth_method, + client_id_issued_at, source_id, federated_read) + VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'}, + ${pgArray((client.redirect_uris || []).map(String))}, + ${pgArray(client.grant_types || ['client_credentials'])}, + ${client.scope || ''}, ${authMethod}, + ${now}, ${'default'}, ${pgArray(['default'])}) + `; + } catch (err) { + if (isUndefinedColumnError(err, 'federated_read')) { + try { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, token_endpoint_auth_method, + client_id_issued_at, source_id) + VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'}, + ${pgArray((client.redirect_uris || []).map(String))}, + ${pgArray(client.grant_types || ['client_credentials'])}, + ${client.scope || ''}, ${authMethod}, + ${now}, ${'default'}) + `; + } catch (err2) { + if (isUndefinedColumnError(err2, 'source_id')) { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, token_endpoint_auth_method, + client_id_issued_at) + VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'}, + ${pgArray((client.redirect_uris || []).map(String))}, + ${pgArray(client.grant_types || ['client_credentials'])}, + ${client.scope || ''}, ${authMethod}, + ${now}) + `; + } else { + throw err2; + } + } + } else if (isUndefinedColumnError(err, 'source_id')) { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, token_endpoint_auth_method, + client_id_issued_at) + VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'}, + ${pgArray((client.redirect_uris || []).map(String))}, + ${pgArray(client.grant_types || ['client_credentials'])}, + ${client.scope || ''}, ${authMethod}, + ${now}) + `; + } else { + throw err; + } + } - return { + // Public clients: omit `client_secret` entirely from the response so + // the wire payload matches RFC 7591 §3.2.1 ("if the client is a + // public client, the authorization server MUST NOT issue a client + // secret"). Confidential clients return the freshly-generated secret + // exactly once — same shape as before. + const response: OAuthClientInformationFull = { ...client, client_id: clientId, - client_secret: clientSecret, client_id_issued_at: now, }; + if (clientSecret) response.client_secret = clientSecret; + return response; } } @@ -403,15 +479,57 @@ export class GBrainOAuthProvider implements OAuthServerProvider { const now = Math.floor(Date.now() / 1000); // Try OAuth tokens first. JOIN oauth_clients in the same query so - // verifyAccessToken returns client_name in AuthInfo — eliminates the - // separate per-request lookup at serve-http.ts that was the N+1 hot - // path (see PR #586 review D14=B). - const oauthRows = await this.sql` - SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name - FROM oauth_tokens t - LEFT JOIN oauth_clients c ON c.client_id = t.client_id - WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' - `; + // verifyAccessToken returns client_name AND source_id in AuthInfo — + // eliminates the separate per-request lookup at serve-http.ts that + // was the N+1 hot path (see PR #586 review D14=B; v0.34.1 #861 D2 + // adds the source_id thread on the same JOIN). + // + // v0.34.1 (#861): the JOIN guards on a c.source_id column that + // migration v60 adds. Pre-v60 brains throw a "column does not exist" + // error here — caught at the boundary via isUndefinedColumnError so + // unmigrated brains degrade to "no source scope" rather than refusing + // every token verification. + let oauthRows: Record[]; + try { + oauthRows = await this.sql` + SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name, + c.source_id, c.federated_read + FROM oauth_tokens t + LEFT JOIN oauth_clients c ON c.client_id = t.client_id + WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' + `; + } catch (err) { + // v0.34.1: pre-v60 brain → source_id column missing. Pre-v61 brain → + // federated_read column missing. Both classes degrade to legacy + // projection so auth keeps working until the operator runs + // apply-migrations. Probe both column names so partial-upgrade brains + // (v60 applied but v61 didn't yet) also fall through cleanly. + if (isUndefinedColumnError(err, 'source_id') || isUndefinedColumnError(err, 'federated_read')) { + // Try the v60-only projection first (source_id but no federated_read). + try { + oauthRows = await this.sql` + SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name, c.source_id + FROM oauth_tokens t + LEFT JOIN oauth_clients c ON c.client_id = t.client_id + WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' + `; + } catch (err2) { + if (isUndefinedColumnError(err2, 'source_id')) { + // Truly pre-v60: no source_id either. Pre-v0.34 projection. + oauthRows = await this.sql` + SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name + FROM oauth_tokens t + LEFT JOIN oauth_clients c ON c.client_id = t.client_id + WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' + `; + } else { + throw err2; + } + } + } else { + throw err; + } + } if (oauthRows.length > 0) { const row = oauthRows[0]; @@ -422,6 +540,15 @@ export class GBrainOAuthProvider implements OAuthServerProvider { if (expiresAt === undefined || expiresAt < now) { throw new Error('Token expired'); } + // v0.34.1 (#876): federated_read normalization. SELECT returns + // either a JS array (Postgres / PGLite text[] driver mapping) or + // undefined when the legacy projection ran (pre-v61 brain). Empty + // array vs undefined matters: empty array = explicit no-federated- + // read; undefined = column missing on this brain. + const federatedRaw = row.federated_read; + const allowedSources = Array.isArray(federatedRaw) + ? (federatedRaw as string[]) + : undefined; return { token, clientId: row.client_id as string, @@ -429,6 +556,14 @@ export class GBrainOAuthProvider implements OAuthServerProvider { scopes: (row.scopes as string[]) || [], expiresAt, resource: row.resource ? new URL(row.resource as string) : undefined, + // v0.34.1 (#861, D2): source-isolation scope from oauth_clients. + // Undefined when the row predates v60 or when the brain itself + // predates v60 (fell through to the legacy projection above). + sourceId: (row.source_id as string | null) ?? undefined, + // v0.34.1 (#876): federated read scope. sourceScopeOpts in + // operations.ts prefers this array over scalar sourceId when set + // and non-empty. + allowedSources, } as AuthInfo; } @@ -452,6 +587,12 @@ export class GBrainOAuthProvider implements OAuthServerProvider { clientName: name, scopes: ['read', 'write', 'admin'], expiresAt: Math.floor(Date.now() / 1000) + 365 * 24 * 3600, // Legacy tokens never expire — set 1yr future + // v0.34.1 (#861, D13): legacy bearer tokens default to 'default' + // source — matches the pre-v0.34 effective behavior where the + // serve-http transport fell back to GBRAIN_SOURCE/'default' for + // any caller without explicit scope. Operators who want a + // narrower scope for legacy tokens migrate to OAuth. + sourceId: 'default', } as AuthInfo; } @@ -570,6 +711,8 @@ export class GBrainOAuthProvider implements OAuthServerProvider { grantTypes: string[], scopes: string, redirectUris: string[] = [], + sourceId: string = 'default', + federatedRead?: string[], ): Promise<{ clientId: string; clientSecret: string }> { // v0.28: ALLOWED_SCOPES allowlist. Reject `--scopes "read flying-unicorn"` // at registration so meaningless scope strings can't pile up in the DB. @@ -582,12 +725,56 @@ export class GBrainOAuthProvider implements OAuthServerProvider { const secretHash = hashToken(clientSecret); const now = Math.floor(Date.now() / 1000); - await this.sql` - INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, - grant_types, scope, client_id_issued_at) - VALUES (${clientId}, ${secretHash}, ${name}, - ${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now}) - `; + // v0.34.1 (#861 + #876): persist source_id AND federated_read so + // verifyAccessToken can populate both AuthInfo fields. Defaults: + // source_id = 'default' (matches v60 backfill) + // federated_read = [source_id] when omitted (a non-federated client + // has read scope == write scope, the v0.33 default) + const federated = federatedRead && federatedRead.length > 0 ? federatedRead : [sourceId]; + try { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, client_id_issued_at, + source_id, federated_read) + VALUES (${clientId}, ${secretHash}, ${name}, + ${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now}, + ${sourceId}, ${pgArray(federated)}) + `; + } catch (err) { + // Pre-v60 / pre-v61 brain: column missing. Fall back through both + // projections so registration still works until apply-migrations. + if (isUndefinedColumnError(err, 'federated_read')) { + // v60-only brain: source_id but no federated_read. + try { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, client_id_issued_at, source_id) + VALUES (${clientId}, ${secretHash}, ${name}, + ${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now}, ${sourceId}) + `; + } catch (err2) { + if (isUndefinedColumnError(err2, 'source_id')) { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, client_id_issued_at) + VALUES (${clientId}, ${secretHash}, ${name}, + ${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now}) + `; + } else { + throw err2; + } + } + } else if (isUndefinedColumnError(err, 'source_id')) { + await this.sql` + INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, + grant_types, scope, client_id_issued_at) + VALUES (${clientId}, ${secretHash}, ${name}, + ${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now}) + `; + } else { + throw err; + } + } return { clientId, clientSecret }; } diff --git a/src/core/operations.ts b/src/core/operations.ts index 758e5fc..2d763ef 100644 --- a/src/core/operations.ts +++ b/src/core/operations.ts @@ -233,6 +233,36 @@ export interface AuthInfo { clientName?: string; scopes: string[]; expiresAt?: number; + /** + * v0.34.1 (#861, D2): the source the calling OAuth client is scoped + * to (write authority). Sourced from `oauth_clients.source_id` at + * token-verification time. The HTTP transport ALSO threads this + * value into `OperationContext.sourceId` at the same site so op + * handlers can consume it via the canonical `ctx.sourceId` (D2 + * dual-write decision — identity surface symmetric with + * `allowedSources` below). + * + * Undefined for legacy bearer tokens that predate v0.34.1 and for + * clients that haven't been scoped yet. Migration v60 backfills + * NULL → 'default' for pre-existing rows so this field is populated + * on the upgrade path; brand-new public-client registrations may + * still leave it null until an operator explicitly scopes via + * `gbrain auth scope-client`. + */ + sourceId?: string; + /** + * v0.34.1 (#876): array of source ids this OAuth client may READ + * from (federation). Sourced from `oauth_clients.federated_read`. + * Independent of `sourceId` (write authority): a "WeCare L3 dept" + * client can write to `source_id='dept-x'` while reading the union + * of `['dept-x', 'wecare-parent', 'shared']`. + * + * Empty array `[]` means "no federated reads beyond `sourceId`". + * Undefined means "the post-v60 backfill hasn't populated this row + * yet" — engines fall back to scalar `sourceId` filtering in that + * case (back-compat). + */ + allowedSources?: string[]; } export interface OperationContext { @@ -354,6 +384,38 @@ export interface OperationContext { sourceId: string; } +/** + * v0.34.1 (#861, D9 — P0 leak seal): resolve the source-scope filter for a + * read-side op handler. Returns an opts fragment ready to spread into the + * engine call. + * + * Precedence: + * 1. `ctx.auth?.allowedSources` (federated read, #876) → emits + * `{sourceIds: [...]}`. Federated semantics subsume the scalar case. + * 2. `ctx.sourceId` (scalar) → emits `{sourceId: '...'}`. + * 3. Neither set → emits `{}`. Local CLI callers (and tests that don't + * populate ctx) keep the pre-v0.34 unscoped behavior. + * + * Both fields default to the engine's "no filter" behavior individually, + * so unset values are safe — the engine sees the same shape it did + * pre-v0.34. The leak this guards against is an authenticated MCP client + * whose ctx.sourceId IS set but whose engine call was constructed without + * threading it (operations.ts:968/1076/1092/935/1469/1471/2241 pre-fix). + * + * Helper rather than inline so every read-side handler routes through the + * same precedence ladder — drift between sites is the bug class. + */ +export function sourceScopeOpts(ctx: OperationContext): { sourceId?: string; sourceIds?: string[] } { + const allowed = ctx.auth?.allowedSources; + // Treat an empty `allowedSources: []` as "no federated read scope" — the + // op-handler defers to scalar `ctx.sourceId` below. An attacker-controlled + // value of `[]` MUST NOT widen scope to "all sources" by being interpreted + // as "no filter." + if (allowed && allowed.length > 0) return { sourceIds: allowed }; + if (ctx.sourceId) return { sourceId: ctx.sourceId }; + return {}; +} + export interface Operation { name: string; description: string; @@ -940,6 +1002,12 @@ const list_pages: Operation = { const sort = rawSort && (LIST_PAGES_SORT_VALUES as readonly string[]).includes(rawSort) ? (rawSort as ListPagesSort) : undefined; + // v0.34.1 (#861 — P0 leak seal): thread the auth'd client's source scope + // into the listPages filter so an OAuth client scoped to src-A cannot + // enumerate src-B pages. Pre-fix, ctx.sourceId / ctx.auth?.allowedSources + // were ignored at this op handler and the engine returned every source's + // pages indiscriminately. + const scope = sourceScopeOpts(ctx); const pages = await ctx.engine.listPages({ type: p.type as any, tag: p.tag as string, @@ -947,6 +1015,7 @@ const list_pages: Operation = { includeDeleted: (p.include_deleted as boolean) === true, updated_after: typeof p.updated_after === 'string' ? p.updated_after : undefined, sort, + ...scope, }); return pages.map(pg => ({ slug: pg.slug, @@ -973,9 +1042,13 @@ const search: Operation = { handler: async (ctx, p) => { const startedAt = Date.now(); const queryText = p.query as string; + // v0.34.1 (#861 — P0 leak seal): thread caller's source scope into + // searchKeyword. Pre-fix this op silently returned cross-source hits + // for any auth'd OAuth client. const raw = await ctx.engine.searchKeyword(queryText, { limit: (p.limit as number) || 20, offset: (p.offset as number) || 0, + ...sourceScopeOpts(ctx), }); const results = dedupResults(raw); const latency_ms = Date.now() - startedAt; @@ -1086,10 +1159,15 @@ const query: Operation = { const [vec] = await embedMultimodal([ { kind: 'image_base64', data: imageData, mime: imageMime }, ]); + // v0.34.1 (#861 F2 — 6th leak surface): the image path bypasses + // hybridSearch and calls searchVector directly, so it needs its + // own thread of the source scope. Pre-fix, this branch leaked + // image pages across sources independent of the text path's fix. const results = await ctx.engine.searchVector(vec, { limit: (p.limit as number) || 20, offset: (p.offset as number) || 0, embeddingColumn: 'embedding_image', + ...sourceScopeOpts(ctx), }); return results; } @@ -1142,6 +1220,10 @@ const query: Operation = { useCache: typeof p.use_cache === 'boolean' ? (p.use_cache as boolean) : undefined, intentWeighting: typeof p.intent_weighting === 'boolean' ? (p.intent_weighting as boolean) : undefined, onMeta: (m) => { capturedMeta = m; }, + // v0.34.1 (#861 — P0 leak seal): thread caller's source scope. The + // hybridSearch internal searchOpts rebuild (hybrid.ts:223) was + // dropping these fields pre-fix even when callers passed them. + ...sourceScopeOpts(ctx), }); const latency_ms = Date.now() - startedAt; @@ -1500,12 +1582,17 @@ const traverse_graph: Operation = { const depth = Math.max(1, Math.min(requestedDepth, TRAVERSE_DEPTH_CAP)); const linkType = p.link_type as string | undefined; const direction = p.direction as 'in' | 'out' | 'both' | undefined; + // v0.34.1 (#861 — P0 leak seal): thread caller's source scope so graph + // walks stay within the auth'd client's accessible sources. Pre-fix, + // traverseGraph / traversePaths happily followed edges into pages from + // foreign sources, leaking topology + page metadata via the graph op. + const scope = sourceScopeOpts(ctx); // Backward compat: when neither link_type nor direction is provided, return // the legacy GraphNode[] shape. Once either is set, switch to GraphPath[]. if (linkType === undefined && direction === undefined) { - return ctx.engine.traverseGraph(slug, depth); + return ctx.engine.traverseGraph(slug, depth, scope); } - return ctx.engine.traversePaths(slug, { depth, linkType, direction }); + return ctx.engine.traversePaths(slug, { depth, linkType, direction, ...scope }); }, scope: 'read', cliHints: { name: 'graph', positional: ['slug'] }, @@ -2269,16 +2356,22 @@ const find_experts: Operation = { description: 'Include factor breakdown per result (expertise, recency, salience).', }, }, - handler: async (_ctx, p) => { + handler: async (ctx, p) => { const { findExperts } = await import('../commands/whoknows.ts'); const topic = typeof p.topic === 'string' ? p.topic : ''; if (!topic.trim()) { throw new OperationError('invalid_params', '`topic` is required and must be a non-empty string.'); } - return findExperts(_ctx.engine, { + // v0.34.1 (#861, D3 — 5th leak surface): find_experts (whoknows) was + // authored against v0.33 after PR #861 was drafted, so the source-scope + // thread was missing entirely. The op calls findExperts → hybridSearch + // internally; without the thread an auth'd src-A whoknows query would + // surface src-B people in the rankings. + return findExperts(ctx.engine, { topic, limit: typeof p.limit === 'number' ? p.limit : undefined, explain: p.explain === true, + ...sourceScopeOpts(ctx), }); }, cliHints: { name: 'whoknows', positional: ['topic'] }, diff --git a/src/core/pglite-engine.ts b/src/core/pglite-engine.ts index 885ad53..7973581 100644 --- a/src/core/pglite-engine.ts +++ b/src/core/pglite-engine.ts @@ -640,8 +640,12 @@ export class PGLiteEngine implements BrainEngine { params.push(escaped); where.push(`p.slug LIKE $${params.length} ESCAPE '\\'`); } - // v0.31.12: scope to a single source when requested. - if (filters?.sourceId) { + // v0.31.12 + v0.34.1 (#876, D9): scope to a single source OR an array + // of sources. Array form wins (federated subsumes scalar). + if (filters?.sourceIds && filters.sourceIds.length > 0) { + params.push(filters.sourceIds); + where.push(`p.source_id = ANY($${params.length}::text[])`); + } else if (filters?.sourceId) { params.push(filters.sourceId); where.push(`p.source_id = $${params.length}`); } @@ -788,6 +792,14 @@ export class PGLiteEngine implements BrainEngine { params.push(opts.beforeDate); extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861 — P0 leak seal): source-isolation. Array wins over scalar. + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + extraFilter += ` AND p.source_id = $${params.length}`; + } const { rows } = await this.db.query( `WITH ranked AS ( @@ -886,6 +898,14 @@ export class PGLiteEngine implements BrainEngine { params.push(opts.beforeDate); extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861 — P0 leak seal): source-isolation on the CJK fallback path. + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + extraFilter += ` AND p.source_id = $${params.length}`; + } // Bigram-frequency count: count occurrences of $qRaw in chunk_text via // (length(chunk) - length(replace(chunk, q, ''))) / length(q). Acts as @@ -1006,6 +1026,16 @@ export class PGLiteEngine implements BrainEngine { params.push(opts.beforeDate); extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861 — P0 leak seal): source-isolation for the chunk-grain + // anchor primitive. Layer 7 two-pass walks from these anchors so a + // foreign-source anchor would let the walk leak into foreign neighbors. + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + extraFilter += ` AND p.source_id = $${params.length}`; + } // visibilityClause already declared above (v0.32.7: hoisted so CJK branch can reuse). @@ -1083,6 +1113,16 @@ export class PGLiteEngine implements BrainEngine { params.push(opts.beforeDate); extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861, F2 — P0 leak seal): source-isolation in the INNER CTE + // so HNSW candidate pool narrows before re-rank. Mirrors postgres-engine + // placement decision (codex flagged this during plan review). + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + extraFilter += ` AND p.source_id = $${params.length}`; + } // v0.26.5: visibility filter applied in the inner CTE so HNSW sees the // same candidate count it always did. See postgres-engine.ts for rationale. @@ -1516,13 +1556,38 @@ export class PGLiteEngine implements BrainEngine { return { slug: row.slug, similarity: row.sim }; } - async traverseGraph(slug: string, depth: number = 5): Promise { + async traverseGraph( + slug: string, + depth: number = 5, + opts?: { sourceId?: string; sourceIds?: string[] }, + ): Promise { + // v0.34.1 (#861 — P0 leak seal): source-scope filters at seed, step, and + // aggregation subquery. Mirrors postgres-engine.traverseGraph placement. + const params: unknown[] = [slug, depth]; + const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0; + let seedScope = ''; + let stepScope = ''; + let aggScope = ''; + if (useSourceIds) { + params.push(opts!.sourceIds); + const idx = params.length; + seedScope = `AND p.source_id = ANY($${idx}::text[])`; + stepScope = `AND p2.source_id = ANY($${idx}::text[])`; + aggScope = `AND p3.source_id = ANY($${idx}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + const idx = params.length; + seedScope = `AND p.source_id = $${idx}`; + stepScope = `AND p2.source_id = $${idx}`; + aggScope = `AND p3.source_id = $${idx}`; + } + // Cycle prevention: visited array tracks page IDs already in the path. // Prevents exponential blowup on cyclic subgraphs (e.g., A->B->A). const { rows } = await this.db.query( `WITH RECURSIVE graph AS ( SELECT p.id, p.slug, p.title, p.type, 0 as depth, ARRAY[p.id] as visited - FROM pages p WHERE p.slug = $1 + FROM pages p WHERE p.slug = $1 ${seedScope} UNION ALL @@ -1532,6 +1597,7 @@ export class PGLiteEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.to_page_id WHERE g.depth < $2 AND NOT (p2.id = ANY(g.visited)) + ${stepScope} ) SELECT DISTINCT g.slug, g.title, g.type, g.depth, coalesce( @@ -1543,12 +1609,12 @@ export class PGLiteEngine implements BrainEngine { (SELECT jsonb_agg(DISTINCT jsonb_build_object('to_slug', p3.slug, 'link_type', l2.link_type)) FROM links l2 JOIN pages p3 ON p3.id = l2.to_page_id - WHERE l2.from_page_id = g.id), + WHERE l2.from_page_id = g.id ${aggScope}), '[]'::jsonb ) as links FROM graph g ORDER BY g.depth, g.slug`, - [slug, depth] + params ); return (rows as Record[]).map(r => ({ @@ -1562,7 +1628,7 @@ export class PGLiteEngine implements BrainEngine { async traversePaths( slug: string, - opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both' }, + opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] }, ): Promise { const depth = opts?.depth ?? 5; const direction = opts?.direction ?? 'out'; @@ -1571,12 +1637,36 @@ export class PGLiteEngine implements BrainEngine { const params: unknown[] = [slug, depth]; if (linkType !== null) params.push(linkType); + // v0.34.1 (#861 — P0 leak seal): source-scope filters at seed + step + + // final SELECT joins (for the 'both' branch's pf + pt). Mirrors + // postgres-engine.traversePaths placement. + const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0; + let seedScope = ''; + let stepScope = ''; + let pfScope = ''; + let ptScope = ''; + if (useSourceIds) { + params.push(opts!.sourceIds); + const idx = params.length; + seedScope = `AND p.source_id = ANY($${idx}::text[])`; + stepScope = `AND p2.source_id = ANY($${idx}::text[])`; + pfScope = `AND pf.source_id = ANY($${idx}::text[])`; + ptScope = `AND pt.source_id = ANY($${idx}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + const idx = params.length; + seedScope = `AND p.source_id = $${idx}`; + stepScope = `AND p2.source_id = $${idx}`; + pfScope = `AND pf.source_id = $${idx}`; + ptScope = `AND pt.source_id = $${idx}`; + } + let sql: string; if (direction === 'out') { sql = ` WITH RECURSIVE walk AS ( SELECT p.id, p.slug, 0::int AS depth, ARRAY[p.id] AS visited - FROM pages p WHERE p.slug = $1 + FROM pages p WHERE p.slug = $1 ${seedScope} UNION ALL SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id FROM walk w @@ -1585,6 +1675,7 @@ export class PGLiteEngine implements BrainEngine { WHERE w.depth < $2 AND NOT (p2.id = ANY(w.visited)) ${linkTypeWhere} + ${stepScope} ) SELECT w.slug AS from_slug, p2.slug AS to_slug, l.link_type, l.context, w.depth + 1 AS depth @@ -1593,13 +1684,14 @@ export class PGLiteEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.to_page_id WHERE w.depth < $2 ${linkTypeWhere} + ${stepScope} ORDER BY depth, from_slug, to_slug `; } else if (direction === 'in') { sql = ` WITH RECURSIVE walk AS ( SELECT p.id, p.slug, 0::int AS depth, ARRAY[p.id] AS visited - FROM pages p WHERE p.slug = $1 + FROM pages p WHERE p.slug = $1 ${seedScope} UNION ALL SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id FROM walk w @@ -1608,6 +1700,7 @@ export class PGLiteEngine implements BrainEngine { WHERE w.depth < $2 AND NOT (p2.id = ANY(w.visited)) ${linkTypeWhere} + ${stepScope} ) SELECT p2.slug AS from_slug, w.slug AS to_slug, l.link_type, l.context, w.depth + 1 AS depth @@ -1616,6 +1709,7 @@ export class PGLiteEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.from_page_id WHERE w.depth < $2 ${linkTypeWhere} + ${stepScope} ORDER BY depth, from_slug, to_slug `; } else { @@ -1624,7 +1718,7 @@ export class PGLiteEngine implements BrainEngine { sql = ` WITH RECURSIVE walk AS ( SELECT p.id, 0::int AS depth, ARRAY[p.id] AS visited - FROM pages p WHERE p.slug = $1 + FROM pages p WHERE p.slug = $1 ${seedScope} UNION ALL SELECT p2.id, w.depth + 1, w.visited || p2.id FROM walk w @@ -1633,6 +1727,7 @@ export class PGLiteEngine implements BrainEngine { WHERE w.depth < $2 AND NOT (p2.id = ANY(w.visited)) ${linkTypeWhere} + ${stepScope} ) SELECT pf.slug AS from_slug, pt.slug AS to_slug, l.link_type, l.context, w.depth + 1 AS depth @@ -1642,6 +1737,8 @@ export class PGLiteEngine implements BrainEngine { JOIN pages pt ON pt.id = l.to_page_id WHERE w.depth < $2 ${linkTypeWhere} + ${pfScope} + ${ptScope} ORDER BY depth, from_slug, to_slug `; } diff --git a/src/core/pglite-schema.ts b/src/core/pglite-schema.ts index 33660dc..3d05dff 100644 --- a/src/core/pglite-schema.ts +++ b/src/core/pglite-schema.ts @@ -594,8 +594,19 @@ CREATE TABLE IF NOT EXISTS oauth_clients ( client_secret_expires_at BIGINT, token_ttl INTEGER, deleted_at TIMESTAMPTZ, + source_id TEXT REFERENCES sources(id) ON DELETE RESTRICT, + federated_read TEXT[] NOT NULL DEFAULT '{}', created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); +-- v0.34.1 (#861, D13 + #876): source_id is the OAuth client's write-source +-- scope; federated_read is its read-source array (a federated client can +-- read sources beyond its source_id). Migration v60 adds source_id; +-- v61-v65 add federated_read + GIN index + flip FK to RESTRICT. Fresh +-- installs land in the post-migration shape via the inline columns above. +CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id + ON oauth_clients(source_id) WHERE source_id IS NOT NULL; +CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read + ON oauth_clients USING GIN (federated_read); CREATE TABLE IF NOT EXISTS oauth_tokens ( token_hash TEXT PRIMARY KEY, diff --git a/src/core/postgres-engine.ts b/src/core/postgres-engine.ts index e1efb49..e57aab1 100644 --- a/src/core/postgres-engine.ts +++ b/src/core/postgres-engine.ts @@ -685,10 +685,14 @@ export class PostgresEngine implements BrainEngine { const slugCondition = slugPrefix ? sql`AND p.slug LIKE ${slugPrefix.replace(/[\\%_]/g, (c) => '\\' + c) + '%'} ESCAPE '\\'` : sql``; - // v0.31.12: scope to a single source when requested. - const sourceCondition = filters?.sourceId - ? sql`AND p.source_id = ${filters.sourceId}` - : sql``; + // v0.31.12 + v0.34.1 (#876, D9): scope to a single source OR an array + // of sources. When BOTH are set, the array wins (federated semantics + // subsume the scalar case). When neither is set, no filter applies. + const sourceCondition = filters?.sourceIds && filters.sourceIds.length > 0 + ? sql`AND p.source_id = ANY(${filters.sourceIds}::text[])` + : filters?.sourceId + ? sql`AND p.source_id = ${filters.sourceId}` + : sql``; // v0.26.5: hide soft-deleted by default; opt in via filters.includeDeleted. const deletedCondition = filters?.includeDeleted === true ? sql`` @@ -828,6 +832,22 @@ export class PostgresEngine implements BrainEngine { params.push(opts.beforeDate); beforeDateClause = `AND COALESCE(p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861 — P0 leak seal): source-isolation filter. When the + // caller's auth scope is set, narrow the inner CTE candidate set so + // an authenticated MCP client cannot see foreign-source pages via + // keyword search. Array form wins over scalar (federated subsumes + // single-source). Index-backed by idx_pages_source_id; the filter is + // pushed to the INNER CTE specifically so HNSW-style downstream + // ranking sees a narrowed candidate set rather than re-ranking a + // cross-source pool. + let sourceClause = ''; + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + sourceClause = `AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + sourceClause = `AND p.source_id = $${params.length}`; + } params.push(innerLimit); const innerLimitParam = `$${params.length}`; params.push(limit); @@ -859,6 +879,7 @@ export class PostgresEngine implements BrainEngine { ${symbolKindClause} ${afterDateClause} ${beforeDateClause} + ${sourceClause} ${hardExcludeClause} ${visibilityClause} -- v0.27.1: hide image rows from text-keyword search so OCR text @@ -962,6 +983,17 @@ export class PostgresEngine implements BrainEngine { params.push(opts.beforeDate); beforeDateClause = `AND COALESCE(p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861 — P0 leak seal): source-isolation. Anchor primitive + // for two-pass retrieval, so cross-source anchors would let the walk + // discover foreign-source neighbors. Filter at chunk-rank time. + let sourceClause = ''; + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + sourceClause = `AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + sourceClause = `AND p.source_id = $${params.length}`; + } params.push(limit); const limitParam = `$${params.length}`; params.push(offset); @@ -988,6 +1020,7 @@ export class PostgresEngine implements BrainEngine { ${symbolKindClause} ${afterDateClause} ${beforeDateClause} + ${sourceClause} ${hardExcludeClause} ${visibilityClause} ORDER BY score DESC @@ -1071,6 +1104,19 @@ export class PostgresEngine implements BrainEngine { params.push(opts.beforeDate); beforeDateClause = `AND COALESCE(p.updated_at, p.created_at) < $${params.length}::timestamptz`; } + // v0.34.1 (#861, F2 — P0 leak seal): source-isolation in the INNER CTE + // specifically. Pushing the filter inside narrows the HNSW candidate set + // before re-rank; pushing it to the outer SELECT would force HNSW to + // over-fetch then post-filter, wasting candidate slots. Codex flagged + // this placement during plan review. Array form wins over scalar. + let sourceClause = ''; + if (opts?.sourceIds && opts.sourceIds.length > 0) { + params.push(opts.sourceIds); + sourceClause = `AND p.source_id = ANY($${params.length}::text[])`; + } else if (opts?.sourceId) { + params.push(opts.sourceId); + sourceClause = `AND p.source_id = $${params.length}`; + } params.push(innerLimit); const innerLimitParam = `$${params.length}`; params.push(limit); @@ -1106,6 +1152,7 @@ export class PostgresEngine implements BrainEngine { ${symbolKindClause} ${afterDateClause} ${beforeDateClause} + ${sourceClause} ${hardExcludeClause} ${visibilityClause} ORDER BY cc.${col} <=> $1::vector @@ -1516,13 +1563,40 @@ export class PostgresEngine implements BrainEngine { return { slug: row.slug, similarity: row.sim }; } - async traverseGraph(slug: string, depth: number = 5): Promise { + async traverseGraph( + slug: string, + depth: number = 5, + opts?: { sourceId?: string; sourceIds?: string[] }, + ): Promise { const sql = this.sql; + // v0.34.1 (#861 — P0 leak seal): scope visited nodes to the caller's + // source(s). Without this, the walk follows edges into pages from + // foreign sources, leaking topology + page metadata. The filter + // applies at BOTH the seed (root must be in scope) AND the recursive + // step (every visited neighbor must be in scope). The aggregation + // subquery also filters so the per-node `links` array only includes + // edges to in-scope pages. + const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0; + const seedScope = useSourceIds + ? sql`AND p.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND p.source_id = ${opts.sourceId}` + : sql``; + const stepScope = useSourceIds + ? sql`AND p2.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND p2.source_id = ${opts.sourceId}` + : sql``; + const aggScope = useSourceIds + ? sql`AND p3.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND p3.source_id = ${opts.sourceId}` + : sql``; // Cycle prevention: visited array tracks page IDs already in the path. const rows = await sql` WITH RECURSIVE graph AS ( SELECT p.id, p.slug, p.title, p.type, 0 as depth, ARRAY[p.id] as visited - FROM pages p WHERE p.slug = ${slug} + FROM pages p WHERE p.slug = ${slug} ${seedScope} UNION ALL @@ -1532,6 +1606,7 @@ export class PostgresEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.to_page_id WHERE g.depth < ${depth} AND NOT (p2.id = ANY(g.visited)) + ${stepScope} ) SELECT DISTINCT g.slug, g.title, g.type, g.depth, coalesce( @@ -1545,7 +1620,7 @@ export class PostgresEngine implements BrainEngine { (SELECT jsonb_agg(DISTINCT jsonb_build_object('to_slug', p3.slug, 'link_type', l2.link_type)) FROM links l2 JOIN pages p3 ON p3.id = l2.to_page_id - WHERE l2.from_page_id = g.id), + WHERE l2.from_page_id = g.id ${aggScope}), '[]'::jsonb ) as links FROM graph g @@ -1563,20 +1638,47 @@ export class PostgresEngine implements BrainEngine { async traversePaths( slug: string, - opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both' }, + opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] }, ): Promise { const sql = this.sql; const depth = opts?.depth ?? 5; const direction = opts?.direction ?? 'out'; const linkType = opts?.linkType ?? null; const linkTypeMatches = linkType !== null; + // v0.34.1 (#861 — P0 leak seal): source-scope filter fragments. Applied + // at seed (root must be in scope) AND at every recursive step (neighbor + // must be in scope) AND in the SELECT join (final edges respect scope). + // The 'both' branch needs filters on BOTH endpoint joins. + const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0; + const seedScope = useSourceIds + ? sql`AND p.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND p.source_id = ${opts.sourceId}` + : sql``; + const stepScope = useSourceIds + ? sql`AND p2.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND p2.source_id = ${opts.sourceId}` + : sql``; + // For the 'both' direction's final SELECT, both endpoint joins (pf, pt) + // get scope filters so edges crossing into a foreign source are dropped. + const pfScope = useSourceIds + ? sql`AND pf.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND pf.source_id = ${opts.sourceId}` + : sql``; + const ptScope = useSourceIds + ? sql`AND pt.source_id = ANY(${opts!.sourceIds!}::text[])` + : opts?.sourceId + ? sql`AND pt.source_id = ${opts.sourceId}` + : sql``; let rows; if (direction === 'out') { rows = await sql` WITH RECURSIVE walk AS ( SELECT p.id, p.slug, 0::int as depth, ARRAY[p.id] as visited - FROM pages p WHERE p.slug = ${slug} + FROM pages p WHERE p.slug = ${slug} ${seedScope} UNION ALL SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id FROM walk w @@ -1585,6 +1687,7 @@ export class PostgresEngine implements BrainEngine { WHERE w.depth < ${depth} AND NOT (p2.id = ANY(w.visited)) AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${stepScope} ) SELECT w.slug as from_slug, p2.slug as to_slug, l.link_type, l.context, w.depth + 1 as depth @@ -1593,13 +1696,14 @@ export class PostgresEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.to_page_id WHERE w.depth < ${depth} AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${stepScope} ORDER BY depth, from_slug, to_slug `; } else if (direction === 'in') { rows = await sql` WITH RECURSIVE walk AS ( SELECT p.id, p.slug, 0::int as depth, ARRAY[p.id] as visited - FROM pages p WHERE p.slug = ${slug} + FROM pages p WHERE p.slug = ${slug} ${seedScope} UNION ALL SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id FROM walk w @@ -1608,6 +1712,7 @@ export class PostgresEngine implements BrainEngine { WHERE w.depth < ${depth} AND NOT (p2.id = ANY(w.visited)) AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${stepScope} ) SELECT p2.slug as from_slug, w.slug as to_slug, l.link_type, l.context, w.depth + 1 as depth @@ -1616,13 +1721,14 @@ export class PostgresEngine implements BrainEngine { JOIN pages p2 ON p2.id = l.from_page_id WHERE w.depth < ${depth} AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${stepScope} ORDER BY depth, from_slug, to_slug `; } else { rows = await sql` WITH RECURSIVE walk AS ( SELECT p.id, 0::int as depth, ARRAY[p.id] as visited - FROM pages p WHERE p.slug = ${slug} + FROM pages p WHERE p.slug = ${slug} ${seedScope} UNION ALL SELECT p2.id, w.depth + 1, w.visited || p2.id FROM walk w @@ -1631,6 +1737,7 @@ export class PostgresEngine implements BrainEngine { WHERE w.depth < ${depth} AND NOT (p2.id = ANY(w.visited)) AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${stepScope} ) SELECT pf.slug as from_slug, pt.slug as to_slug, l.link_type, l.context, w.depth + 1 as depth @@ -1640,6 +1747,8 @@ export class PostgresEngine implements BrainEngine { JOIN pages pt ON pt.id = l.to_page_id WHERE w.depth < ${depth} AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''}) + ${pfScope} + ${ptScope} ORDER BY depth, from_slug, to_slug `; } diff --git a/src/core/schema-embedded.ts b/src/core/schema-embedded.ts index 0d8f800..dbfb707 100644 --- a/src/core/schema-embedded.ts +++ b/src/core/schema-embedded.ts @@ -428,8 +428,17 @@ CREATE TABLE IF NOT EXISTS oauth_clients ( client_secret_expires_at BIGINT, token_ttl INTEGER, deleted_at TIMESTAMPTZ, + source_id TEXT REFERENCES sources(id) ON DELETE RESTRICT, + federated_read TEXT[] NOT NULL DEFAULT '{}', created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); +-- v0.34.1 (#861, D13 + #876): source_id is the write-source scope; +-- federated_read is the read-source array. Migrations v60-v65 land both +-- columns on upgrade; fresh installs include them inline above. +CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id + ON oauth_clients(source_id) WHERE source_id IS NOT NULL; +CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read + ON oauth_clients USING GIN (federated_read); CREATE TABLE IF NOT EXISTS oauth_tokens ( token_hash TEXT PRIMARY KEY, diff --git a/src/core/search/hybrid.ts b/src/core/search/hybrid.ts index 8195ab6..e21836e 100644 --- a/src/core/search/hybrid.ts +++ b/src/core/search/hybrid.ts @@ -280,6 +280,16 @@ export async function hybridSearch( // PR #618 callers compiling while the new names are the public surface. afterDate: opts?.since ?? opts?.afterDate, beforeDate: opts?.until ?? opts?.beforeDate, + // v0.34.1 (#861, D9 — P0 leak seal): thread source-scoping through so the + // inner engine.searchKeyword / engine.searchVector calls apply the + // WHERE source_id filter at SQL level. Pre-fix, this explicit pick + // silently DROPPED these fields and every authenticated MCP client + // could see pages from foreign sources via the hybrid search hot + // path. New SearchOpts fields scoped to source isolation MUST be + // added here too; the rebuild shape is intentional (HNSW inner-CTE + // ordering means we can't lazy-spread the full opts). + sourceId: opts?.sourceId, + sourceIds: opts?.sourceIds, }; // Track what actually ran for the optional onMeta callback (v0.25.0). // Caller leaves onMeta undefined → these flags are computed but never diff --git a/src/core/types.ts b/src/core/types.ts index 0d15f12..468ef13 100644 --- a/src/core/types.ts +++ b/src/core/types.ts @@ -196,6 +196,15 @@ export interface PageFilters { * operations to a single source. */ sourceId?: string; + /** + * v0.34.1 (#876, D9): filter to ANY of these sources (federated read). + * Engine applies `WHERE p.source_id = ANY($N::text[])` when array is set. + * Caller precedence: if BOTH `sourceId` and `sourceIds` are set, the array + * wins (the federated semantics subsume the single-source case via an + * array of length 1). When neither is set, no filter applies — the + * pre-v0.34 unscoped behavior is preserved for local CLI callers. + */ + sourceIds?: string[]; } /** v0.26.5 — opts for getPage / softDeletePage / restorePage. */ @@ -457,6 +466,18 @@ export interface SearchOpts { * undefined to search all sources. */ sourceId?: string; + /** + * v0.34.1 (#876, D9): filter to ANY of these sources (federated read). + * Engine applies `WHERE p.source_id = ANY($N::text[])` when the array + * is set. Caller precedence: if BOTH `sourceId` and `sourceIds` are + * provided, the array wins (the federated semantics subsume the + * single-source case). When neither is set, no filter applies — the + * pre-v0.34 unscoped behavior is preserved for local CLI callers. + * + * The op-handler layer resolves: `ctx.auth?.allowedSources` (federated + * client) → `sourceIds`; otherwise `ctx.sourceId` (scalar) → `sourceId`. + */ + sourceIds?: string[]; /** * v0.27.1: target column for vector search. 'embedding' (default) hits * the brain's primary text-embedding column. 'embedding_image' targets diff --git a/src/mcp/http-transport.ts b/src/mcp/http-transport.ts index 6356208..df5825a 100644 --- a/src/mcp/http-transport.ts +++ b/src/mcp/http-transport.ts @@ -66,6 +66,14 @@ interface AuthResult { tokenName?: string; /** v0.28: per-token allow-list for takes.holder. Default ['world'] when permissions row absent. */ takesHoldersAllowList?: string[]; + /** + * v0.34.1 (#861, D13): source-isolation scope for the auth'd request. + * Legacy bearer tokens here default to 'default' to match the v0.33 + * effective behavior (the now-removed serve-http.ts fallback chain). + * Operators migrate to the full OAuth transport (gbrain serve --http) + * for narrower scoping. + */ + sourceId?: string; } /** Read up to `cap` bytes off req.body. Returns null if cap exceeded. */ @@ -180,6 +188,12 @@ export async function startHttpTransport(opts: HttpTransportOptions) { tokenId: rowId, tokenName: rowName, takesHoldersAllowList: allowList, + // v0.34.1 (#861, D13): legacy bearer tokens default to 'default' + // source. Preserves the pre-v0.34 effective behavior of the + // serve-http fallback chain that was removed for OAuth clients + // (migration v60 backfills oauth_clients.source_id). This path + // is for the older v0.22.7 access_tokens transport. + sourceId: 'default', }; } catch { return { ok: false }; @@ -328,9 +342,12 @@ export async function startHttpTransport(opts: HttpTransportOptions) { const args: Record = params?.arguments ?? {}; // v0.28: thread per-token takes-holder allow-list so takes_list / // takes_search / query (when it returns takes) can server-side filter. + // v0.34.1 (#861): thread source-isolation scope. Legacy access_tokens + // path defaults to 'default' per AuthResult.sourceId above. const result = await dispatchToolCall(engine, toolName, args, { remote: true, takesHoldersAllowList: auth.takesHoldersAllowList, + sourceId: auth.sourceId, }); const status = result.isError ? 'error' : 'success'; logRequest(auth.tokenName!, `tools/call:${toolName}`, status, Date.now() - startedMs); diff --git a/src/mcp/server.ts b/src/mcp/server.ts index e268cbb..94da322 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -62,8 +62,15 @@ export async function startMcpServer(engine: BrainEngine) { .catch(() => {}) .finally(() => process.exit(code)); }; - process.stdin.on('end', () => shutdown('stdin end')); - process.stdin.on('close', () => shutdown('stdin close')); + // v0.34.1 (#870): when MCP_STDIO=1, the wrapping gateway (OpenClaw's + // bundle-mcp layer, others) often pipes the JSON-RPC handshake then + // closes its stdin half. Treating that as a permanent disconnect kills + // the server before the first tool call arrives. Signal handlers and + // transport.onclose still cover the legitimate shutdown paths. + if (process.env.MCP_STDIO !== '1') { + process.stdin.on('end', () => shutdown('stdin end')); + process.stdin.on('close', () => shutdown('stdin close')); + } // @ts-ignore — SDK exposes onclose on transport transport.onclose = () => shutdown('transport close'); process.on('SIGTERM', () => shutdown('SIGTERM')); diff --git a/src/schema.sql b/src/schema.sql index bf9fa84..c19221d 100644 --- a/src/schema.sql +++ b/src/schema.sql @@ -424,8 +424,17 @@ CREATE TABLE IF NOT EXISTS oauth_clients ( client_secret_expires_at BIGINT, token_ttl INTEGER, deleted_at TIMESTAMPTZ, + source_id TEXT REFERENCES sources(id) ON DELETE RESTRICT, + federated_read TEXT[] NOT NULL DEFAULT '{}', created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); +-- v0.34.1 (#861, D13 + #876): source_id is the write-source scope; +-- federated_read is the read-source array. Migrations v60-v65 land both +-- columns on upgrade; fresh installs include them inline above. +CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id + ON oauth_clients(source_id) WHERE source_id IS NOT NULL; +CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read + ON oauth_clients USING GIN (federated_read); CREATE TABLE IF NOT EXISTS oauth_tokens ( token_hash TEXT PRIMARY KEY, diff --git a/test/book-mirror.test.ts b/test/book-mirror.test.ts index 25905c3..0ee6599 100644 --- a/test/book-mirror.test.ts +++ b/test/book-mirror.test.ts @@ -32,35 +32,57 @@ async function runCli(args: string[]): Promise<{ stderr: string; exit: number; }> { - const proc = Bun.spawn( - ['bun', 'run', 'src/cli.ts', 'book-mirror', ...args], - { - cwd: REPO_ROOT, - stdout: 'pipe', - stderr: 'pipe', - env: { ...process.env, DATABASE_URL: '' }, - }, + // v0.34.1 (#876): set GBRAIN_HOME to a fresh tempdir so the test isn't + // sensitive to the developer's local ~/.gbrain/config.json. Pre-fix this + // test would silently inherit the user's database_url and try to connect + // to a real Postgres + run migrations, which both extends runtime past + // the default test timeout and produces different exit semantics + // depending on the local DB state. With GBRAIN_HOME pointing at an + // empty dir, the CLI hits "No brain configured" and exits 1 immediately + // — which is what this test was originally written to cover. + const tmpHome = await import('os').then(os => + import('fs').then(fs => fs.mkdtempSync(`${os.tmpdir()}/gbrain-test-`)), ); - const stdout = await new Response(proc.stdout).text(); - const stderr = await new Response(proc.stderr).text(); - const exit = await proc.exited; - return { stdout, stderr, exit }; + try { + const proc = Bun.spawn( + ['bun', 'run', 'src/cli.ts', 'book-mirror', ...args], + { + cwd: REPO_ROOT, + stdout: 'pipe', + stderr: 'pipe', + env: { ...process.env, DATABASE_URL: '', GBRAIN_HOME: tmpHome }, + }, + ); + const stdout = await new Response(proc.stdout).text(); + const stderr = await new Response(proc.stderr).text(); + const exit = await proc.exited; + return { stdout, stderr, exit }; + } finally { + const fs = await import('fs'); + fs.rmSync(tmpHome, { recursive: true, force: true }); + } } describe('gbrain book-mirror — CLI registration', () => { + // v0.34.1 (#876): migrations v60-v65 added oauth_clients.source_id + + // federated_read FK plumbing. The cold-spawned subprocess's initSchema + // chain now takes ~1s longer end-to-end; bump these tests' timeout to + // 30s so the test-harness budget covers Bun cold-start + PGLite init + + // migration apply + Postgres-retry exhaustion. The CLI itself still + // exits in 7-10s on a misconfigured DB. it('book-mirror is in CLI_ONLY (does not get "Unknown command")', async () => { const { stderr } = await runCli([]); // Without DB, the command will fail — but on the connect path, // not as "Unknown command". This proves dispatch reached // handleCliOnly's switch statement. expect(stderr).not.toContain('Unknown command'); - }); + }, 30000); it('without DB, never reaches queue submission', async () => { const { stderr, exit } = await runCli(['--slug', 'noop']); expect(exit).not.toBe(0); expect(stderr).not.toContain('submitted:'); - }); + }, 30000); }); describe('gbrain book-mirror — source file invariants', () => { diff --git a/test/e2e/source-isolation-pglite.test.ts b/test/e2e/source-isolation-pglite.test.ts new file mode 100644 index 0000000..71567d8 --- /dev/null +++ b/test/e2e/source-isolation-pglite.test.ts @@ -0,0 +1,267 @@ +/** + * v0.34.1 (#861 — P0 source-isolation leak seal) E2E regression. + * + * The wave's IRON RULE: every read-side op must filter by source_id when + * the caller supplies it via SearchOpts/PageFilters/TraverseOpts. Pre-fix, + * authenticated MCP clients scoped to source-A could enumerate source-B + * pages via search / query / list_pages / traverse_graph / find_experts. + * + * Runs against PGLite in-memory so the test exercises both engines' parity + * surface (the schema-drift E2E covers Postgres independently) without + * needing a Docker container. + * + * Coverage: searchKeyword, searchVector (synthetic embedding), listPages, + * traverseGraph, traversePaths. find_experts is exercised via the same + * hybridSearch path the op handler calls. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../../src/core/pglite-engine.ts'; +import { resetPgliteState } from '../helpers/reset-pglite.ts'; + +let engine: PGLiteEngine; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await resetPgliteState(engine); + // Two sources so we can prove the filter excludes cross-source rows. + // 'default' source is created by initSchema's seed; we add src-b. + await engine.executeRaw( + `INSERT INTO sources (id, name, config) VALUES ('src-b', 'src-b', '{}'::jsonb) ON CONFLICT DO NOTHING`, + ); + // Seed one person page in each source. Same slug intentionally — + // proves the composite (source_id, slug) key is honored, not just slug. + // upsertChunks is needed because searchKeyword scans content_chunks, not + // pages.compiled_truth directly. Each page gets one chunk that mirrors + // its compiled_truth so search-by-keyword has something to find. + await engine.putPage('people/alice', { + type: 'person', + title: 'Alice Source-A', + compiled_truth: 'Alice works on widgets in source A. Important context here.', + timeline: '', + frontmatter: {}, + }, { sourceId: 'default' }); + await engine.upsertChunks('people/alice', [{ + chunk_index: 0, + chunk_text: 'Alice works on widgets in source A. Important context here.', + chunk_source: 'compiled_truth', + token_count: 12, + }], { sourceId: 'default' }); + + await engine.putPage('people/alice', { + type: 'person', + title: 'Alice Source-B', + compiled_truth: 'Alice works on gadgets in source B. Important context here.', + timeline: '', + frontmatter: {}, + }, { sourceId: 'src-b' }); + await engine.upsertChunks('people/alice', [{ + chunk_index: 0, + chunk_text: 'Alice works on gadgets in source B. Important context here.', + chunk_source: 'compiled_truth', + token_count: 12, + }], { sourceId: 'src-b' }); + + await engine.putPage('people/bob', { + type: 'person', + title: 'Bob Source-B Only', + compiled_truth: 'Bob lives only in source B. Important context here.', + timeline: '', + frontmatter: {}, + }, { sourceId: 'src-b' }); + await engine.upsertChunks('people/bob', [{ + chunk_index: 0, + chunk_text: 'Bob lives only in source B. Important context here.', + chunk_source: 'compiled_truth', + token_count: 11, + }], { sourceId: 'src-b' }); +}); + +describe('v0.34.1 source-isolation regression (#861)', () => { + test('searchKeyword with sourceId=default excludes src-b rows', async () => { + const results = await engine.searchKeyword('widgets', { sourceId: 'default' }); + // Should find Alice from source-A only (mentions "widgets"). src-b's + // Alice mentions "gadgets" not "widgets" but the test guards against + // accidentally pulling in src-b rows via a missing WHERE clause. + expect(results.length).toBeGreaterThan(0); + for (const r of results) { + expect(r.source_id).toBe('default'); + } + }); + + test('searchKeyword with sourceId=src-b excludes default rows', async () => { + const results = await engine.searchKeyword('gadgets', { sourceId: 'src-b' }); + expect(results.length).toBeGreaterThan(0); + for (const r of results) { + expect(r.source_id).toBe('src-b'); + } + }); + + test('searchKeyword with sourceIds=[default,src-b] returns both', async () => { + // Federated read (D9 array form) returns the union. + const results = await engine.searchKeyword('Important context', { + sourceIds: ['default', 'src-b'], + }); + const sources = new Set(results.map(r => r.source_id)); + expect(sources.has('default')).toBe(true); + expect(sources.has('src-b')).toBe(true); + }); + + test('searchKeyword with sourceIds=[default] only returns default', async () => { + // Array form with a single element behaves like scalar. + const results = await engine.searchKeyword('Important context', { + sourceIds: ['default'], + }); + for (const r of results) { + expect(r.source_id).toBe('default'); + } + }); + + test('searchKeyword with no source scope returns all sources', async () => { + // Local CLI / unscoped callers preserve pre-v0.34 behavior. + const results = await engine.searchKeyword('Important context'); + const sources = new Set(results.map(r => r.source_id)); + expect(sources.size).toBeGreaterThanOrEqual(2); + }); + + test('listPages with sourceId=default hides src-b rows', async () => { + const pages = await engine.listPages({ sourceId: 'default', limit: 100 }); + expect(pages.length).toBeGreaterThan(0); + expect(pages.find(p => p.title === 'Bob Source-B Only')).toBeUndefined(); + expect(pages.find(p => p.title === 'Alice Source-B')).toBeUndefined(); + }); + + test('listPages with sourceIds=[default,src-b] returns the union', async () => { + const pages = await engine.listPages({ sourceIds: ['default', 'src-b'], limit: 100 }); + const titles = new Set(pages.map(p => p.title)); + expect(titles.has('Alice Source-A')).toBe(true); + expect(titles.has('Alice Source-B')).toBe(true); + expect(titles.has('Bob Source-B Only')).toBe(true); + }); + + test('traverseGraph with sourceId=default does not surface src-b roots', async () => { + // Seeding the walk at src-b's bob with sourceId=default produces an + // empty result — the seed itself is filtered out, so the walk never + // discovers anything. Pre-fix, the walk would still return bob's + // neighbors via cross-source edge following. + const nodes = await engine.traverseGraph('people/bob', 5, { sourceId: 'default' }); + expect(nodes.length).toBe(0); + }); + + test('traverseGraph with sourceId=src-b finds the src-b page', async () => { + const nodes = await engine.traverseGraph('people/bob', 5, { sourceId: 'src-b' }); + expect(nodes.length).toBeGreaterThanOrEqual(1); + expect(nodes.find(n => n.slug === 'people/bob')).toBeDefined(); + }); + + test('traversePaths with sourceId=default does not surface src-b paths', async () => { + // Same seed-filter check via the edge-based traversal. + const paths = await engine.traversePaths('people/bob', { depth: 3, sourceId: 'default' }); + expect(paths.length).toBe(0); + }); + + test('searchVector with sourceId filters HNSW candidate pool', async () => { + // No real embeddings on the test pages; the WHERE cc.embedding IS NOT NULL + // gate filters them out. We assert the contract via an empty result + // rather than a positive match: with sourceId set, the SQL still runs + // (no type or undefined-column errors). + const synth = new Float32Array(1536).fill(0.01); + const results = await engine.searchVector(synth, { sourceId: 'src-b' }); + // Either empty (no embeddings) or all from src-b. Both prove the + // filter is wired without a runtime error. + for (const r of results) { + expect(r.source_id).toBe('src-b'); + } + }); + + test('AuthInfo path: ctx.sourceId scoping propagates through op handlers', async () => { + // The op handler at operations.ts:search threads ctx.sourceId via + // sourceScopeOpts. Simulate the dispatcher's threading and verify + // the engine sees the filter — this is the layer the regression + // tests need to cover most directly. + const { operations } = await import('../../src/core/operations.ts'); + const searchOp = operations.find(o => o.name === 'search'); + expect(searchOp).toBeDefined(); + + const ctx = { + engine, + config: { engine: 'pglite' as const }, + logger: { info: () => {}, warn: () => {}, error: () => {} }, + dryRun: false, + remote: true, + sourceId: 'src-b', + }; + const result = await searchOp!.handler(ctx as any, { query: 'gadgets' }); + const rows = result as Array<{ source_id?: string }>; + expect(rows.length).toBeGreaterThan(0); + for (const r of rows) { + expect(r.source_id).toBe('src-b'); + } + }); + + test('AuthInfo.allowedSources path: ctx.auth.allowedSources widens read scope', async () => { + // D9 federated read: when AuthInfo.allowedSources is populated, the + // sourceScopeOpts helper produces sourceIds array (array wins over + // scalar ctx.sourceId). Verify the op handler routes through this. + const { operations } = await import('../../src/core/operations.ts'); + const searchOp = operations.find(o => o.name === 'search'); + const ctx = { + engine, + config: { engine: 'pglite' as const }, + logger: { info: () => {}, warn: () => {}, error: () => {} }, + dryRun: false, + remote: true, + sourceId: 'default', // scalar would scope to default-only + auth: { + token: 'test', + clientId: 'test', + scopes: ['read'], + sourceId: 'default', + allowedSources: ['default', 'src-b'], // ... but the array wins + }, + }; + const result = await searchOp!.handler(ctx as any, { query: 'Important context' }); + const rows = result as Array<{ source_id?: string }>; + const sources = new Set(rows.map(r => r.source_id)); + expect(sources.has('default')).toBe(true); + expect(sources.has('src-b')).toBe(true); + }); + + test('#876 federated_read empty array means no federated reads', async () => { + // sourceScopeOpts treats allowedSources: [] (explicit empty) as "no + // federated scope" and falls back to scalar sourceId. An empty array + // MUST NOT widen scope to "all sources" — that's the silent-widen + // footgun. Verify the precedence ladder is correct. + const { operations } = await import('../../src/core/operations.ts'); + const searchOp = operations.find(o => o.name === 'search'); + const ctx = { + engine, + config: { engine: 'pglite' as const }, + logger: { info: () => {}, warn: () => {}, error: () => {} }, + dryRun: false, + remote: true, + sourceId: 'default', + auth: { + token: 'test', + clientId: 'test', + scopes: ['read'], + sourceId: 'default', + allowedSources: [], // explicit empty — must NOT widen scope + }, + }; + const result = await searchOp!.handler(ctx as any, { query: 'Important context' }); + const rows = result as Array<{ source_id?: string }>; + for (const r of rows) { + expect(r.source_id).toBe('default'); + } + }); +}); diff --git a/test/oauth.test.ts b/test/oauth.test.ts index dd7829f..fa549a5 100644 --- a/test/oauth.test.ts +++ b/test/oauth.test.ts @@ -1148,3 +1148,106 @@ describe('F12 dcrDisabled constructor option', () => { expect(result.clientSecret).toStartWith('gbrain_cs_'); }); }); + +// --------------------------------------------------------------------------- +// v0.34.1 (#909) — PKCE public-client DCR (RFC 7591 §3.2.1) +// --------------------------------------------------------------------------- +// +// Per RFC 7591 §3.2.1, when a DCR client declares +// `token_endpoint_auth_method: "none"` (PKCE-only public clients like Claude +// Code, Cursor), the authorization server MUST NOT issue a client_secret. +// Pre-fix, unconditional secret generation made the MCP SDK's clientAuth +// middleware reject valid public-client flows on /token. + +describe('PKCE DCR public-client gate (#909)', () => { + test("registerClient with token_endpoint_auth_method='none' omits client_secret", async () => { + const result = await provider.clientsStore.registerClient!({ + client_name: 'public-pkce-client', + redirect_uris: ['https://example.com/callback'], + grant_types: ['authorization_code'], + scope: 'read', + token_endpoint_auth_method: 'none', + }); + expect(result.client_id).toStartWith('gbrain_cl_'); + // RFC 7591 §3.2.1: public clients get NO client_secret in the response. + expect(result.client_secret).toBeUndefined(); + expect(result.token_endpoint_auth_method).toBe('none'); + }); + + test('default auth_method (omitted) still issues a client_secret', async () => { + // Regression guard: confidential clients (the existing default) must + // keep their secret-issuing behavior unchanged. + const result = await provider.clientsStore.registerClient!({ + client_name: 'confidential-default', + redirect_uris: ['https://example.com/callback'], + grant_types: ['authorization_code'], + scope: 'read', + // token_endpoint_auth_method omitted; falls back to 'client_secret_post' + }); + expect(result.client_id).toStartWith('gbrain_cl_'); + expect(result.client_secret).toStartWith('gbrain_cs_'); + }); + + test('explicit client_secret_post still issues a client_secret', async () => { + const result = await provider.clientsStore.registerClient!({ + client_name: 'confidential-explicit', + redirect_uris: ['https://example.com/callback'], + grant_types: ['authorization_code'], + scope: 'read', + token_endpoint_auth_method: 'client_secret_post', + }); + expect(result.client_id).toStartWith('gbrain_cl_'); + expect(result.client_secret).toStartWith('gbrain_cs_'); + }); + + test('getClient on a public client returns client_secret=undefined (NULL normalized)', async () => { + // The SDK's clientAuth middleware checks `client.client_secret === undefined` + // (not `=== null`) to decide whether to enforce secret comparison on /token. + // Without normalization, Postgres NULL would reach the SDK as JS null and + // the secret check would mis-fire on every public client. + const reg = await provider.clientsStore.registerClient!({ + client_name: 'public-getclient-norm', + redirect_uris: ['https://example.com/callback'], + grant_types: ['authorization_code'], + scope: 'read', + token_endpoint_auth_method: 'none', + }); + const stored = await provider.clientsStore.getClient(reg.client_id); + expect(stored).toBeDefined(); + expect(stored!.client_secret).toBeUndefined(); + expect(stored!.token_endpoint_auth_method).toBe('none'); + }); + + test('PKCE flow end-to-end: public client /authorize then /token, no secret needed', async () => { + // Full F7 regression #15: public client completes auth_code → token + // exchange without ever presenting a client_secret. + const reg = await provider.clientsStore.registerClient!({ + client_name: 'pkce-roundtrip', + redirect_uris: ['http://localhost:3000/callback'], + grant_types: ['authorization_code'], + scope: 'read', + token_endpoint_auth_method: 'none', + }); + + // Re-fetch via getClient to mirror what the SDK middleware sees. + const client = (await provider.clientsStore.getClient(reg.client_id))!; + expect(client.client_secret).toBeUndefined(); + + let redirectUrl = ''; + const mockRes = { redirect: (url: string) => { redirectUrl = url; } } as any; + await provider.authorize(client, { + codeChallenge: 'test-challenge-value', + redirectUri: 'http://localhost:3000/callback', + scopes: ['read'], + }, mockRes); + const code = new URL(redirectUrl).searchParams.get('code')!; + expect(code).toMatch(/^gbrain_code_/); + + // Exchange the code — public client; no secret on the wire. + const tokens = await provider.exchangeAuthorizationCode(client, code); + expect(tokens.access_token).toStartWith('gbrain_at_'); + // SDK normalizes token_type per RFC 6750 §6.1.1 (case-insensitive); + // implementations may emit "bearer" lowercase. + expect(String(tokens.token_type).toLowerCase()).toBe('bearer'); + }); +}); diff --git a/test/openai-compat-multimodal.test.ts b/test/openai-compat-multimodal.test.ts new file mode 100644 index 0000000..0aecdfd --- /dev/null +++ b/test/openai-compat-multimodal.test.ts @@ -0,0 +1,248 @@ +// v0.34.1 (#875): multimodal embedding for openai-compatible recipes via +// LiteLLM (or any other openai-compatible proxy). Sibling to +// voyage-multimodal.test.ts; covers the new embedMultimodalOpenAICompat +// path including D12 dim validation. + +import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; +import { configureGateway, embedMultimodal, resetGateway } from '../src/core/ai/gateway.ts'; +import { AIConfigError, AITransientError } from '../src/core/ai/errors.ts'; + +type FetchHandler = (url: string, init: RequestInit) => Promise; +let fetchHandler: FetchHandler | null = null; +const origFetch = globalThis.fetch; + +beforeEach(() => { + fetchHandler = null; + globalThis.fetch = (async (url: string | URL | Request, init?: RequestInit) => { + if (!fetchHandler) { + throw new Error('fetch called but no handler installed'); + } + return fetchHandler(typeof url === 'string' ? url : url.toString(), init ?? {}); + }) as typeof fetch; +}); + +afterEach(() => { + globalThis.fetch = origFetch; + resetGateway(); +}); + +function configureLitellm(env: Record = {}, dims = 1024) { + configureGateway({ + embedding_model: 'litellm:gpt-4o-multimodal', + embedding_dimensions: dims, + env: { + LITELLM_API_KEY: 'test-litellm-key', + LITELLM_BASE_URL: 'http://localhost:4000', + ...env, + }, + base_urls: { litellm: 'http://localhost:4000' }, + }); +} + +function okResponse(dims: number, count: number = 1): Response { + const vec = Array(dims).fill(0).map((_, i) => 0.001 * i); + return new Response( + JSON.stringify({ data: Array.from({ length: count }, () => ({ embedding: vec })) }), + { status: 200, headers: { 'Content-Type': 'application/json' } }, + ); +} + +describe('embedMultimodal — openai-compat routing (#875)', () => { + test('LiteLLM recipe accepts a single image input and returns one embedding', async () => { + configureLitellm(); + let capturedUrl = ''; + let capturedBody: any = null; + let capturedAuth = ''; + fetchHandler = async (url, init) => { + capturedUrl = url; + capturedAuth = (init.headers as Record).Authorization ?? ''; + capturedBody = JSON.parse(init.body as string); + return okResponse(1024, 1); + }; + + const result = await embedMultimodal([ + { kind: 'image_base64', data: 'fake-base64-bytes', mime: 'image/png' }, + ]); + + expect(result.length).toBe(1); + expect(result[0].length).toBe(1024); + expect(capturedUrl).toBe('http://localhost:4000/embeddings'); + expect(capturedAuth).toBe('Bearer test-litellm-key'); + expect(capturedBody.model).toBe('gpt-4o-multimodal'); + expect(capturedBody.input[0].type).toBe('image_url'); + expect(capturedBody.input[0].image_url.url).toBe('data:image/png;base64,fake-base64-bytes'); + }); + + test('multiple inputs trigger sequential /embeddings calls', async () => { + configureLitellm(); + let calls = 0; + fetchHandler = async () => { + calls += 1; + return okResponse(1024, 1); + }; + + const result = await embedMultimodal([ + { kind: 'image_base64', data: 'img1', mime: 'image/jpeg' }, + { kind: 'image_base64', data: 'img2', mime: 'image/png' }, + { kind: 'image_base64', data: 'img3', mime: 'image/webp' }, + ]); + + expect(calls).toBe(3); + expect(result.length).toBe(3); + }); + + test('LiteLLM without LITELLM_API_KEY still works (proxy may run unauthenticated)', async () => { + configureGateway({ + embedding_model: 'litellm:multimodal-foo', + embedding_dimensions: 768, + env: { LITELLM_BASE_URL: 'http://localhost:4000' }, // no API key + base_urls: { litellm: 'http://localhost:4000' }, + }); + let capturedAuth: string | null | undefined; + fetchHandler = async (_url, init) => { + capturedAuth = (init.headers as Record).Authorization; + return okResponse(768, 1); + }; + const result = await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + expect(result.length).toBe(1); + // defaultResolveAuth sends 'Bearer unauthenticated' when no api key is + // configured — servers like Ollama / llama-server ignore the value but + // the SDK contract still requires SOME Authorization header. + expect(capturedAuth).toBe('Bearer unauthenticated'); + }); + + test('D12 — provider returns wrong-dim vector throws AIConfigError', async () => { + // Brain configured for 1024; provider returns 768. D12 catches the + // mismatch BEFORE the vector lands in the DB column. + configureLitellm({}, 1024); + fetchHandler = async () => okResponse(768, 1); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AIConfigError); + expect((caught as Error).message).toContain('768-dim vector'); + expect((caught as Error).message).toContain('expected 1024'); + expect((caught as Error).message).toContain('gpt-4o-multimodal'); + }); + + test('D12 — default embedding_dimensions (1536) applies when not explicitly set', async () => { + // configureGateway normalizes embedding_dimensions to 1536 when unset + // (the DEFAULT_EMBEDDING_DIMENSIONS). LiteLLM recipe's default_dims=0 + // so we fall back to the brain's configured value. This test pins the + // "always validate via the configured/default dim" contract — there + // is no skip-when-unset path in practice because configureGateway + // always populates it. + configureGateway({ + embedding_model: 'litellm:any-model', + // intentionally NO embedding_dimensions → falls back to 1536 + env: { LITELLM_BASE_URL: 'http://localhost:4000' }, + base_urls: { litellm: 'http://localhost:4000' }, + }); + fetchHandler = async () => okResponse(1536, 1); + const result = await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + expect(result.length).toBe(1); + expect(result[0].length).toBe(1536); + }); + + test('provider returns 401 → AIConfigError with model id in message', async () => { + configureLitellm(); + fetchHandler = async () => + new Response('invalid key', { status: 401, headers: { 'Content-Type': 'text/plain' } }); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AIConfigError); + expect((caught as Error).message).toContain('401'); + }); + + test('provider returns 400 (model does not support multimodal) → AITransientError surfaces body', async () => { + configureLitellm(); + fetchHandler = async () => + new Response('model does not support image inputs', { status: 400 }); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AITransientError); + expect((caught as Error).message).toContain('400'); + expect((caught as Error).message).toContain('model does not support image inputs'); + }); + + test('malformed JSON response → AITransientError', async () => { + configureLitellm(); + fetchHandler = async () => + new Response('not json', { status: 200, headers: { 'Content-Type': 'application/json' } }); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AITransientError); + expect((caught as Error).message).toContain('malformed JSON'); + }); + + test('non-array embedding payload → AITransientError', async () => { + configureLitellm(); + fetchHandler = async () => + new Response(JSON.stringify({ data: [{ embedding: 'not-array' }] }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AITransientError); + expect((caught as Error).message).toContain('non-array'); + }); + + test('empty data array → AITransientError', async () => { + configureLitellm(); + fetchHandler = async () => + new Response(JSON.stringify({ data: [] }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }); + + let caught: unknown; + try { + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(AITransientError); + }); + + test('Voyage recipe still routes to /multimodalembeddings (regression)', async () => { + // Ensure the new openai-compat route doesn't accidentally hijack Voyage. + configureGateway({ + embedding_model: 'voyage:voyage-multimodal-3', + embedding_dimensions: 1024, + env: { VOYAGE_API_KEY: 'voyage-key' }, + }); + let capturedUrl = ''; + fetchHandler = async (url) => { + capturedUrl = url; + return okResponse(1024, 1); + }; + + await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]); + expect(capturedUrl).toContain('/multimodalembeddings'); + }); +}); diff --git a/test/serve-stdio-lifecycle.test.ts b/test/serve-stdio-lifecycle.test.ts index e548b01..e250ad2 100644 --- a/test/serve-stdio-lifecycle.test.ts +++ b/test/serve-stdio-lifecycle.test.ts @@ -81,6 +81,7 @@ function makeHarness(opts: { isTTY?: boolean; initialParentPid?: number; probeWatchdog?: boolean; + mcpStdio?: boolean; } = {}): Harness { const engine = new StubEngine(); const stdin = new EventEmitter() as EventEmitter & { isTTY?: boolean }; @@ -120,6 +121,7 @@ function makeHarness(opts: { setInterval: timers.setInterval, clearInterval: timers.clearInterval, probeWatchdog: () => probeWatchdogResult, + mcpStdio: opts.mcpStdio, }; return { @@ -439,4 +441,63 @@ describe('runServe stdio lifecycle', () => { expect(code).toBe(0); expect(h.logs.some(l => l.includes('cleanup error: synthetic disconnect failure'))).toBe(true); }); + + // v0.34.1 (#870): OpenClaw gateway / bundle-mcp wrappers pipe the + // JSON-RPC handshake on stdin then close their stdin half. Without + // MCP_STDIO=1 the server treats that as a permanent disconnect and + // exits before handling tools/call. The guard skips the stdin 'end' / + // 'close' hooks when MCP_STDIO=1; signals and parent watchdog still + // cover legitimate shutdown. + describe('MCP_STDIO=1 piped-stdin guard (#870)', () => { + test('stdin end with mcpStdio=true does NOT trigger shutdown', async () => { + const h = makeHarness({ mcpStdio: true }); + await startInBackground(h.engine, [], h.opts); + + // Without the guard this would shutdown; with the guard it must not. + h.stdin.emit('end'); + + // Give the event loop a microtask turn to catch any erroneous shutdown + // path. We assert NO exit was registered. + await new Promise((r) => setTimeout(r, 10)); + expect(h.engine.disconnectCalls).toBe(0); + + // Then trigger SIGTERM to drive the test to completion; signal handlers + // remain active even with mcpStdio=true (codex would catch if they didn't). + h.signals.emit('SIGTERM'); + const code = await h.exited; + expect(code).toBe(0); + expect(h.engine.disconnectCalls).toBe(1); + expect(h.logs.some(l => l.includes('graceful exit (SIGTERM)'))).toBe(true); + }); + + test('stdin close with mcpStdio=true does NOT trigger shutdown', async () => { + const h = makeHarness({ mcpStdio: true }); + await startInBackground(h.engine, [], h.opts); + + h.stdin.emit('close'); + + await new Promise((r) => setTimeout(r, 10)); + expect(h.engine.disconnectCalls).toBe(0); + + h.signals.emit('SIGINT'); + const code = await h.exited; + expect(code).toBe(0); + expect(h.engine.disconnectCalls).toBe(1); + }); + + test('mcpStdio=false (default) preserves stdin EOF shutdown', async () => { + // Regression guard: the guard must not over-trigger. With the env + // unset, stdin EOF must still drive shutdown so existing CLI usage + // (gbrain serve under launchd, claude-desktop's stdio MCP) is + // unchanged. + const h = makeHarness({ mcpStdio: false }); + await startInBackground(h.engine, [], h.opts); + + h.stdin.emit('end'); + const code = await h.exited; + expect(code).toBe(0); + expect(h.engine.disconnectCalls).toBe(1); + expect(h.logs.some(l => l.includes('graceful exit (stdin-end)'))).toBe(true); + }); + }); });