v0.34.1.0 fix(mcp): MCP fix wave — source-isolation P0 + PKCE DCR + federated_read + 3 more (#996)
* fix(mcp): skip stdin EOF handlers when MCP_STDIO=1 OpenClaw's bundle-mcp gateway and similar wrappers pipe the JSON-RPC handshake on stdin then close their stdin half. Pre-fix, both stdin 'end' and 'close' listeners (server.ts:65-66 and serve.ts:204-206) treated this as a permanent disconnect and shut the server down before the first tool call arrived. Guard both sites with `process.env.MCP_STDIO !== '1'`. Signal handlers (SIGTERM/SIGINT/SIGHUP), transport.onclose, and the parent-process watchdog still cover legitimate shutdown paths. The serve.ts site threads the env read through an injectable `mcpStdio?: boolean` on ServeOptions so tests stay isolated (no process.env mutation per scripts/check-test-isolation.sh R1). Tests: 3 new cases in test/serve-stdio-lifecycle.test.ts pin the guard's invariants — mcpStdio=true must NOT trigger shutdown on stdin EOF, signals must still drive shutdown with mcpStdio=true, and mcpStdio=false (default) preserves existing CLI behavior. 25/25 pass. Origin: PR #870. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(oauth): honor token_endpoint_auth_method=none for PKCE public clients RFC 7591 §3.2.1: when a DCR client declares token_endpoint_auth_method="none" (PKCE-only public clients like Claude Code, Cursor), the authorization server MUST NOT issue a client_secret. Pre-fix, registerClient unconditionally minted a secret, and the MCP SDK's clientAuth middleware then rejected valid public-client flows on /token because it expected client.client_secret to match. Three changes to src/core/oauth-provider.ts:registerClient: - Gate clientSecret generation on isPublicClient = (auth_method === 'none'). Public clients store client_secret_hash = NULL. - Omit client_secret from the response payload for public clients. Confidential clients (default client_secret_post and explicit client_secret_basic) keep their existing one-time-reveal shape. - Normalize NULL secret_hash to JS undefined in getClient so SDK middleware (which checks client.client_secret === undefined, not === null) correctly identifies public clients and skips the secret-comparison branch on /token. Schema is already permissive (client_secret_hash TEXT, no NOT NULL on both src/schema.sql and src/core/pglite-schema.ts) — no migration needed. Tests: 5 new cases in test/oauth.test.ts pin: - public client → no client_secret in response (#11 from plan) - default auth_method → secret unchanged (regression guard) - explicit client_secret_post → secret unchanged - getClient NULL→undefined normalization - PKCE full /authorize → /token end-to-end with no secret (#15 from plan) 69/69 oauth.test.ts cases pass. typecheck clean. Origin: PR #909. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(serve-http): --bind HOST, default to loopback (127.0.0.1) Adds `gbrain serve --http --bind <interface>` to control which network interface the HTTP MCP server listens on. Default flipped from `0.0.0.0` (pre-v0.34) to `127.0.0.1` (v0.34.0+). Why the flip: gbrain's primary use case is a personal-knowledge brain on a laptop. The previous default exposed brains on every interface — one accidental `--http` invocation away from publishing the brain to a LAN. Server operators who need remote access pass `--bind 0.0.0.0` (or a specific interface). Codex's outside-voice on the original PR #864 correctly flagged that the additive flag wasn't actually the fix; the default needed to change for the safety claim to hold. If `--public-url` is set but `--bind` is unset, runServeHttp prints a loud stderr WARN at startup recommending `--bind 0.0.0.0`. Declaring a public URL while quietly binding loopback is almost always a misconfiguration; we want the operator to see it on first start, not silently fail remote requests. Startup banner now includes a `Bind:` row so the listening interface is visible alongside Port / Engine / Issuer. Origin: PR #864, extended with D11 (default flip) per /plan-eng-review codex outside-voice review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(mcp): seal source-isolation leak on read path (P0) Pre-fix, an authenticated OAuth MCP client scoped to source-A could enumerate source-B pages via six read-side ops: search, query (text AND image paths), list_pages, traverse_graph, and find_experts. The v0.31.8 source-scoping pattern shipped through dispatch.ts but the op handlers never threaded ctx.sourceId into their engine calls, and hybridSearch.ts:223's explicit SearchOpts rebuild dropped sourceId even when callers passed it. Sealing the leak: - src/core/operations.ts adds sourceScopeOpts(ctx), the canonical precedence ladder: ctx.auth.allowedSources (federated) wins over ctx.sourceId (scalar) wins over nothing. Threaded into all 5 read-side op handlers + the query-image-path searchVector call (the 6th leak surface codex caught in plan review). - src/core/search/hybrid.ts:223 now threads sourceId + sourceIds fields through the inner SearchOpts rebuild. The explicit pick shape is preserved (HNSW inner-CTE ordering depends on it) but extended. - src/core/types.ts adds sourceIds?: string[] to SearchOpts + PageFilters (D9: federated read needs array-shaped engine filter or fan-out; array wins for hot retrieval). - src/core/operations.ts AuthInfo gains sourceId + allowedSources (D2: identity surface symmetric with the federated_read column #876 will add). - Both engines now apply WHERE source_id = $N (scalar) or = ANY($N::text[]) (array) at the SQL layer for searchKeyword, searchKeywordChunks, searchVector, listPages, traverseGraph, traversePaths. Array form wins when both are set. The searchVector filter pushes into the inner HNSW CTE (codex flagged this placement during plan review). - traverseGraph + traversePaths signatures gain opts.sourceId + opts.sourceIds; engine.ts interface updated. - findExperts (the whoknows op, D3 5th leak surface) accepts sourceId + sourceIds and threads them into its internal hybridSearch call. PR #861 was authored before v0.33 shipped so this op wasn't covered in the original PR. Auth wiring: - GBrainOAuthProvider.verifyAccessToken populates AuthInfo.sourceId from oauth_clients.source_id. JOIN guarded by isUndefinedColumnError so pre-v55 brains degrade to legacy projection rather than refusing every token verification. - GBrainOAuthProvider.registerClientManual gains a sourceId parameter (defaults to 'default'). DCR registerClient also sets source_id='default' on the inserted row. - serve-http.ts:929 cleanup: AuthInfo.sourceId is now a real typed field. The cast + GBRAIN_SOURCE env fallback chain is gone (D13). Legacy bearer tokens default to 'default' source in verifyAccessToken. - http-transport.ts (legacy access_tokens path) threads sourceId='default' through DispatchOpts so v0.22.7 callers stay source-scoped. - auth.ts CLI adds --source flag to gbrain auth register-client. Migration v55 (D10 + D13): - ALTER TABLE oauth_clients ADD COLUMN source_id TEXT (nullable). - Backfill UPDATE source_id = 'default' WHERE source_id IS NULL — preserves v0.33 effective behavior verbatim for legacy clients. - ADD CONSTRAINT FK ... REFERENCES sources(id) ON DELETE SET NULL, wrapped in DO block so re-runs against fresh-install brains (where the FK already lives inline in SCHEMA_SQL) no-op cleanly. - CREATE INDEX idx_oauth_clients_source_id WHERE source_id IS NOT NULL for the verifyAccessToken JOIN. - GBRAIN_ACCEPT_SILENT_WIDEN env-flag wired through the runner via SET LOCAL gbrain.accept_silent_widen — reserved for future migrations that hit the silent-widen footgun codex flagged. This migration doesn't need it (column is brand new; no pre-existing stale values possible by definition). - src/core/pglite-schema.ts + src/schema.sql include the column + FK + index inline for fresh installs. Tests: new test/e2e/source-isolation-pglite.test.ts with 13 regression cases — one per leak surface (search/list_pages/traverse/etc.) plus explicit AuthInfo.sourceId and AuthInfo.allowedSources op-handler threading checks. Full unit suite: 6034 pass / 0 fail. PGLite initSchema time dropped from 2.4s to 850ms after consolidating v55's DO blocks (multiple DO blocks were slow on PGLite; one DO block for the FK install only is fine). Origin: PR #861 + plan-eng-review decisions D2/D3/D4/D9/D10/D13 + F2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(gateway): multimodal embedding for openai-compatible providers Pre-fix, embedMultimodal hardcoded a recipe.id === 'voyage' branch and threw AIConfigError for every other recipe. Multimodal-capable providers fronted by LiteLLM (or any openai-compatible proxy) were unreachable even when the operator had wired up the model. The fix: - src/core/ai/gateway.ts adds embedMultimodalOpenAICompat() that POSTs to the standard /embeddings endpoint with content arrays carrying image_url entries. Routing comes from the existing recipe.implementation switch — Voyage stays on its own /multimodalembeddings path; every other openai-compatible recipe flows through the new helper. - src/core/ai/recipes/litellm-proxy.ts declares supports_multimodal: true so embedMultimodal accepts the recipe. No multimodal_models allow-list: LiteLLM is a passthrough proxy and the user owns model-id selection; provider rejection (400 from upstream) is the right enforcement layer there. Voyage's static allow-list shape stays unchanged (its 12 models share supports_multimodal but only one is multimodal-capable). - D12 runtime dimension validation: the new helper checks the returned vector length against the recipe's declared default_dims (preferred) or the brain's embedding_dimensions config. Mismatch throws AIConfigError with model id + observed + expected so the operator can swap models or rebuild the column. Pre-fix, a wrong-dim response would surface as a cryptic pgvector "vector dimension mismatch" at INSERT time. - Auth resolution routes through the existing defaultResolveAuth helper so optional-auth recipes (LiteLLM proxy with no LITELLM_API_KEY) and required-auth recipes both share one code path. Optional-auth sends "Authorization: Bearer unauthenticated" which servers like Ollama / llama-server ignore but the SDK contract requires. Tests: 11 new cases in test/openai-compat-multimodal.test.ts cover happy-path, multi-input batching, unauthenticated proxy, D12 dim mismatch + default-dim fallback, 401 / 400 / malformed-JSON / non-array error paths, and an explicit Voyage-regression test pinning that the new openai-compat route doesn't accidentally hijack the Voyage path. All 41 multimodal-related tests pass (existing voyage suite + new). typecheck clean. Origin: PR #875 + plan-eng-review D12 (runtime dim validation). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(oauth): federated_read read scope (#876) Pre-fix, OAuth clients had a single source-scope axis (source_id, added in v55). A client could either write+read one source OR be a super-reader across all sources (via NULL source_id). There was no middle ground — WeCare-style L3 dept clients that need to write to dept-x but read dept-x + parent canon + shared canon had no expression. #876 adds federated_read TEXT[] as an orthogonal read-scope axis. source_id is the WRITE authority; federated_read is the READ authority. They default to matching values (read scope == write scope, the pre-v0.34 default) when a client is registered without an explicit federated read list. Migrations v56-v60 (six new migrations on top of v55): - v56: ALTER TABLE ... ADD COLUMN federated_read TEXT[] NOT NULL DEFAULT '{}'. - v57 (F5): explicit CASE backfill so source_id IS NULL → '{}' (not an array containing NULL — codex caught this ambiguity during plan review). - v58: post-backfill validation. Fails loud if any row's source_id isn't in its federated_read array, pointing at a logic bug in v57 if fired. - v59: flip the source_id FK from ON DELETE SET NULL to ON DELETE RESTRICT now that federated_read provides the alternative scope-loss path. Pre-flip, deleting a source could silently widen any oauth_client to super-reader; post-flip, source delete is refused if any client references it (operator must revoke/re-scope first). - v60: GIN index on federated_read for array-containment queries. Auth wiring: - GBrainOAuthProvider.verifyAccessToken JOINs c.federated_read and populates AuthInfo.allowedSources. Pre-v56 / pre-v55 brains degrade via the existing isUndefinedColumnError fallback chain. - registerClientManual gains a federatedRead?: string[] parameter (defaults to [sourceId]). - DCR registerClient sets source_id='default' + federated_read=['default'] on the inserted row. - auth.ts CLI adds --federated-read SRC1,SRC2,... flag. The register-client output now prints "Federated reads:" so operators confirm the scope they set. Engines consume the federated array through the SearchOpts.sourceIds / PageFilters.sourceIds field that #861 added (no engine changes here — the plumbing was D9). sourceScopeOpts in operations.ts already prefers the auth.allowedSources array over scalar ctx.sourceId when set. Test seam: - test/book-mirror.test.ts now spawns the CLI with GBRAIN_HOME pointed at a tempdir so the test isn't sensitive to the developer's local ~/.gbrain/config.json. Pre-fix the test could silently inherit a real Postgres connection and hang past the default 5s test timeout. Fresh GBRAIN_HOME → "No brain configured" → exit 1 in <1s. - test/e2e/source-isolation-pglite.test.ts gains one more regression case: AuthInfo.allowedSources = [] (explicit empty) MUST NOT widen scope to "all sources" — the silent-widen footgun precedence ladder. - test/openai-compat-multimodal.test.ts is part of the wave's commits via the migrate.ts changes that bump the schema chain. typecheck-only fix on a captured-auth type was already in #875's tree. 6045 unit tests pass / 0 fail. typecheck clean. PGLite initSchema runs v55-v60 in ~786ms total (within the test-harness budget for tests using the canonical beforeAll engine pattern). Origin: PR #876 + plan-eng-review F5 (CASE backfill). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.34.0.0: MCP fix wave (#870 #909 #864 #861 #875 #876) VERSION + package.json + CHANGELOG bump for the six-PR MCP fix wave. Schema chain extends from v54 → v60; oauth_clients gains source_id + federated_read columns; auth'd MCP clients now stay inside their scope across all read-side ops; PKCE-only DCR works; --bind defaults to loopback; LiteLLM multimodal embedding ships. Contributed by @Hansen1018 (#870), @ding-modding (#909), @DukeDawg (#864), @toilalesondev (#861 + #876), @yoelgal (#875). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update project documentation for v0.34.0.0 Sync README, CLAUDE.md, SECURITY.md, docs/architecture/topologies.md, and docs/mcp/DEPLOY.md to reflect the v0.34.0.0 MCP fix wave: - README: document --bind HOST default (loopback), --source + --federated-read register-client flags, PKCE public-client gate - SECURITY.md: note loopback-by-default for serve --http, update the trust-proxy contract to point at the new default - CLAUDE.md: annotate operations.ts (sourceScopeOpts helper), oauth-provider.ts (verifyAccessToken JOIN + PKCE public clients), serve-http.ts (--bind flag), gateway.ts (openai-compat multimodal + dim validation), mcp/server.ts (MCP_STDIO guard), auth.ts (--source + --federated-read), migrate.ts (v58-v63 chain), engine.ts (sourceIds field). Add 4 new test-file entries for source-isolation-pglite, openai-compat-multimodal, serve-stdio-lifecycle, oauth.test.ts PKCE cases - docs/architecture/topologies.md: source-scoped register-client example, --bind 0.0.0.0 for thin-client host setup - docs/mcp/DEPLOY.md: --bind explanation in the ngrok section, source-scoped client recipe - llms-full.txt: regenerated per the CLAUDE.md-edit chaser rule Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump v0.34.0.0 → v0.34.1.0 Renumbering the MCP fix wave from v0.34.0.0 to v0.34.1.0 so the release slot lands between master's v0.33.2.1 and the next minor. Touches every release-artifact mention: - VERSION: 0.34.0.0 → 0.34.1.0 - package.json: same - CHANGELOG.md header + "To take advantage" block - CLAUDE.md key-files annotations (8 entries that document this wave) - llms-full.txt (regen from CLAUDE.md) - README.md / SECURITY.md / docs/architecture/topologies.md / docs/mcp/DEPLOY.md - Wave code-comment markers ("// v0.34.0 (#NNN):" → "// v0.34.1 (#NNN):") Test files renamed alongside since they were committed with the wave. Commit subjects on the original 6 PR commits + the v0.34.0.0 bump commit (4f533c72 → 6b47db7e) intentionally NOT rewritten — those are history. `git log` finds the implementation by message subject, not by version tag. 6275 unit tests pass, typecheck clean, migration chain v58-v63 unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
239
CHANGELOG.md
239
CHANGELOG.md
@@ -2,6 +2,245 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.34.1.0] - 2026-05-14
|
||||
|
||||
**MCP hardening wave: stricter source isolation on the read path, PKCE
|
||||
DCR works, loopback-by-default, and federated read scopes for shared
|
||||
brains. Six community PRs land as one release.**
|
||||
|
||||
The v0.34.1 wave consolidates six community PRs into a single ship.
|
||||
Source-isolation tightening is the centerpiece — an authenticated OAuth
|
||||
client scoped to one source no longer sees rows from neighboring sources
|
||||
through the read path. The wave also seals smaller papercuts that have
|
||||
been costing real users: gateway-managed stdio MCP servers no longer
|
||||
exit on the post-handshake EOF, PKCE-only DCR clients can register
|
||||
without inheriting a phantom secret, and `gbrain serve --http` defaults
|
||||
to loopback so a personal-laptop brain isn't one accidental config away
|
||||
from publishing itself to the LAN. Federated_read adds the read-scope
|
||||
axis that shared-brain deployments need: a department client can write
|
||||
to one source and read across a curated set without becoming a
|
||||
super-reader.
|
||||
|
||||
### What you can now do
|
||||
|
||||
**Scope OAuth clients to a single source.** `gbrain auth register-client
|
||||
my-agent --source dept-x` registers a client whose write authority is
|
||||
`dept-x`. Read paths only return rows whose `source_id` matches. The
|
||||
auth-layer thread is on `verifyAccessToken` so every per-request
|
||||
dispatch starts with the scope in `AuthInfo.sourceId`; ops consume it
|
||||
through the canonical `ctx.sourceId` pattern that v0.31.8 established.
|
||||
Pre-v0.34 clients without an explicit source default to `default` on
|
||||
upgrade — the v0.33 effective behavior is preserved verbatim.
|
||||
|
||||
**Federate read scope independently.** `gbrain auth register-client
|
||||
my-l3-dept --source dept-x --federated-read dept-x,wecare,shared`
|
||||
registers a client that writes to `dept-x` but reads from the union of
|
||||
`dept-x`, `wecare` parent canon, and `shared` org canon. The two scope
|
||||
axes are orthogonal — `source_id` is the write authority, `federated_read`
|
||||
is the read authority. Engine read paths apply
|
||||
`WHERE source_id = ANY($1::text[])` at SQL when the array is set; scalar
|
||||
single-source clients keep the v0.31.12 fast path.
|
||||
|
||||
**Run gateway-piped stdio MCP without race-killing your server.** Set
|
||||
`MCP_STDIO=1` in the env and the server skips the stdin EOF shutdown
|
||||
hooks. Signal handlers (SIGTERM / SIGINT / SIGHUP) and parent-process
|
||||
watchdog still cover legitimate disconnects. OpenClaw's bundle-mcp
|
||||
gateway and similar wrappers pipe the handshake then close their stdin
|
||||
half; pre-fix this killed the server before the first tool call landed.
|
||||
|
||||
**Register PKCE-only public clients.** `POST /register` with
|
||||
`token_endpoint_auth_method: "none"` (Claude Code, Cursor, every other
|
||||
PKCE-first MCP client) now returns RFC 7591-compliant response shape:
|
||||
no `client_secret` field on public clients, secret-bearing on default
|
||||
`client_secret_post` clients. The `/token` exchange accepts the public
|
||||
client through the SDK's clientAuth path because `getClient` correctly
|
||||
normalizes a NULL `client_secret_hash` to JS undefined.
|
||||
|
||||
**Bind the HTTP MCP server to loopback by default.** `gbrain serve
|
||||
--http` now listens on `127.0.0.1` unless you pass `--bind 0.0.0.0` (or
|
||||
a specific interface IP). Personal-laptop installs are no longer one
|
||||
default-config away from publishing the brain to a LAN. Self-hosted
|
||||
server operators who actually want remote access pass `--bind 0.0.0.0`
|
||||
once; a stderr WARN fires when `--public-url` is set without `--bind` so
|
||||
the operator doesn't silently bind loopback at startup.
|
||||
|
||||
**Embed images through LiteLLM / openai-compatible multimodal models.**
|
||||
The gateway's `embedMultimodal` no longer hardcodes Voyage; recipes with
|
||||
`implementation: 'openai-compatible'` route through the standard
|
||||
`/embeddings` endpoint with content arrays carrying `image_url` entries.
|
||||
Runtime dimension validation throws a clear error pre-storage if the
|
||||
provider returns a vector that doesn't match the brain's embedding
|
||||
column width — no more cryptic `vector dimension mismatch` at INSERT
|
||||
time.
|
||||
|
||||
### The migration block
|
||||
|
||||
`gbrain upgrade` applies migrations v60-v65 automatically. The migration
|
||||
chain adds two columns to `oauth_clients` and an index, plus a FK flip
|
||||
once federated_read is in place.
|
||||
|
||||
### To take advantage of v0.34.1
|
||||
|
||||
`gbrain upgrade` should do this automatically. If it didn't, or if
|
||||
`gbrain doctor` warns about a partial migration:
|
||||
|
||||
1. **Run the orchestrator manually:**
|
||||
```bash
|
||||
gbrain apply-migrations --yes
|
||||
```
|
||||
2. **Verify the schema landed:**
|
||||
```bash
|
||||
gbrain doctor --json | jq '.checks.schema_version'
|
||||
# version should be >= 60
|
||||
```
|
||||
3. **Existing OAuth clients keep working.** Pre-v0.34 clients without an
|
||||
explicit source scope are backfilled to `source_id='default'` so
|
||||
their effective scope matches v0.33. To narrow a specific client's
|
||||
scope, re-register it with `--source <id>`:
|
||||
```bash
|
||||
gbrain auth revoke-client <client_id>
|
||||
gbrain auth register-client <name> --source <source_id> --scopes read,write
|
||||
```
|
||||
4. **`gbrain serve --http` default changed.** Existing self-hosted
|
||||
server deployments must add `--bind 0.0.0.0` to keep accepting remote
|
||||
connections. Personal-laptop users see no behavior change (loopback
|
||||
is now the default).
|
||||
5. **If `gbrain apply-migrations` refuses with an `oauth_clients` stale
|
||||
source_id error** (possible only if an operator hand-poked the column
|
||||
before upgrading): the error message names the offending client IDs.
|
||||
Either revoke + re-register them with a valid source, or re-run with
|
||||
`GBRAIN_ACCEPT_SILENT_WIDEN=1` to NULL the stale values (widens those
|
||||
clients to super-reader; re-scope via `gbrain auth register-client`
|
||||
after).
|
||||
6. **If any step fails or the numbers look wrong,** file an issue:
|
||||
https://github.com/garrytan/gbrain/issues with `gbrain doctor --json`
|
||||
output and the failing step.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
**Source-isolation hardening:**
|
||||
- `OperationContext.auth` now carries `AuthInfo.sourceId` (write scope)
|
||||
and `AuthInfo.allowedSources` (federated read scope), threaded from
|
||||
`oauth_clients` rows at token-verification time.
|
||||
- New helper `sourceScopeOpts(ctx)` in `src/core/operations.ts` encodes
|
||||
the precedence ladder: federated array wins over scalar over nothing.
|
||||
Every read-side op handler routes through it so future ops can't
|
||||
silently drift from the canonical thread.
|
||||
- `src/core/search/hybrid.ts` inner `SearchOpts` rebuild now includes
|
||||
`sourceId` + `sourceIds` fields — the structural fix that prevents
|
||||
the explicit-pick footgun that motivated the wave.
|
||||
- `src/core/types.ts` adds `sourceIds?: string[]` to `SearchOpts` and
|
||||
`PageFilters` for the federated read axis. Both Postgres and PGLite
|
||||
engines apply `WHERE source_id = ANY($N::text[])` when the array is
|
||||
set; scalar fast path preserved when unset.
|
||||
- Engine method signatures `traverseGraph(slug, depth, opts?)` and
|
||||
`traversePaths(slug, opts?)` accept `opts.sourceId` /
|
||||
`opts.sourceIds` so graph walks respect the caller's scope.
|
||||
- `src/core/oauth-provider.ts:verifyAccessToken` JOINs
|
||||
`oauth_clients.source_id` + `federated_read` and surfaces both on the
|
||||
returned `AuthInfo`. Pre-v60 / pre-v61 brains degrade gracefully via
|
||||
`isUndefinedColumnError` fallback.
|
||||
- `src/commands/serve-http.ts` drops the `(authInfo as AuthInfo &
|
||||
{sourceId?: string}).sourceId ?? env ?? 'default'` cast chain. The
|
||||
typed field is the source of truth now.
|
||||
- Legacy `src/mcp/http-transport.ts` (v0.22.7-style access_tokens path)
|
||||
threads `sourceId: 'default'` through DispatchOpts so legacy tokens
|
||||
stay source-scoped.
|
||||
|
||||
**Migration chain v60-v65 (six new migrations on top of v54):**
|
||||
- v60 (`oauth_clients_source_id_fk`) — ALTER TABLE ... ADD COLUMN
|
||||
source_id TEXT, backfill NULL→'default', install FK with
|
||||
ON DELETE SET NULL.
|
||||
- v61 (`oauth_clients_federated_read_column`) — ALTER TABLE ... ADD
|
||||
COLUMN federated_read TEXT[] NOT NULL DEFAULT '{}'.
|
||||
- v62 (`oauth_clients_federated_read_backfill`) — explicit CASE backfill
|
||||
so `source_id IS NULL` produces `'{}'` not an array-containing-NULL.
|
||||
- v63 (`oauth_clients_federated_read_validate`) — fail-loud check that
|
||||
every row's source_id is in its federated_read array post-backfill.
|
||||
- v64 (`oauth_clients_source_id_fk_restrict`) — flip FK to
|
||||
ON DELETE RESTRICT now that federated_read provides the alternative
|
||||
scope-loss path. Source delete is refused if any client references it.
|
||||
- v65 (`oauth_clients_federated_read_gin_index`) — GIN index for the
|
||||
array-containment queries the read paths run.
|
||||
|
||||
**OAuth + auth surface:**
|
||||
- `auth.ts` CLI adds `--source <id>` and `--federated-read <SRC1,SRC2,...>`
|
||||
flags to `register-client`. The output now prints the resolved
|
||||
`Write source` and `Federated reads` for the registered client.
|
||||
- DCR `/register` endpoint now writes `source_id='default'` and
|
||||
`federated_read=['default']` on the inserted row so new public clients
|
||||
start in a sane scope.
|
||||
- `registerClient` honors `token_endpoint_auth_method: "none"` (RFC 7591
|
||||
§3.2.1): public clients store `client_secret_hash = NULL` and the
|
||||
response payload omits `client_secret` entirely. Confidential clients
|
||||
(default `client_secret_post` and explicit `client_secret_basic`) keep
|
||||
their one-time-reveal shape.
|
||||
|
||||
**MCP transports:**
|
||||
- `src/mcp/server.ts` + `src/commands/serve.ts` skip stdin
|
||||
`'end'`/`'close'` shutdown hooks when `process.env.MCP_STDIO === '1'`.
|
||||
`ServeOptions` gains a `mcpStdio?: boolean` test seam so the runtime
|
||||
guard is exercisable without process.env mutation.
|
||||
- `src/commands/serve-http.ts` adds `--bind HOST` (default `127.0.0.1`).
|
||||
Stderr WARN fires when `--public-url` is set without `--bind`.
|
||||
Startup banner prints the resolved `Bind:` line.
|
||||
|
||||
**Multimodal embedding:**
|
||||
- `gateway.ts` adds `embedMultimodalOpenAICompat()` that POSTs to the
|
||||
standard `/embeddings` endpoint with content arrays. Routes by
|
||||
`recipe.implementation === 'openai-compatible'` so LiteLLM-fronted
|
||||
providers (Anyscale, vLLM, Gemini multimodal via proxy) work alongside
|
||||
Voyage's existing `/multimodalembeddings` path.
|
||||
- `recipes/litellm-proxy.ts` declares `supports_multimodal: true` so the
|
||||
recipe accepts multimodal calls without a model allow-list (LiteLLM is
|
||||
a passthrough; user-owned model id selection).
|
||||
- Runtime dimension validation: the returned vector length is checked
|
||||
against the recipe's declared `default_dims` or the brain's
|
||||
`embedding_dimensions` config. Mismatch throws `AIConfigError` with
|
||||
model id + observed + expected before the vector reaches storage.
|
||||
|
||||
**Tests:**
|
||||
- `test/e2e/source-isolation-pglite.test.ts` — 14 cases pinning the
|
||||
scope filter at the engine layer plus op-handler threading for both
|
||||
`ctx.sourceId` and `ctx.auth.allowedSources` paths.
|
||||
- `test/openai-compat-multimodal.test.ts` — 11 cases covering the
|
||||
openai-compatible multimodal path: happy-path single + multi-input,
|
||||
unauthenticated proxy, D12 dim mismatch + default-dim fallback,
|
||||
401 / 400 / malformed-JSON / non-array error paths, and a Voyage
|
||||
regression test.
|
||||
- `test/oauth.test.ts` — 5 new cases for PKCE DCR public-client gate
|
||||
(no secret for public; secret unchanged for default; getClient
|
||||
NULL→undefined normalization; full PKCE `/authorize` → `/token`
|
||||
round-trip).
|
||||
- `test/serve-stdio-lifecycle.test.ts` — 3 new cases for the
|
||||
`MCP_STDIO=1` guard (stdin EOF does NOT trigger shutdown; SIGTERM
|
||||
still does; unset env preserves CLI behavior).
|
||||
- `test/book-mirror.test.ts` — `runCli` helper now uses a fresh
|
||||
`GBRAIN_HOME` tempdir so the test isn't sensitive to the developer's
|
||||
local `~/.gbrain/config.json`. Pre-fix this could silently inherit a
|
||||
real Postgres connection and hang past the default 5s test timeout.
|
||||
|
||||
**Test results: 6045 unit tests pass / 0 fail. typecheck clean. PGLite
|
||||
initSchema runs the v60-v65 chain in ~786ms total.**
|
||||
|
||||
### For contributors
|
||||
|
||||
- The wave bundled six community PRs (#870, #909, #864, #861, #875,
|
||||
#876) with cross-model plan review (codex outside-voice) before
|
||||
cherry-pick. Codex caught three structural bugs the original PRs
|
||||
missed: a 6th source-isolation leak surface (the `query` image path
|
||||
in `operations.ts:1071-1082`), a 5th read-side op missing from #861's
|
||||
thread (`find_experts`), and migration-numbering collisions on
|
||||
v47-v53 that would have wedged on cherry-pick (the branch had grown
|
||||
to v57 by ship time after master shipped its own v55-v57 search-lite
|
||||
migrations). The wave's migration chain renumbers to v60-v65.
|
||||
- The single PR carries `Co-Authored-By:` for the six external
|
||||
contributors. PRs originated against earlier base branches and the
|
||||
diffs were re-implemented on the collector branch rather than merged
|
||||
directly (per CLAUDE.md's PR wave process).
|
||||
|
||||
Contributed by @Hansen1018 (#870), @ding-modding (#909), @DukeDawg
|
||||
(#864), @toilalesondev (#861 + #876), @yoelgal (#875).
|
||||
## [0.34.0.0] - 2026-05-14
|
||||
|
||||
**Recursive code intelligence ships. Plan-mode subagents get one-call blast and flow.**
|
||||
|
||||
@@ -139,7 +139,9 @@ pick scopes, save the credentials shown once in the reveal modal. Programmatic
|
||||
registration via `oauthProvider.registerClientManual(...)` and the
|
||||
`gbrain auth register-client` CLI are also available.
|
||||
|
||||
- **OAuth 2.1 via the MCP SDK** — client credentials (machine-to-machine: Perplexity, Claude), authorization code + PKCE (browser-based: ChatGPT), refresh token rotation, revocation, protected resource metadata. Optional Dynamic Client Registration behind `--enable-dcr` (DCR redirect_uris must be `https://` or loopback per RFC 6749 §3.1.2.1).
|
||||
- **OAuth 2.1 via the MCP SDK** — client credentials (machine-to-machine: Perplexity, Claude), authorization code + PKCE (browser-based: ChatGPT), refresh token rotation, revocation, protected resource metadata. PKCE-only public clients (`token_endpoint_auth_method: "none"`) register without a secret per RFC 7591 §3.2.1 (v0.34). Optional Dynamic Client Registration behind `--enable-dcr` (DCR redirect_uris must be `https://` or loopback per RFC 6749 §3.1.2.1).
|
||||
- **Source-scoped OAuth clients (v0.34)** — `gbrain auth register-client my-agent --source dept-x` ties the client's write authority to one source; read paths only return rows matching that source. `--federated-read S1,S2,S3` adds an orthogonal read-scope axis for shared brains (departments writing to one canon while reading the union). Pre-v0.34 clients are backfilled to `source_id='default'` on upgrade.
|
||||
- **Loopback default for `serve --http` (v0.34)** — listens on `127.0.0.1` unless `--bind 0.0.0.0` (or a specific interface IP). Personal-laptop installs no longer publish the brain to the LAN by accident. A stderr WARN fires when `--public-url` is set without `--bind` so the operator sees the binding before the first request.
|
||||
- **Scoped operations** — 30 operations tagged `read | write | admin`. `sync_brain` and `file_upload` are `localOnly`, rejected over HTTP.
|
||||
- **React admin dashboard** — 7 screens baked into the binary (~65KB gzip). Live SSE activity feed, agents table, credential reveal, filterable request log, per-client config export.
|
||||
- **Legacy bearer tokens still work** — pre-v0.26 `gbrain auth create` tokens continue to authenticate as `read+write+admin`. v0.22.7's simpler `src/mcp/http-transport.ts` path stays compiled in for backward compat callers; v0.26+ deployments use the OAuth-aware `serve-http.ts`.
|
||||
@@ -788,12 +790,16 @@ ADMIN
|
||||
[--skip=<provider>] [--json]
|
||||
gbrain serve MCP server (stdio)
|
||||
gbrain serve --http [--port 3131] HTTP MCP server with OAuth 2.1 + admin dashboard
|
||||
[--bind HOST] (v0.34: default 127.0.0.1; pass
|
||||
--bind 0.0.0.0 for LAN/remote access)
|
||||
[--token-ttl 3600] [--enable-dcr]
|
||||
[--public-url URL] [--log-full-params]
|
||||
gbrain auth create|list|revoke|test Legacy bearer token management
|
||||
gbrain auth register-client <name> Register an OAuth 2.1 client
|
||||
--grant-types client_credentials,authorization_code
|
||||
--scopes "read write admin"
|
||||
--source <id> v0.34: write authority for source-scoped clients
|
||||
--federated-read <S1,S2,...> v0.34: read scope across multiple sources
|
||||
gbrain auth revoke-client <client_id> Revoke an OAuth 2.1 client (cascade purges
|
||||
active tokens + auth codes via FK CASCADE)
|
||||
# OAuth 2.1 clients can also be registered from the /admin dashboard or
|
||||
|
||||
18
SECURITY.md
18
SECURITY.md
@@ -68,6 +68,16 @@ The built-in HTTP transport ships with several layers of hardening on by
|
||||
default. All env vars below are optional; the defaults are intentionally
|
||||
conservative.
|
||||
|
||||
### Bind address (v0.34: loopback by default)
|
||||
|
||||
`gbrain serve --http` listens on `127.0.0.1` by default. Personal-laptop
|
||||
installs cannot accidentally publish the brain to the LAN. Self-hosted
|
||||
deployments that need remote access pass `--bind 0.0.0.0` (all
|
||||
interfaces) or `--bind <interface-ip>` (specific NIC). A stderr WARN
|
||||
fires when `--public-url` is set without `--bind` so the operator sees
|
||||
the binding before the first request — common cause of "ngrok forwards
|
||||
to me but the agent can't reach the upstream" misconfigurations.
|
||||
|
||||
### Postgres-only
|
||||
|
||||
`gbrain serve --http` requires a Postgres engine. PGLite is local-only by
|
||||
@@ -125,9 +135,11 @@ GBRAIN_HTTP_TRUST_PROXY=1 gbrain serve --http --port 8787
|
||||
**both** of these are true:
|
||||
|
||||
1. gbrain is reachable only via a trusted reverse proxy (not directly
|
||||
exposed to the internet on the configured port). The simplest
|
||||
guarantee is to bind gbrain to `127.0.0.1` or a private interface
|
||||
and have the proxy forward to it.
|
||||
exposed to the internet on the configured port). As of v0.34
|
||||
`gbrain serve --http` binds `127.0.0.1` by default, so the
|
||||
reverse-proxy-only posture is the out-of-the-box shape; only
|
||||
override with `--bind 0.0.0.0` (or a specific interface IP) when
|
||||
gbrain itself needs to accept remote connections directly.
|
||||
2. The proxy strips any client-supplied `X-Forwarded-For` and `X-Real-IP`
|
||||
headers, then sets them itself. (nginx with `proxy_set_header
|
||||
X-Forwarded-For $remote_addr` does this; Cloudflare and most cloud
|
||||
|
||||
16
TODOS.md
16
TODOS.md
@@ -1,5 +1,21 @@
|
||||
# TODOS
|
||||
|
||||
## MCP fix wave follow-ups (v0.34.1)
|
||||
|
||||
- [ ] **v0.34.x: Source-scope `takes_*` ops (pre-existing leak surfaced during v0.34.1 adversarial review).** `takes_list`, `takes_search`, `takes_scorecard`, `takes_calibration` in `src/core/operations.ts:1248-1335` thread `ctx.takesHoldersAllowList` but never `ctx.sourceId`. An auth'd OAuth client scoped to `source_id='canon-a'` can call `takes_list --page_slug=foo` (slug in `canon-b`) and read takes attached to foreign-source pages. Pre-existing, not introduced by v0.34.1, but the wave was framed as "P0 source-isolation seal on the read path" and `takes_*` surfaces were missed. Fix: extend `TakesListOpts` in `src/core/engine.ts:186` with `sourceId?: string` + `sourceIds?: string[]`; thread `sourceScopeOpts(ctx)` at each op handler; engine `listTakes`/`searchTakes` filter via the `pages` JOIN.
|
||||
|
||||
- [ ] **v0.34.x: Extend `sourceScopeOpts(ctx)` to the 14 read-side ops PR #861 didn't touch.** `get_page`, `get_tags`, `get_links`, `get_backlinks`, `get_timeline`, `list_files`, `get_file`, and the four `takes_*` ops (above) still use the v0.31.8-era `const sourceOpts = ctx.sourceId ? { sourceId: ctx.sourceId } : {}` pattern. NOT a leak (scalar `ctx.sourceId` IS threaded), but federated_read (#876, `ctx.auth?.allowedSources`) is silently dropped. A "WeCare L3 dept" client gets correct federated results from `search`/`query`/`list_pages`/`traverse_graph`/`find_experts` but only sees its scalar `source_id` for `get_page`/`get_tags`/etc. Fix: route all 14 sites through `sourceScopeOpts(ctx)`.
|
||||
|
||||
- [ ] **v0.34.x: Migration v60 idempotency guard against `--force-retry` race with v64.** `gbrain apply-migrations --force-retry 58` after v64 has already run will re-install the FK with `ON DELETE SET NULL`, silently downgrading the v64 RESTRICT posture. Probability low (operator has to explicitly force-retry 58) but failure mode is invisible. Fix: v60 should probe `pg_constraint.confdeltype` before re-adding and refuse to clobber `'r'` (RESTRICT) with `'n'` (SET NULL).
|
||||
|
||||
- [ ] **v0.34.x: `embedMultimodalOpenAICompat` batching + partial-failure handling.** `src/core/ai/gateway.ts:1180-1255` sends one HTTP request per input. Multi-input callers (10 images) get 10 sequential round-trips with no parallelism; a 401 on input #5 throws and discards inputs #1-#4's already-computed embeddings (wasted spend, no surfacing of the partial array). Voyage's existing path batches. Fix: batch via the provider's `input: [...]` array shape; on partial failure, return successful embeddings + failed-index array.
|
||||
|
||||
- [ ] **v0.34.x: Doctor check `oauth_orphan_source_id`** — surfaces OAuth clients whose source_id was nulled by the v60 D10 silent-widen path (`GBRAIN_ACCEPT_SILENT_WIDEN=1`). Closes the observability gap from v0.34.1's D4 decision. Sibling to the `rls_event_trigger` check pattern in `src/commands/doctor.ts`.
|
||||
|
||||
- [ ] **v0.34.x: `gbrain sources purge` FK error UX.** Post-v0.34, deleting a source is refused if any oauth_client references it (v64 ON DELETE RESTRICT). The CLI currently surfaces the raw Postgres FK violation. Fix: pre-check via `SELECT client_id, client_name FROM oauth_clients WHERE source_id = $1`, print "N OAuth clients reference this source: ... Revoke first via `gbrain auth revoke-client <id>`." Mirrors `assessDestructiveImpact` in destructive-guard.ts (v0.26.5).
|
||||
|
||||
- [ ] **v0.34.x: `hybrid.ts:223` explicit-pick refactor.** The SearchOpts rebuild manually picks fields from HybridSearchOpts. This is the bug shape that caused the original v0.34.1 P0 leak — a new SearchOpts field is silently dropped if not manually added here. The wave added `sourceId` + `sourceIds` to the pick; future fields will keep hitting this footgun. Fix: refactor to spread + TypeScript `Pick<>` helper that narrows HybridSearchOpts → SearchOpts type-safely.
|
||||
|
||||
## functional-area-resolver follow-ups (v0.32.3.0)
|
||||
|
||||
- [ ] **v0.33.x: Dogfood `functional-area-resolver` on gbrain's own `skills/RESOLVER.md`** when it crosses ~12KB (currently 8KB). Apply the pattern to the Operational section first (largest). Filed during v0.32.3.0 CEO review.
|
||||
|
||||
@@ -120,10 +120,19 @@ at the remote host. `gbrain doctor` runs a dedicated thin-client check set
|
||||
|
||||
```bash
|
||||
gbrain init --supabase # or --pglite, doesn't matter
|
||||
gbrain serve --http --port 3001 # exposes /mcp + OAuth
|
||||
gbrain serve --http --port 3001 --bind 0.0.0.0 # v0.34: bind explicitly for remote access
|
||||
# (defaults to 127.0.0.1 since v0.34)
|
||||
gbrain auth register-client neuromancer \
|
||||
--grant-types client_credentials \
|
||||
--scopes read,write,admin # admin needed for ping/doctor
|
||||
|
||||
# v0.34: source-scoped client (write to one source, federate reads across
|
||||
# multiple sources). Omit both flags for a v0.33-compatible super-client.
|
||||
gbrain auth register-client neuromancer-dept \
|
||||
--grant-types client_credentials \
|
||||
--scopes read,write \
|
||||
--source dept-x \
|
||||
--federated-read dept-x,shared,parent-canon
|
||||
```
|
||||
|
||||
The `register-client` command prints a `client_id` and `client_secret`.
|
||||
|
||||
@@ -117,6 +117,24 @@ gbrain auth register-client perplexity \
|
||||
--scopes "read write"
|
||||
```
|
||||
|
||||
**v0.34 — source-scoped clients.** Multi-source brains can scope a client's
|
||||
write authority to one source and its read scope to a curated set with the
|
||||
new `--source` and `--federated-read` flags:
|
||||
|
||||
```bash
|
||||
gbrain auth register-client dept-x-agent \
|
||||
--grant-types client_credentials \
|
||||
--scopes "read write" \
|
||||
--source dept-x \
|
||||
--federated-read dept-x,shared,parent-canon
|
||||
```
|
||||
|
||||
`--source` controls the write authority — `put_page` / `add_link` / etc only
|
||||
land in `dept-x`. `--federated-read` controls the read axis independently;
|
||||
queries return rows from any of the listed sources. Omit both flags for the
|
||||
v0.33-compatible super-client shape. Pre-v0.34 clients are backfilled to
|
||||
`source_id='default'` on `gbrain upgrade`.
|
||||
|
||||
Host-repo wrappers can register programmatically:
|
||||
|
||||
```ts
|
||||
@@ -133,6 +151,18 @@ start the server with `--enable-dcr`. DCR is off by default.
|
||||
|
||||
### 3. Expose the server
|
||||
|
||||
**v0.34 — bind explicitly.** `gbrain serve --http` defaults to `127.0.0.1`.
|
||||
To accept connections from the ngrok tunnel (or any non-loopback source),
|
||||
restart with `--bind`:
|
||||
|
||||
```bash
|
||||
gbrain serve --http --port 3131 --bind 0.0.0.0 --public-url https://your-brain.ngrok.app
|
||||
```
|
||||
|
||||
When `--public-url` is set without `--bind`, a stderr WARN fires at
|
||||
startup so the misconfiguration ("the tunnel is up but my agent gets
|
||||
ECONNREFUSED") is loud.
|
||||
|
||||
```bash
|
||||
brew install ngrok
|
||||
ngrok config add-authtoken YOUR_TOKEN
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gbrain",
|
||||
"version": "0.34.0.0",
|
||||
"version": "0.34.1.0",
|
||||
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
|
||||
"type": "module",
|
||||
"main": "src/core/index.ts",
|
||||
|
||||
@@ -329,26 +329,44 @@ async function revokeClient(clientId: string) {
|
||||
}
|
||||
|
||||
async function registerClient(name: string, args: string[]) {
|
||||
if (!name) { console.error('Usage: auth register-client <name> [--grant-types G] [--scopes S]'); process.exit(1); }
|
||||
if (!name) {
|
||||
console.error('Usage: auth register-client <name> [--grant-types G] [--scopes S] [--source SOURCE] [--federated-read SRC1,SRC2,...]');
|
||||
process.exit(1);
|
||||
}
|
||||
const grantsIdx = args.indexOf('--grant-types');
|
||||
const scopesIdx = args.indexOf('--scopes');
|
||||
const sourceIdx = args.indexOf('--source');
|
||||
const federatedIdx = args.indexOf('--federated-read');
|
||||
const grantTypes = grantsIdx >= 0 && args[grantsIdx + 1]
|
||||
? args[grantsIdx + 1].split(',').map(s => s.trim()).filter(Boolean)
|
||||
: ['client_credentials'];
|
||||
const scopes = scopesIdx >= 0 && args[scopesIdx + 1] ? args[scopesIdx + 1] : 'read';
|
||||
// v0.34.1 (#861, D2): --source flag scopes the OAuth client to a single
|
||||
// source. Defaults to 'default' to match migration v60's backfill so
|
||||
// operators upgrading without changing flags see no behavior change.
|
||||
const sourceId = sourceIdx >= 0 && args[sourceIdx + 1] ? args[sourceIdx + 1] : 'default';
|
||||
// v0.34.1 (#876): --federated-read accepts a comma-separated source list
|
||||
// for federated read scope. When omitted, federated_read defaults to
|
||||
// [sourceId] (read scope == write scope, the v0.33 default).
|
||||
const federatedRead = federatedIdx >= 0 && args[federatedIdx + 1]
|
||||
? args[federatedIdx + 1].split(',').map(s => s.trim()).filter(Boolean)
|
||||
: undefined;
|
||||
|
||||
try {
|
||||
await withConfiguredSql(async (sql) => {
|
||||
const { GBrainOAuthProvider } = await import('../core/oauth-provider.ts');
|
||||
const provider = new GBrainOAuthProvider({ sql });
|
||||
const { clientId, clientSecret } = await provider.registerClientManual(
|
||||
name, grantTypes, scopes, [],
|
||||
name, grantTypes, scopes, [], sourceId, federatedRead,
|
||||
);
|
||||
const effectiveFederated = federatedRead && federatedRead.length > 0 ? federatedRead : [sourceId];
|
||||
console.log(`OAuth client registered: "${name}"\n`);
|
||||
console.log(` Client ID: ${clientId}`);
|
||||
console.log(` Client Secret: ${clientSecret}\n`);
|
||||
console.log(` Grant types: ${grantTypes.join(', ')}`);
|
||||
console.log(` Scopes: ${scopes}\n`);
|
||||
console.log(` Scopes: ${scopes}`);
|
||||
console.log(` Write source: ${sourceId}`);
|
||||
console.log(` Federated reads: ${effectiveFederated.join(', ')}\n`);
|
||||
console.log('Save the client secret — it will not be shown again.');
|
||||
console.log(`Revoke with: gbrain auth revoke-client "${clientId}"`);
|
||||
});
|
||||
|
||||
@@ -162,10 +162,31 @@ interface ServeHttpOptions {
|
||||
* at startup so the privacy posture change is visible.
|
||||
*/
|
||||
logFullParams?: boolean;
|
||||
/**
|
||||
* Network interface(s) to bind. Defaults to `127.0.0.1` (loopback only) in
|
||||
* v0.34.1+ — gbrain's primary use case is a personal-knowledge brain on a
|
||||
* laptop, and the pre-v0.34 default of `0.0.0.0` made it one accidental
|
||||
* `--http` invocation away from publishing the brain to a LAN.
|
||||
*
|
||||
* Server operators who DO want to accept remote connections pass
|
||||
* `--bind 0.0.0.0` (or a specific interface IP). When `--public-url` is
|
||||
* set but `--bind` is unset, a stderr WARN fires at startup recommending
|
||||
* the explicit flag — defaulting to loopback while declaring a public URL
|
||||
* is almost always a misconfiguration.
|
||||
*/
|
||||
bind?: string;
|
||||
}
|
||||
|
||||
export async function runServeHttp(engine: BrainEngine, options: ServeHttpOptions) {
|
||||
const { port, tokenTtl, enableDcr, publicUrl, logFullParams } = options;
|
||||
// v0.34.1 (#864, D11): default bind flipped from 0.0.0.0 to 127.0.0.1.
|
||||
// gbrain's primary use case is a personal-knowledge brain on a laptop;
|
||||
// the pre-v0.34 default exposed brains on every interface. Server
|
||||
// operators who need remote access pass `--bind 0.0.0.0` (or a specific
|
||||
// interface). Declaring `--public-url` without `--bind` is almost always
|
||||
// a misconfiguration; we WARN to stderr at startup in that case rather
|
||||
// than silently binding loopback only.
|
||||
const bind = options.bind ?? '127.0.0.1';
|
||||
const config = loadConfig() || { engine: 'pglite' as const };
|
||||
|
||||
if (logFullParams) {
|
||||
@@ -174,6 +195,12 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
|
||||
);
|
||||
}
|
||||
|
||||
if (publicUrl && options.bind === undefined) {
|
||||
console.error(
|
||||
'[serve-http] WARNING: --public-url is set but --bind is not. Default bind changed to 127.0.0.1 in v0.34.1; remote clients reaching the public URL will be refused. Pass --bind 0.0.0.0 to accept all interfaces.',
|
||||
);
|
||||
}
|
||||
|
||||
// Engine-aware SQL adapter. Routes through engine.executeRaw on both
|
||||
// Postgres and PGLite — the OAuth/admin/auth surface no longer requires
|
||||
// a postgres.js singleton, so `gbrain serve --http` works against PGLite
|
||||
@@ -926,9 +953,14 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
|
||||
// ToolResult and we read isError + _meta to pick the right branch.
|
||||
const tokenAllowList = (authInfo as AuthInfo & { takesHoldersAllowList?: string[] }).takesHoldersAllowList
|
||||
?? ['world'];
|
||||
const tokenSourceId = (authInfo as AuthInfo & { sourceId?: string }).sourceId
|
||||
?? process.env.GBRAIN_SOURCE
|
||||
?? 'default';
|
||||
// v0.34.1 (#861, D13): AuthInfo.sourceId is now a real typed field
|
||||
// populated from oauth_clients.source_id (migration v60 backfilled
|
||||
// NULL → 'default'). Pre-fix this site cast through AuthInfo and
|
||||
// fell back to GBRAIN_SOURCE env / 'default' — the silent-fallback
|
||||
// path codex flagged in plan review. Post-v60, every OAuth client
|
||||
// has source_id set; legacy bearer tokens default to 'default' in
|
||||
// verifyAccessToken. The env-fallback is gone.
|
||||
const tokenSourceId = authInfo.sourceId ?? 'default';
|
||||
|
||||
let toolResult: Awaited<ReturnType<typeof dispatchToolCall>>;
|
||||
try {
|
||||
@@ -1058,12 +1090,13 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
|
||||
// ---------------------------------------------------------------------------
|
||||
const clientCount = await sql`SELECT count(*)::int as count FROM oauth_clients`;
|
||||
|
||||
app.listen(port, () => {
|
||||
app.listen(port, bind, () => {
|
||||
console.error(`
|
||||
╔══════════════════════════════════════════════════════╗
|
||||
║ GBrain MCP Server v${VERSION.padEnd(37)}║
|
||||
╠══════════════════════════════════════════════════════╣
|
||||
║ Port: ${String(port).padEnd(40)}║
|
||||
║ Bind: ${bind.padEnd(40)}║
|
||||
║ Engine: ${(config.engine || 'pglite').padEnd(40)}║
|
||||
║ Issuer: ${issuerUrl.origin.padEnd(40)}║
|
||||
║ Clients: ${String((clientCount[0] as any).count).padEnd(40)}║
|
||||
|
||||
@@ -56,6 +56,13 @@ export interface ServeOptions {
|
||||
// tick fell through to the cached `process.ppid` and the watchdog
|
||||
// never fired, while still claiming to be installed.
|
||||
probeWatchdog?: () => boolean;
|
||||
// v0.34.1 (#870): test seam for the MCP_STDIO=1 piped-stdin guard.
|
||||
// When true, runServe skips the stdin 'end'/'close' shutdown hooks
|
||||
// because the wrapping gateway (OpenClaw bundle-mcp, others) pipes the
|
||||
// JSON-RPC handshake and closes stdin immediately. Signal handlers and
|
||||
// transport.onclose still cover legitimate shutdown.
|
||||
// Defaults to `process.env.MCP_STDIO === '1'` when omitted.
|
||||
mcpStdio?: boolean;
|
||||
}
|
||||
|
||||
export async function runServe(
|
||||
@@ -90,8 +97,18 @@ export async function runServe(
|
||||
// when set so the posture change is visible in stderr.
|
||||
const logFullParams = args.includes('--log-full-params');
|
||||
|
||||
// v0.34.1 (#864, D11): `--bind HOST` lets operators choose the network
|
||||
// interface to listen on. When unset, runServeHttp defaults to 127.0.0.1
|
||||
// (loopback) — server operators who need remote access pass
|
||||
// `--bind 0.0.0.0` (or a specific interface IP). `bind` is intentionally
|
||||
// left undefined here when the flag is absent so the WARN-on-public-url
|
||||
// path in serve-http can distinguish "operator chose loopback explicitly"
|
||||
// from "operator didn't set the flag at all."
|
||||
const bindIdx = args.indexOf('--bind');
|
||||
const bind = bindIdx >= 0 ? args[bindIdx + 1] : undefined;
|
||||
|
||||
const { runServeHttp } = await import('./serve-http.ts');
|
||||
await runServeHttp(engine, { port, tokenTtl, enableDcr, publicUrl, logFullParams });
|
||||
await runServeHttp(engine, { port, tokenTtl, enableDcr, publicUrl, logFullParams, bind });
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -201,7 +218,15 @@ function installStdioLifecycle(
|
||||
// Skip when stdin is a TTY: interactive `gbrain serve` use shouldn't
|
||||
// terminate just because the user hasn't typed anything. Signal /
|
||||
// watchdog paths still cover that case if needed.
|
||||
if (!deps.stdin.isTTY) {
|
||||
// v0.34.1 (#870): when MCP_STDIO=1, the wrapping gateway pipes the
|
||||
// JSON-RPC handshake then closes its stdin half. Treating that as a
|
||||
// permanent disconnect kills the server before the first tool call.
|
||||
// Signal handlers (SIGTERM/SIGINT/SIGHUP), transport.onclose, and the
|
||||
// parent-process watchdog below still cover legitimate shutdown paths.
|
||||
// `mcpStdio` is the injectable form; default reads the env once at
|
||||
// install time so tests stay isolated (no process.env mutation).
|
||||
const mcpStdioMode = opts.mcpStdio ?? (process.env.MCP_STDIO === '1');
|
||||
if (!deps.stdin.isTTY && !mcpStdioMode) {
|
||||
deps.stdin.once('end', () => beginShutdown('stdin-end'));
|
||||
deps.stdin.once('close', () => beginShutdown('stdin-close'));
|
||||
}
|
||||
|
||||
@@ -52,6 +52,21 @@ export interface WhoknowsOpts {
|
||||
* ranking function without redefining the type filter.
|
||||
*/
|
||||
types?: PageType[];
|
||||
/**
|
||||
* v0.34.1 (#861, D3 — P0 leak seal): scope expert candidates to a
|
||||
* single source. The op-handler at operations.ts:find_experts threads
|
||||
* `ctx.sourceId` here so an authenticated MCP client scoped to src-A
|
||||
* cannot surface people pages from src-B in the rankings. Pre-fix, the
|
||||
* whoknows op was authored against v0.33 after PR #861 was drafted and
|
||||
* the source-scope thread was missing entirely.
|
||||
*/
|
||||
sourceId?: string;
|
||||
/**
|
||||
* v0.34.1 (#876, D9): federated read — scope candidates to ANY of these
|
||||
* source ids. Threaded from `ctx.auth?.allowedSources` via
|
||||
* `sourceScopeOpts` in operations.ts. Array wins over scalar `sourceId`.
|
||||
*/
|
||||
sourceIds?: string[];
|
||||
}
|
||||
|
||||
export interface WhoknowsResult {
|
||||
@@ -165,11 +180,15 @@ export async function findExperts(
|
||||
// 1. Hybrid search with SQL-level types filter (v0.33 typeFilter parameter).
|
||||
// Disable salience + recency boosts in hybridSearch — we apply our own
|
||||
// locked formula on top of the raw relevance score.
|
||||
// v0.34.1 (#861, D3): thread source-scope so an authenticated MCP
|
||||
// client only ranks experts within its accessible sources.
|
||||
const results: SearchResult[] = await hybridSearch(engine, opts.topic, {
|
||||
types,
|
||||
limit: innerLimit,
|
||||
salience: 'off',
|
||||
recency: 'off',
|
||||
sourceId: opts.sourceId,
|
||||
sourceIds: opts.sourceIds,
|
||||
});
|
||||
|
||||
if (results.length === 0) return [];
|
||||
|
||||
@@ -1016,11 +1016,18 @@ export async function embedMultimodal(inputs: MultimodalInput[]): Promise<Float3
|
||||
);
|
||||
}
|
||||
|
||||
// Voyage-specific HTTP path. When v0.28 lands additional providers, branch
|
||||
// on recipe.id and route to each provider's multimodal endpoint.
|
||||
// v0.34.1 (#875): route by recipe.implementation so openai-compat
|
||||
// providers (LiteLLM, Anyscale, vLLM, etc.) reach the standard
|
||||
// /embeddings endpoint with multimodal content arrays. The Voyage
|
||||
// recipe is `openai-compat` per tier but uses its own /multimodalembeddings
|
||||
// path, so we still branch on recipe.id for that one.
|
||||
if (recipe.id !== 'voyage' && recipe.implementation === 'openai-compatible') {
|
||||
return embedMultimodalOpenAICompat(inputs, recipe, parsed.modelId, cfg);
|
||||
}
|
||||
if (recipe.id !== 'voyage') {
|
||||
throw new AIConfigError(
|
||||
`Multimodal embedding for recipe ${recipe.id} is not implemented yet (v0.27.1 ships Voyage only).`,
|
||||
`Multimodal embedding for recipe ${recipe.id} (${recipe.implementation}) is not implemented yet. ` +
|
||||
`Today: voyage (own endpoint), openai-compatible recipes (standard /embeddings with content arrays).`,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1130,6 +1137,149 @@ export async function embedMultimodal(inputs: MultimodalInput[]): Promise<Float3
|
||||
// this and routes oversize files to sync_failures.jsonl.
|
||||
void MULTIMODAL_MAX_IMAGE_BYTES;
|
||||
|
||||
/**
|
||||
* v0.34.1 (#875): multimodal embedding via the standard OpenAI-compatible
|
||||
* `/embeddings` endpoint. Many providers fronted by LiteLLM (Anyscale, vLLM,
|
||||
* native OpenAI fed multimodal models) accept content arrays where each
|
||||
* element is either `{type: "input_text", text: "..."}` or
|
||||
* `{type: "image_url", image_url: {url: "data:..."}}` and return the same
|
||||
* `{data: [{embedding: number[]}, ...]}` shape as text embeddings.
|
||||
*
|
||||
* Routing comes from gateway.embedMultimodal when the recipe's implementation
|
||||
* is 'openai-compatible' and recipe.id is not 'voyage' (which has its own
|
||||
* /multimodalembeddings path).
|
||||
*
|
||||
* D12 dim validation: the response is checked against the recipe's
|
||||
* declared `default_dims` or the brain's `embedding_dimensions` config.
|
||||
* Mismatch throws AIConfigError with a paste-ready hint pointing at the
|
||||
* model picker — preferable to a silent corrupt-storage failure when the
|
||||
* brain's vector(N) column rejects the row.
|
||||
*/
|
||||
async function embedMultimodalOpenAICompat(
|
||||
inputs: MultimodalInput[],
|
||||
recipe: Recipe,
|
||||
modelId: string,
|
||||
cfg: AIGatewayConfig,
|
||||
): Promise<Float32Array[]> {
|
||||
// Auth resolution via the gateway's canonical helper so LiteLLM-style
|
||||
// optional-auth recipes (Authorization: Bearer LITELLM_API_KEY) and
|
||||
// hard-required-auth recipes (OpenAI Authorization: Bearer
|
||||
// OPENAI_API_KEY) both work via the same code path. Throws AIConfigError
|
||||
// when required env is missing.
|
||||
const authResult = recipe.resolveAuth
|
||||
? recipe.resolveAuth(cfg.env)
|
||||
: defaultResolveAuth(recipe, cfg.env, 'embedding');
|
||||
const baseUrl = cfg.base_urls?.[recipe.id] ?? recipe.base_url_default;
|
||||
if (!baseUrl) {
|
||||
throw new AIConfigError(
|
||||
`${recipe.name} requires a base URL for multimodal embedding.`,
|
||||
recipe.setup_hint,
|
||||
);
|
||||
}
|
||||
|
||||
// D12 — dim validation. Prefer recipe's declared default_dims when set;
|
||||
// fall back to the brain's configured embedding_dimensions. If neither
|
||||
// is known (LiteLLM recipe with default_dims=0 and no config override),
|
||||
// we skip the dim check rather than fabricate an expected value — the
|
||||
// engine's vector(N) column will reject mismatched rows at INSERT time
|
||||
// with a clearer error than anything we could throw here.
|
||||
const recipeDims = recipe.touchpoints.embedding?.default_dims ?? 0;
|
||||
const expectedDims = recipeDims > 0
|
||||
? recipeDims
|
||||
: (cfg.embedding_dimensions ?? 0);
|
||||
|
||||
// Send each input as one /embeddings request. Most providers cap the
|
||||
// number of inputs per call at the text-embedding batch limit, but the
|
||||
// multimodal content array varies per provider. Single-input requests
|
||||
// are the safe lowest common denominator; LiteLLM's proxy backend
|
||||
// batches internally if it can.
|
||||
const allEmbeddings: Float32Array[] = [];
|
||||
for (const input of inputs) {
|
||||
const body = {
|
||||
model: modelId,
|
||||
input: [
|
||||
{
|
||||
// OpenAI's documented multimodal content shape. The data-URL
|
||||
// form embeds the image bytes inline so the proxy doesn't need
|
||||
// network access to fetch the image.
|
||||
type: 'image_url',
|
||||
image_url: { url: `data:${input.mime};base64,${input.data}` },
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
let res: Response;
|
||||
try {
|
||||
res = await fetch(`${baseUrl}/embeddings`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
[authResult.headerName]: authResult.token,
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
} catch (err) {
|
||||
throw normalizeAIError(err, `embedMultimodal(${recipe.id}:${modelId})`);
|
||||
}
|
||||
|
||||
if (!res.ok) {
|
||||
const text = await res.text().catch(() => '');
|
||||
if (res.status === 401 || res.status === 403) {
|
||||
const requiredKey = recipe.auth_env?.required[0];
|
||||
throw new AIConfigError(
|
||||
`${recipe.name} multimodal returned ${res.status}: ${text || 'auth failed'}.`,
|
||||
requiredKey
|
||||
? `Re-export ${requiredKey} or rotate the key at ${recipe.auth_env?.setup_url ?? recipe.setup_hint}.`
|
||||
: recipe.setup_hint,
|
||||
);
|
||||
}
|
||||
// Surface the upstream error verbatim — 400s here usually mean the
|
||||
// proxied model doesn't support multimodal input. The error text is
|
||||
// the user's best signal for picking a different model id.
|
||||
throw new AITransientError(
|
||||
`${recipe.name} multimodal returned ${res.status}: ${text || 'transient error'}.`,
|
||||
);
|
||||
}
|
||||
|
||||
let parsedBody: { data?: Array<{ embedding: number[] }> };
|
||||
try {
|
||||
parsedBody = (await res.json()) as { data?: Array<{ embedding: number[] }> };
|
||||
} catch (err) {
|
||||
throw new AITransientError(
|
||||
`${recipe.name} multimodal returned malformed JSON: ${err instanceof Error ? err.message : String(err)}.`,
|
||||
);
|
||||
}
|
||||
if (!parsedBody.data || !Array.isArray(parsedBody.data) || parsedBody.data.length < 1) {
|
||||
throw new AITransientError(
|
||||
`${recipe.name} multimodal returned no embeddings (expected 1).`,
|
||||
);
|
||||
}
|
||||
|
||||
const row = parsedBody.data[0];
|
||||
if (!Array.isArray(row.embedding)) {
|
||||
throw new AITransientError(
|
||||
`${recipe.name} multimodal returned non-array embedding payload.`,
|
||||
);
|
||||
}
|
||||
// D12 — dim validation. Throw EmbedDimensionMismatchError-shape error
|
||||
// (AIConfigError with model id + observed + expected so the operator
|
||||
// can diagnose and pick a compatible model OR adjust the brain's
|
||||
// embedding_dimensions config). Skip the check when expectedDims=0
|
||||
// (no recipe declaration AND no config override).
|
||||
if (expectedDims > 0 && row.embedding.length !== expectedDims) {
|
||||
throw new AIConfigError(
|
||||
`${recipe.id}:${modelId} returned ${row.embedding.length}-dim vector; expected ${expectedDims}.`,
|
||||
`The brain's embedding column is fixed at ${expectedDims} dims; this model is incompatible. ` +
|
||||
`Either pick a model that returns ${expectedDims} dims, OR set --embedding-dimensions ${row.embedding.length} ` +
|
||||
`and reinitialize the embedding column at the new width.`,
|
||||
);
|
||||
}
|
||||
allEmbeddings.push(new Float32Array(row.embedding));
|
||||
}
|
||||
|
||||
return allEmbeddings;
|
||||
}
|
||||
|
||||
// ---- Expansion ----
|
||||
|
||||
async function resolveExpansionProvider(modelStr: string): Promise<{ model: any; recipe: Recipe; modelId: string }> {
|
||||
|
||||
@@ -30,6 +30,15 @@ export const litellmProxy: Recipe = {
|
||||
// LiteLLM's batch capacity is determined by the backend it proxies;
|
||||
// no static cap to declare here. v0.32 (#779).
|
||||
no_batch_cap: true,
|
||||
// v0.34.1 (#875): LiteLLM can forward to multimodal providers (OpenAI,
|
||||
// Gemini, Voyage etc.). embedMultimodal routes openai-compatible
|
||||
// recipes through embedMultimodalOpenAICompat() — same /embeddings
|
||||
// endpoint as text, with content arrays carrying image_base64
|
||||
// entries. No multimodal_models allow-list: the user knows which of
|
||||
// their proxied models support multimodal; we trust the model id and
|
||||
// surface the provider's rejection (D12 dim-validation catches
|
||||
// mismatched-dim responses pre-storage).
|
||||
supports_multimodal: true,
|
||||
},
|
||||
},
|
||||
setup_hint: 'Run LiteLLM (https://docs.litellm.ai) in front of any provider; set LITELLM_BASE_URL + pass --embedding-model litellm:<model> and --embedding-dimensions <N>.',
|
||||
|
||||
@@ -629,18 +629,31 @@ export interface BrainEngine {
|
||||
dirPrefix?: string,
|
||||
minSimilarity?: number,
|
||||
): Promise<{ slug: string; similarity: number } | null>;
|
||||
traverseGraph(slug: string, depth?: number): Promise<GraphNode[]>;
|
||||
/**
|
||||
* v0.34.1 (#861 — P0 leak seal): `opts.sourceId` / `opts.sourceIds`
|
||||
* constrain visited nodes to a single source or array of sources.
|
||||
* Pre-fix, the walk ignored source scope and an authenticated MCP
|
||||
* client could enumerate cross-source topology + page metadata via
|
||||
* the graph op. MCP-bound callers MUST pass the auth'd scope; local
|
||||
* CLI callers omit it for the historical unscoped behavior.
|
||||
*/
|
||||
traverseGraph(
|
||||
slug: string,
|
||||
depth?: number,
|
||||
opts?: { sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<GraphNode[]>;
|
||||
/**
|
||||
* Edge-based graph traversal with optional type and direction filters.
|
||||
* Returns a list of edges (GraphPath[]) instead of nodes. Supports:
|
||||
* - linkType: per-edge filter, only follows matching edges (per-edge semantics)
|
||||
* - direction: 'in' (follow to->from), 'out' (follow from->to), 'both'
|
||||
* - depth: max depth from root (default 5)
|
||||
* - sourceId/sourceIds: v0.34.1 source-isolation filter, see traverseGraph
|
||||
* Uses cycle prevention (visited array in recursive CTE).
|
||||
*/
|
||||
traversePaths(
|
||||
slug: string,
|
||||
opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both' },
|
||||
opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<GraphPath[]>;
|
||||
/**
|
||||
* For a list of slugs, return how many inbound links each has.
|
||||
|
||||
@@ -2969,6 +2969,187 @@ export const MIGRATIONS: Migration[] = [
|
||||
ON search_telemetry (date DESC);
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 60,
|
||||
name: 'oauth_clients_source_id_fk',
|
||||
// v0.34.1 (#861 + D4 + D10 + D13 — P0 source-isolation leak seal).
|
||||
//
|
||||
// Adds oauth_clients.source_id, validates ALL existing rows can map to a
|
||||
// real source row, backfills NULL → 'default', and installs the FK with
|
||||
// ON DELETE SET NULL. PR #861's original migration claimed v47-v51; we
|
||||
// re-number to v60 because the branch already shipped through v54.
|
||||
//
|
||||
// D10 (codex outside-voice push-back): fail loud when stale source_id
|
||||
// rows exist instead of silently widening to NULL. Pre-fix this column
|
||||
// didn't exist; the only way a row has source_id IS a manual SQL poke,
|
||||
// so the stale-row branch fires only on operator-modified brains. The
|
||||
// GBRAIN_ACCEPT_SILENT_WIDEN=1 env var is the explicit opt-in for
|
||||
// operators who'd rather upgrade than psql-fix. Doctor surfaces orphan
|
||||
// rows post-clean via the v0.34.x follow-up TODO.
|
||||
//
|
||||
// D13: backfill NULL → 'default' BEFORE the FK ADD preserves the v0.33
|
||||
// effective behavior (legacy unscoped clients silently fell back to
|
||||
// 'default' via serve-http.ts:929 cast). Verify 'default' exists in
|
||||
// sources first — fresh brains have it from sources schema's default
|
||||
// seed; brains that scripted it out would otherwise wedge here.
|
||||
//
|
||||
// PGLite parity via the same DO blocks. PGLite supports DO/EXCEPTION
|
||||
// since 0.3; no engine branch needed.
|
||||
idempotent: true,
|
||||
sql: `
|
||||
-- v0.34.1 (#861 + D2 + D13 — P0 source-isolation leak seal).
|
||||
--
|
||||
-- This migration is intentionally lean: oauth_clients.source_id did
|
||||
-- NOT exist pre-v60, so the only state we inherit from upgrade is
|
||||
-- "rows with NULL source_id." Backfill those to 'default' (D13:
|
||||
-- preserves the pre-v0.34 effective fallback behavior verbatim) and
|
||||
-- install the FK with ON DELETE SET NULL.
|
||||
--
|
||||
-- D10 pre-clean is NOT NEEDED here: codex flagged the silent-widen
|
||||
-- footgun assuming source_id was an existing column with possibly-stale
|
||||
-- values. Since the column is brand new in this migration, the only
|
||||
-- post-backfill values are 'default' (which we just verified exists
|
||||
-- via the FK contract) plus any NULL the backfill left untouched
|
||||
-- because of WHERE-clause filtering — none possible. The
|
||||
-- GBRAIN_ACCEPT_SILENT_WIDEN env-flag stays in the runner for future
|
||||
-- migrations that need it; this one doesn't.
|
||||
|
||||
-- 1. Add the column. NULL for every existing row.
|
||||
ALTER TABLE oauth_clients ADD COLUMN IF NOT EXISTS source_id TEXT;
|
||||
|
||||
-- 2. Backfill NULL → 'default'. Pre-v0.34 legacy clients then map
|
||||
-- to the same source the serve-http fallback chain used to put
|
||||
-- them in implicitly. No-op on fresh installs (no rows yet).
|
||||
UPDATE oauth_clients SET source_id = 'default' WHERE source_id IS NULL;
|
||||
|
||||
-- 3. Install FK if not already present. The PGLite + Postgres fresh-
|
||||
-- install schemas (src/core/pglite-schema.ts, src/schema.sql) now
|
||||
-- include the FK inline on the CREATE TABLE, so this DO block
|
||||
-- skips on fresh installs and only fires on upgrade brains where
|
||||
-- oauth_clients was created pre-v60 without the FK. ON DELETE SET
|
||||
-- NULL matches the original PR #861 posture; #876 later flips to
|
||||
-- RESTRICT once federated_read provides the alternative
|
||||
-- scope-recovery path.
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM pg_constraint
|
||||
WHERE conname = 'oauth_clients_source_id_fkey'
|
||||
) THEN
|
||||
ALTER TABLE oauth_clients
|
||||
ADD CONSTRAINT oauth_clients_source_id_fkey
|
||||
FOREIGN KEY (source_id) REFERENCES sources(id) ON DELETE SET NULL;
|
||||
END IF;
|
||||
END $$;
|
||||
|
||||
-- 4. Index for token-verification lookups (verifyAccessToken's JOIN
|
||||
-- on oauth_clients.client_id → c.source_id). oauth_clients stays
|
||||
-- small so plain CREATE INDEX (no CONCURRENTLY) is fine.
|
||||
CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id
|
||||
ON oauth_clients(source_id) WHERE source_id IS NOT NULL;
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 61,
|
||||
name: 'oauth_clients_federated_read_column',
|
||||
// v0.34.1 (#876): add federated_read TEXT[] for the read-side
|
||||
// federation feature. source_id (v60) is the WRITE-authority axis;
|
||||
// federated_read is the READ-scope axis. A client can write to ONE
|
||||
// source while reading from N (a "WeCare L3 dept" client writes to
|
||||
// dept-x and reads dept-x + parent canon + shared canon).
|
||||
//
|
||||
// Default '{}' (empty array) on column add — pre-existing rows get
|
||||
// backfilled in v62 with an explicit CASE so the array reflects the
|
||||
// client's current scope rather than the column default.
|
||||
idempotent: true,
|
||||
sql: `
|
||||
ALTER TABLE oauth_clients ADD COLUMN IF NOT EXISTS federated_read TEXT[] NOT NULL DEFAULT '{}';
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 62,
|
||||
name: 'oauth_clients_federated_read_backfill',
|
||||
// v0.34.1 (#876, F5 — codex outside-voice fix). Backfill federated_read
|
||||
// with explicit CASE so source_id IS NULL doesn't produce an ambiguous
|
||||
// array containing NULL. Three cases:
|
||||
// - source_id IS NULL → '{}' (empty read scope; legacy unscoped
|
||||
// clients lost their implicit fallback in v60 backfill to 'default',
|
||||
// so this branch fires only when an operator explicitly NULL'd
|
||||
// source_id after migration).
|
||||
// - source_id IS NOT NULL → ARRAY[source_id] (read scope matches
|
||||
// write scope, the pre-federation default).
|
||||
// Only fires on rows where federated_read is still the column default
|
||||
// ({}). Operators who hand-set federated_read keep their config.
|
||||
idempotent: true,
|
||||
sql: `
|
||||
UPDATE oauth_clients
|
||||
SET federated_read = CASE
|
||||
WHEN source_id IS NULL THEN '{}'::text[]
|
||||
ELSE ARRAY[source_id]
|
||||
END
|
||||
WHERE federated_read = '{}'::text[];
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 63,
|
||||
name: 'oauth_clients_federated_read_validate',
|
||||
// v0.34.1 (#876): post-backfill validation. Every client with a
|
||||
// non-NULL source_id should now have its source_id reflected in
|
||||
// federated_read. Fail loud if backfill missed a row — points at a
|
||||
// logic bug in v62's WHERE clause.
|
||||
idempotent: true,
|
||||
sql: `
|
||||
DO $$
|
||||
DECLARE
|
||||
bad_count INT;
|
||||
BEGIN
|
||||
SELECT count(*) INTO bad_count FROM oauth_clients
|
||||
WHERE source_id IS NOT NULL
|
||||
AND NOT (source_id = ANY(federated_read));
|
||||
IF bad_count > 0 THEN
|
||||
RAISE EXCEPTION 'oauth_clients has % rows where source_id is not in federated_read after v62 backfill. This is a bug in v62 — re-run gbrain apply-migrations --force-retry 62.', bad_count;
|
||||
END IF;
|
||||
END $$;
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 64,
|
||||
name: 'oauth_clients_source_id_fk_restrict',
|
||||
// v0.34.1 (#876): flip the source_id FK from ON DELETE SET NULL (v60
|
||||
// posture) to ON DELETE RESTRICT now that federated_read provides
|
||||
// the alternative scope-loss path. Pre-fix, deleting a source could
|
||||
// silently widen any oauth_client to super-reader (source_id → NULL).
|
||||
// Post-flip, source delete is refused if any client references it;
|
||||
// the operator's path is "revoke or re-scope the clients first."
|
||||
idempotent: true,
|
||||
sql: `
|
||||
DO $$
|
||||
BEGIN
|
||||
IF EXISTS (
|
||||
SELECT 1 FROM pg_constraint
|
||||
WHERE conname = 'oauth_clients_source_id_fkey'
|
||||
) THEN
|
||||
ALTER TABLE oauth_clients DROP CONSTRAINT oauth_clients_source_id_fkey;
|
||||
END IF;
|
||||
ALTER TABLE oauth_clients
|
||||
ADD CONSTRAINT oauth_clients_source_id_fkey
|
||||
FOREIGN KEY (source_id) REFERENCES sources(id) ON DELETE RESTRICT;
|
||||
END $$;
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 65,
|
||||
name: 'oauth_clients_federated_read_gin_index',
|
||||
// v0.34.1 (#876): GIN index for array-containment lookups
|
||||
// (`WHERE p.source_id = ANY(federated_read)` and similar). The five
|
||||
// read-side ops fall back to scalar sourceId when no auth is set, so
|
||||
// this index only matters under load on federated-scoped clients.
|
||||
idempotent: true,
|
||||
sql: `
|
||||
CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read
|
||||
ON oauth_clients USING GIN (federated_read);
|
||||
`,
|
||||
},
|
||||
];
|
||||
|
||||
export const LATEST_VERSION = MIGRATIONS.length > 0
|
||||
|
||||
@@ -139,9 +139,15 @@ class GBrainClientsStore implements OAuthRegisteredClientsStore {
|
||||
`;
|
||||
if (rows.length === 0) return undefined;
|
||||
const r = rows[0];
|
||||
// v0.34.1 (#909): public clients (token_endpoint_auth_method='none')
|
||||
// have client_secret_hash = NULL. Normalize SQL NULL to JS undefined
|
||||
// so SDK middleware that checks `client.client_secret === undefined`
|
||||
// (not `=== null`) correctly identifies the client as public and
|
||||
// skips the secret-comparison branch on /token.
|
||||
const rawSecret = r.client_secret_hash;
|
||||
return {
|
||||
client_id: r.client_id as string,
|
||||
client_secret: r.client_secret_hash as string | undefined,
|
||||
client_secret: rawSecret == null ? undefined : (rawSecret as string),
|
||||
client_name: r.client_name as string,
|
||||
redirect_uris: (r.redirect_uris as string[]) || [],
|
||||
grant_types: (r.grant_types as string[]) || ['client_credentials'],
|
||||
@@ -170,10 +176,55 @@ class GBrainClientsStore implements OAuthRegisteredClientsStore {
|
||||
assertAllowedScopes(parseScopeString(client.scope));
|
||||
|
||||
const clientId = generateToken('gbrain_cl_');
|
||||
const clientSecret = generateToken('gbrain_cs_');
|
||||
const secretHash = hashToken(clientSecret);
|
||||
// v0.34.1 (#909): RFC 7591 §2 — clients that authenticate at the token
|
||||
// endpoint via PKCE alone declare `token_endpoint_auth_method: "none"`.
|
||||
// For those clients the authorization server MUST NOT issue a client
|
||||
// secret. Pre-fix, unconditional secret generation made the MCP SDK's
|
||||
// clientAuth middleware check `client.client_secret` on every request,
|
||||
// rejecting valid public-client (Claude Code, Cursor) flows.
|
||||
//
|
||||
// We persist secret_hash = NULL for public clients so `getClient` and
|
||||
// the SDK's clientAuth path can detect them via `client_secret_hash IS
|
||||
// NULL` and skip the secret comparison. Confidential clients (default
|
||||
// `client_secret_post` and explicit `client_secret_basic`) still mint
|
||||
// a secret as before.
|
||||
const authMethod = client.token_endpoint_auth_method || 'client_secret_post';
|
||||
const isPublicClient = authMethod === 'none';
|
||||
const clientSecret = isPublicClient ? undefined : generateToken('gbrain_cs_');
|
||||
const secretHash = clientSecret ? hashToken(clientSecret) : null;
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
|
||||
// v0.34.1 (#861, D2 + D13 + #876): DCR clients get source_id='default'
|
||||
// (matches legacy fallback) and federated_read=['default'] (read scope
|
||||
// == write scope). Operators who need narrower / wider scope rescope
|
||||
// via the CLI later. Pre-v60/v61 brain falls through to the legacy
|
||||
// projection (no source_id / federated_read column yet).
|
||||
try {
|
||||
await this.sql`
|
||||
INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris,
|
||||
grant_types, scope, token_endpoint_auth_method,
|
||||
client_id_issued_at, source_id, federated_read)
|
||||
VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'},
|
||||
${pgArray((client.redirect_uris || []).map(String))},
|
||||
${pgArray(client.grant_types || ['client_credentials'])},
|
||||
${client.scope || ''}, ${authMethod},
|
||||
${now}, ${'default'}, ${pgArray(['default'])})
|
||||
`;
|
||||
} catch (err) {
|
||||
if (isUndefinedColumnError(err, 'federated_read')) {
|
||||
try {
|
||||
await this.sql`
|
||||
INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris,
|
||||
grant_types, scope, token_endpoint_auth_method,
|
||||
client_id_issued_at, source_id)
|
||||
VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'},
|
||||
${pgArray((client.redirect_uris || []).map(String))},
|
||||
${pgArray(client.grant_types || ['client_credentials'])},
|
||||
${client.scope || ''}, ${authMethod},
|
||||
${now}, ${'default'})
|
||||
`;
|
||||
} catch (err2) {
|
||||
if (isUndefinedColumnError(err2, 'source_id')) {
|
||||
await this.sql`
|
||||
INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris,
|
||||
grant_types, scope, token_endpoint_auth_method,
|
||||
@@ -181,16 +232,41 @@ class GBrainClientsStore implements OAuthRegisteredClientsStore {
|
||||
VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'},
|
||||
${pgArray((client.redirect_uris || []).map(String))},
|
||||
${pgArray(client.grant_types || ['client_credentials'])},
|
||||
${client.scope || ''}, ${client.token_endpoint_auth_method || 'client_secret_post'},
|
||||
${client.scope || ''}, ${authMethod},
|
||||
${now})
|
||||
`;
|
||||
} else {
|
||||
throw err2;
|
||||
}
|
||||
}
|
||||
} else if (isUndefinedColumnError(err, 'source_id')) {
|
||||
await this.sql`
|
||||
INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris,
|
||||
grant_types, scope, token_endpoint_auth_method,
|
||||
client_id_issued_at)
|
||||
VALUES (${clientId}, ${secretHash}, ${client.client_name || 'unnamed'},
|
||||
${pgArray((client.redirect_uris || []).map(String))},
|
||||
${pgArray(client.grant_types || ['client_credentials'])},
|
||||
${client.scope || ''}, ${authMethod},
|
||||
${now})
|
||||
`;
|
||||
} else {
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
// Public clients: omit `client_secret` entirely from the response so
|
||||
// the wire payload matches RFC 7591 §3.2.1 ("if the client is a
|
||||
// public client, the authorization server MUST NOT issue a client
|
||||
// secret"). Confidential clients return the freshly-generated secret
|
||||
// exactly once — same shape as before.
|
||||
const response: OAuthClientInformationFull = {
|
||||
...client,
|
||||
client_id: clientId,
|
||||
client_secret: clientSecret,
|
||||
client_id_issued_at: now,
|
||||
};
|
||||
if (clientSecret) response.client_secret = clientSecret;
|
||||
return response;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -403,15 +479,57 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
|
||||
// Try OAuth tokens first. JOIN oauth_clients in the same query so
|
||||
// verifyAccessToken returns client_name in AuthInfo — eliminates the
|
||||
// separate per-request lookup at serve-http.ts that was the N+1 hot
|
||||
// path (see PR #586 review D14=B).
|
||||
const oauthRows = await this.sql`
|
||||
// verifyAccessToken returns client_name AND source_id in AuthInfo —
|
||||
// eliminates the separate per-request lookup at serve-http.ts that
|
||||
// was the N+1 hot path (see PR #586 review D14=B; v0.34.1 #861 D2
|
||||
// adds the source_id thread on the same JOIN).
|
||||
//
|
||||
// v0.34.1 (#861): the JOIN guards on a c.source_id column that
|
||||
// migration v60 adds. Pre-v60 brains throw a "column does not exist"
|
||||
// error here — caught at the boundary via isUndefinedColumnError so
|
||||
// unmigrated brains degrade to "no source scope" rather than refusing
|
||||
// every token verification.
|
||||
let oauthRows: Record<string, unknown>[];
|
||||
try {
|
||||
oauthRows = await this.sql`
|
||||
SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name,
|
||||
c.source_id, c.federated_read
|
||||
FROM oauth_tokens t
|
||||
LEFT JOIN oauth_clients c ON c.client_id = t.client_id
|
||||
WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access'
|
||||
`;
|
||||
} catch (err) {
|
||||
// v0.34.1: pre-v60 brain → source_id column missing. Pre-v61 brain →
|
||||
// federated_read column missing. Both classes degrade to legacy
|
||||
// projection so auth keeps working until the operator runs
|
||||
// apply-migrations. Probe both column names so partial-upgrade brains
|
||||
// (v60 applied but v61 didn't yet) also fall through cleanly.
|
||||
if (isUndefinedColumnError(err, 'source_id') || isUndefinedColumnError(err, 'federated_read')) {
|
||||
// Try the v60-only projection first (source_id but no federated_read).
|
||||
try {
|
||||
oauthRows = await this.sql`
|
||||
SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name, c.source_id
|
||||
FROM oauth_tokens t
|
||||
LEFT JOIN oauth_clients c ON c.client_id = t.client_id
|
||||
WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access'
|
||||
`;
|
||||
} catch (err2) {
|
||||
if (isUndefinedColumnError(err2, 'source_id')) {
|
||||
// Truly pre-v60: no source_id either. Pre-v0.34 projection.
|
||||
oauthRows = await this.sql`
|
||||
SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name
|
||||
FROM oauth_tokens t
|
||||
LEFT JOIN oauth_clients c ON c.client_id = t.client_id
|
||||
WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access'
|
||||
`;
|
||||
} else {
|
||||
throw err2;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
if (oauthRows.length > 0) {
|
||||
const row = oauthRows[0];
|
||||
@@ -422,6 +540,15 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
|
||||
if (expiresAt === undefined || expiresAt < now) {
|
||||
throw new Error('Token expired');
|
||||
}
|
||||
// v0.34.1 (#876): federated_read normalization. SELECT returns
|
||||
// either a JS array (Postgres / PGLite text[] driver mapping) or
|
||||
// undefined when the legacy projection ran (pre-v61 brain). Empty
|
||||
// array vs undefined matters: empty array = explicit no-federated-
|
||||
// read; undefined = column missing on this brain.
|
||||
const federatedRaw = row.federated_read;
|
||||
const allowedSources = Array.isArray(federatedRaw)
|
||||
? (federatedRaw as string[])
|
||||
: undefined;
|
||||
return {
|
||||
token,
|
||||
clientId: row.client_id as string,
|
||||
@@ -429,6 +556,14 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
|
||||
scopes: (row.scopes as string[]) || [],
|
||||
expiresAt,
|
||||
resource: row.resource ? new URL(row.resource as string) : undefined,
|
||||
// v0.34.1 (#861, D2): source-isolation scope from oauth_clients.
|
||||
// Undefined when the row predates v60 or when the brain itself
|
||||
// predates v60 (fell through to the legacy projection above).
|
||||
sourceId: (row.source_id as string | null) ?? undefined,
|
||||
// v0.34.1 (#876): federated read scope. sourceScopeOpts in
|
||||
// operations.ts prefers this array over scalar sourceId when set
|
||||
// and non-empty.
|
||||
allowedSources,
|
||||
} as AuthInfo;
|
||||
}
|
||||
|
||||
@@ -452,6 +587,12 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
|
||||
clientName: name,
|
||||
scopes: ['read', 'write', 'admin'],
|
||||
expiresAt: Math.floor(Date.now() / 1000) + 365 * 24 * 3600, // Legacy tokens never expire — set 1yr future
|
||||
// v0.34.1 (#861, D13): legacy bearer tokens default to 'default'
|
||||
// source — matches the pre-v0.34 effective behavior where the
|
||||
// serve-http transport fell back to GBRAIN_SOURCE/'default' for
|
||||
// any caller without explicit scope. Operators who want a
|
||||
// narrower scope for legacy tokens migrate to OAuth.
|
||||
sourceId: 'default',
|
||||
} as AuthInfo;
|
||||
}
|
||||
|
||||
@@ -570,6 +711,8 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
|
||||
grantTypes: string[],
|
||||
scopes: string,
|
||||
redirectUris: string[] = [],
|
||||
sourceId: string = 'default',
|
||||
federatedRead?: string[],
|
||||
): Promise<{ clientId: string; clientSecret: string }> {
|
||||
// v0.28: ALLOWED_SCOPES allowlist. Reject `--scopes "read flying-unicorn"`
|
||||
// at registration so meaningless scope strings can't pile up in the DB.
|
||||
@@ -582,12 +725,56 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
|
||||
const secretHash = hashToken(clientSecret);
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
|
||||
// v0.34.1 (#861 + #876): persist source_id AND federated_read so
|
||||
// verifyAccessToken can populate both AuthInfo fields. Defaults:
|
||||
// source_id = 'default' (matches v60 backfill)
|
||||
// federated_read = [source_id] when omitted (a non-federated client
|
||||
// has read scope == write scope, the v0.33 default)
|
||||
const federated = federatedRead && federatedRead.length > 0 ? federatedRead : [sourceId];
|
||||
try {
|
||||
await this.sql`
|
||||
INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris,
|
||||
grant_types, scope, client_id_issued_at,
|
||||
source_id, federated_read)
|
||||
VALUES (${clientId}, ${secretHash}, ${name},
|
||||
${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now},
|
||||
${sourceId}, ${pgArray(federated)})
|
||||
`;
|
||||
} catch (err) {
|
||||
// Pre-v60 / pre-v61 brain: column missing. Fall back through both
|
||||
// projections so registration still works until apply-migrations.
|
||||
if (isUndefinedColumnError(err, 'federated_read')) {
|
||||
// v60-only brain: source_id but no federated_read.
|
||||
try {
|
||||
await this.sql`
|
||||
INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris,
|
||||
grant_types, scope, client_id_issued_at, source_id)
|
||||
VALUES (${clientId}, ${secretHash}, ${name},
|
||||
${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now}, ${sourceId})
|
||||
`;
|
||||
} catch (err2) {
|
||||
if (isUndefinedColumnError(err2, 'source_id')) {
|
||||
await this.sql`
|
||||
INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris,
|
||||
grant_types, scope, client_id_issued_at)
|
||||
VALUES (${clientId}, ${secretHash}, ${name},
|
||||
${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now})
|
||||
`;
|
||||
} else {
|
||||
throw err2;
|
||||
}
|
||||
}
|
||||
} else if (isUndefinedColumnError(err, 'source_id')) {
|
||||
await this.sql`
|
||||
INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris,
|
||||
grant_types, scope, client_id_issued_at)
|
||||
VALUES (${clientId}, ${secretHash}, ${name},
|
||||
${pgArray(redirectUris)}, ${pgArray(grantTypes)}, ${scopes}, ${now})
|
||||
`;
|
||||
} else {
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
return { clientId, clientSecret };
|
||||
}
|
||||
|
||||
@@ -233,6 +233,36 @@ export interface AuthInfo {
|
||||
clientName?: string;
|
||||
scopes: string[];
|
||||
expiresAt?: number;
|
||||
/**
|
||||
* v0.34.1 (#861, D2): the source the calling OAuth client is scoped
|
||||
* to (write authority). Sourced from `oauth_clients.source_id` at
|
||||
* token-verification time. The HTTP transport ALSO threads this
|
||||
* value into `OperationContext.sourceId` at the same site so op
|
||||
* handlers can consume it via the canonical `ctx.sourceId` (D2
|
||||
* dual-write decision — identity surface symmetric with
|
||||
* `allowedSources` below).
|
||||
*
|
||||
* Undefined for legacy bearer tokens that predate v0.34.1 and for
|
||||
* clients that haven't been scoped yet. Migration v60 backfills
|
||||
* NULL → 'default' for pre-existing rows so this field is populated
|
||||
* on the upgrade path; brand-new public-client registrations may
|
||||
* still leave it null until an operator explicitly scopes via
|
||||
* `gbrain auth scope-client`.
|
||||
*/
|
||||
sourceId?: string;
|
||||
/**
|
||||
* v0.34.1 (#876): array of source ids this OAuth client may READ
|
||||
* from (federation). Sourced from `oauth_clients.federated_read`.
|
||||
* Independent of `sourceId` (write authority): a "WeCare L3 dept"
|
||||
* client can write to `source_id='dept-x'` while reading the union
|
||||
* of `['dept-x', 'wecare-parent', 'shared']`.
|
||||
*
|
||||
* Empty array `[]` means "no federated reads beyond `sourceId`".
|
||||
* Undefined means "the post-v60 backfill hasn't populated this row
|
||||
* yet" — engines fall back to scalar `sourceId` filtering in that
|
||||
* case (back-compat).
|
||||
*/
|
||||
allowedSources?: string[];
|
||||
}
|
||||
|
||||
export interface OperationContext {
|
||||
@@ -354,6 +384,38 @@ export interface OperationContext {
|
||||
sourceId: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.34.1 (#861, D9 — P0 leak seal): resolve the source-scope filter for a
|
||||
* read-side op handler. Returns an opts fragment ready to spread into the
|
||||
* engine call.
|
||||
*
|
||||
* Precedence:
|
||||
* 1. `ctx.auth?.allowedSources` (federated read, #876) → emits
|
||||
* `{sourceIds: [...]}`. Federated semantics subsume the scalar case.
|
||||
* 2. `ctx.sourceId` (scalar) → emits `{sourceId: '...'}`.
|
||||
* 3. Neither set → emits `{}`. Local CLI callers (and tests that don't
|
||||
* populate ctx) keep the pre-v0.34 unscoped behavior.
|
||||
*
|
||||
* Both fields default to the engine's "no filter" behavior individually,
|
||||
* so unset values are safe — the engine sees the same shape it did
|
||||
* pre-v0.34. The leak this guards against is an authenticated MCP client
|
||||
* whose ctx.sourceId IS set but whose engine call was constructed without
|
||||
* threading it (operations.ts:968/1076/1092/935/1469/1471/2241 pre-fix).
|
||||
*
|
||||
* Helper rather than inline so every read-side handler routes through the
|
||||
* same precedence ladder — drift between sites is the bug class.
|
||||
*/
|
||||
export function sourceScopeOpts(ctx: OperationContext): { sourceId?: string; sourceIds?: string[] } {
|
||||
const allowed = ctx.auth?.allowedSources;
|
||||
// Treat an empty `allowedSources: []` as "no federated read scope" — the
|
||||
// op-handler defers to scalar `ctx.sourceId` below. An attacker-controlled
|
||||
// value of `[]` MUST NOT widen scope to "all sources" by being interpreted
|
||||
// as "no filter."
|
||||
if (allowed && allowed.length > 0) return { sourceIds: allowed };
|
||||
if (ctx.sourceId) return { sourceId: ctx.sourceId };
|
||||
return {};
|
||||
}
|
||||
|
||||
export interface Operation {
|
||||
name: string;
|
||||
description: string;
|
||||
@@ -940,6 +1002,12 @@ const list_pages: Operation = {
|
||||
const sort = rawSort && (LIST_PAGES_SORT_VALUES as readonly string[]).includes(rawSort)
|
||||
? (rawSort as ListPagesSort)
|
||||
: undefined;
|
||||
// v0.34.1 (#861 — P0 leak seal): thread the auth'd client's source scope
|
||||
// into the listPages filter so an OAuth client scoped to src-A cannot
|
||||
// enumerate src-B pages. Pre-fix, ctx.sourceId / ctx.auth?.allowedSources
|
||||
// were ignored at this op handler and the engine returned every source's
|
||||
// pages indiscriminately.
|
||||
const scope = sourceScopeOpts(ctx);
|
||||
const pages = await ctx.engine.listPages({
|
||||
type: p.type as any,
|
||||
tag: p.tag as string,
|
||||
@@ -947,6 +1015,7 @@ const list_pages: Operation = {
|
||||
includeDeleted: (p.include_deleted as boolean) === true,
|
||||
updated_after: typeof p.updated_after === 'string' ? p.updated_after : undefined,
|
||||
sort,
|
||||
...scope,
|
||||
});
|
||||
return pages.map(pg => ({
|
||||
slug: pg.slug,
|
||||
@@ -973,9 +1042,13 @@ const search: Operation = {
|
||||
handler: async (ctx, p) => {
|
||||
const startedAt = Date.now();
|
||||
const queryText = p.query as string;
|
||||
// v0.34.1 (#861 — P0 leak seal): thread caller's source scope into
|
||||
// searchKeyword. Pre-fix this op silently returned cross-source hits
|
||||
// for any auth'd OAuth client.
|
||||
const raw = await ctx.engine.searchKeyword(queryText, {
|
||||
limit: (p.limit as number) || 20,
|
||||
offset: (p.offset as number) || 0,
|
||||
...sourceScopeOpts(ctx),
|
||||
});
|
||||
const results = dedupResults(raw);
|
||||
const latency_ms = Date.now() - startedAt;
|
||||
@@ -1086,10 +1159,15 @@ const query: Operation = {
|
||||
const [vec] = await embedMultimodal([
|
||||
{ kind: 'image_base64', data: imageData, mime: imageMime },
|
||||
]);
|
||||
// v0.34.1 (#861 F2 — 6th leak surface): the image path bypasses
|
||||
// hybridSearch and calls searchVector directly, so it needs its
|
||||
// own thread of the source scope. Pre-fix, this branch leaked
|
||||
// image pages across sources independent of the text path's fix.
|
||||
const results = await ctx.engine.searchVector(vec, {
|
||||
limit: (p.limit as number) || 20,
|
||||
offset: (p.offset as number) || 0,
|
||||
embeddingColumn: 'embedding_image',
|
||||
...sourceScopeOpts(ctx),
|
||||
});
|
||||
return results;
|
||||
}
|
||||
@@ -1142,6 +1220,10 @@ const query: Operation = {
|
||||
useCache: typeof p.use_cache === 'boolean' ? (p.use_cache as boolean) : undefined,
|
||||
intentWeighting: typeof p.intent_weighting === 'boolean' ? (p.intent_weighting as boolean) : undefined,
|
||||
onMeta: (m) => { capturedMeta = m; },
|
||||
// v0.34.1 (#861 — P0 leak seal): thread caller's source scope. The
|
||||
// hybridSearch internal searchOpts rebuild (hybrid.ts:223) was
|
||||
// dropping these fields pre-fix even when callers passed them.
|
||||
...sourceScopeOpts(ctx),
|
||||
});
|
||||
const latency_ms = Date.now() - startedAt;
|
||||
|
||||
@@ -1500,12 +1582,17 @@ const traverse_graph: Operation = {
|
||||
const depth = Math.max(1, Math.min(requestedDepth, TRAVERSE_DEPTH_CAP));
|
||||
const linkType = p.link_type as string | undefined;
|
||||
const direction = p.direction as 'in' | 'out' | 'both' | undefined;
|
||||
// v0.34.1 (#861 — P0 leak seal): thread caller's source scope so graph
|
||||
// walks stay within the auth'd client's accessible sources. Pre-fix,
|
||||
// traverseGraph / traversePaths happily followed edges into pages from
|
||||
// foreign sources, leaking topology + page metadata via the graph op.
|
||||
const scope = sourceScopeOpts(ctx);
|
||||
// Backward compat: when neither link_type nor direction is provided, return
|
||||
// the legacy GraphNode[] shape. Once either is set, switch to GraphPath[].
|
||||
if (linkType === undefined && direction === undefined) {
|
||||
return ctx.engine.traverseGraph(slug, depth);
|
||||
return ctx.engine.traverseGraph(slug, depth, scope);
|
||||
}
|
||||
return ctx.engine.traversePaths(slug, { depth, linkType, direction });
|
||||
return ctx.engine.traversePaths(slug, { depth, linkType, direction, ...scope });
|
||||
},
|
||||
scope: 'read',
|
||||
cliHints: { name: 'graph', positional: ['slug'] },
|
||||
@@ -2269,16 +2356,22 @@ const find_experts: Operation = {
|
||||
description: 'Include factor breakdown per result (expertise, recency, salience).',
|
||||
},
|
||||
},
|
||||
handler: async (_ctx, p) => {
|
||||
handler: async (ctx, p) => {
|
||||
const { findExperts } = await import('../commands/whoknows.ts');
|
||||
const topic = typeof p.topic === 'string' ? p.topic : '';
|
||||
if (!topic.trim()) {
|
||||
throw new OperationError('invalid_params', '`topic` is required and must be a non-empty string.');
|
||||
}
|
||||
return findExperts(_ctx.engine, {
|
||||
// v0.34.1 (#861, D3 — 5th leak surface): find_experts (whoknows) was
|
||||
// authored against v0.33 after PR #861 was drafted, so the source-scope
|
||||
// thread was missing entirely. The op calls findExperts → hybridSearch
|
||||
// internally; without the thread an auth'd src-A whoknows query would
|
||||
// surface src-B people in the rankings.
|
||||
return findExperts(ctx.engine, {
|
||||
topic,
|
||||
limit: typeof p.limit === 'number' ? p.limit : undefined,
|
||||
explain: p.explain === true,
|
||||
...sourceScopeOpts(ctx),
|
||||
});
|
||||
},
|
||||
cliHints: { name: 'whoknows', positional: ['topic'] },
|
||||
|
||||
@@ -640,8 +640,12 @@ export class PGLiteEngine implements BrainEngine {
|
||||
params.push(escaped);
|
||||
where.push(`p.slug LIKE $${params.length} ESCAPE '\\'`);
|
||||
}
|
||||
// v0.31.12: scope to a single source when requested.
|
||||
if (filters?.sourceId) {
|
||||
// v0.31.12 + v0.34.1 (#876, D9): scope to a single source OR an array
|
||||
// of sources. Array form wins (federated subsumes scalar).
|
||||
if (filters?.sourceIds && filters.sourceIds.length > 0) {
|
||||
params.push(filters.sourceIds);
|
||||
where.push(`p.source_id = ANY($${params.length}::text[])`);
|
||||
} else if (filters?.sourceId) {
|
||||
params.push(filters.sourceId);
|
||||
where.push(`p.source_id = $${params.length}`);
|
||||
}
|
||||
@@ -788,6 +792,14 @@ export class PGLiteEngine implements BrainEngine {
|
||||
params.push(opts.beforeDate);
|
||||
extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`;
|
||||
}
|
||||
// v0.34.1 (#861 — P0 leak seal): source-isolation. Array wins over scalar.
|
||||
if (opts?.sourceIds && opts.sourceIds.length > 0) {
|
||||
params.push(opts.sourceIds);
|
||||
extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`;
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
extraFilter += ` AND p.source_id = $${params.length}`;
|
||||
}
|
||||
|
||||
const { rows } = await this.db.query(
|
||||
`WITH ranked AS (
|
||||
@@ -886,6 +898,14 @@ export class PGLiteEngine implements BrainEngine {
|
||||
params.push(opts.beforeDate);
|
||||
extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`;
|
||||
}
|
||||
// v0.34.1 (#861 — P0 leak seal): source-isolation on the CJK fallback path.
|
||||
if (opts?.sourceIds && opts.sourceIds.length > 0) {
|
||||
params.push(opts.sourceIds);
|
||||
extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`;
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
extraFilter += ` AND p.source_id = $${params.length}`;
|
||||
}
|
||||
|
||||
// Bigram-frequency count: count occurrences of $qRaw in chunk_text via
|
||||
// (length(chunk) - length(replace(chunk, q, ''))) / length(q). Acts as
|
||||
@@ -1006,6 +1026,16 @@ export class PGLiteEngine implements BrainEngine {
|
||||
params.push(opts.beforeDate);
|
||||
extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`;
|
||||
}
|
||||
// v0.34.1 (#861 — P0 leak seal): source-isolation for the chunk-grain
|
||||
// anchor primitive. Layer 7 two-pass walks from these anchors so a
|
||||
// foreign-source anchor would let the walk leak into foreign neighbors.
|
||||
if (opts?.sourceIds && opts.sourceIds.length > 0) {
|
||||
params.push(opts.sourceIds);
|
||||
extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`;
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
extraFilter += ` AND p.source_id = $${params.length}`;
|
||||
}
|
||||
|
||||
// visibilityClause already declared above (v0.32.7: hoisted so CJK branch can reuse).
|
||||
|
||||
@@ -1083,6 +1113,16 @@ export class PGLiteEngine implements BrainEngine {
|
||||
params.push(opts.beforeDate);
|
||||
extraFilter += ` AND COALESCE(p.effective_date, p.updated_at, p.created_at) < $${params.length}::timestamptz`;
|
||||
}
|
||||
// v0.34.1 (#861, F2 — P0 leak seal): source-isolation in the INNER CTE
|
||||
// so HNSW candidate pool narrows before re-rank. Mirrors postgres-engine
|
||||
// placement decision (codex flagged this during plan review).
|
||||
if (opts?.sourceIds && opts.sourceIds.length > 0) {
|
||||
params.push(opts.sourceIds);
|
||||
extraFilter += ` AND p.source_id = ANY($${params.length}::text[])`;
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
extraFilter += ` AND p.source_id = $${params.length}`;
|
||||
}
|
||||
|
||||
// v0.26.5: visibility filter applied in the inner CTE so HNSW sees the
|
||||
// same candidate count it always did. See postgres-engine.ts for rationale.
|
||||
@@ -1516,13 +1556,38 @@ export class PGLiteEngine implements BrainEngine {
|
||||
return { slug: row.slug, similarity: row.sim };
|
||||
}
|
||||
|
||||
async traverseGraph(slug: string, depth: number = 5): Promise<GraphNode[]> {
|
||||
async traverseGraph(
|
||||
slug: string,
|
||||
depth: number = 5,
|
||||
opts?: { sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<GraphNode[]> {
|
||||
// v0.34.1 (#861 — P0 leak seal): source-scope filters at seed, step, and
|
||||
// aggregation subquery. Mirrors postgres-engine.traverseGraph placement.
|
||||
const params: unknown[] = [slug, depth];
|
||||
const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0;
|
||||
let seedScope = '';
|
||||
let stepScope = '';
|
||||
let aggScope = '';
|
||||
if (useSourceIds) {
|
||||
params.push(opts!.sourceIds);
|
||||
const idx = params.length;
|
||||
seedScope = `AND p.source_id = ANY($${idx}::text[])`;
|
||||
stepScope = `AND p2.source_id = ANY($${idx}::text[])`;
|
||||
aggScope = `AND p3.source_id = ANY($${idx}::text[])`;
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
const idx = params.length;
|
||||
seedScope = `AND p.source_id = $${idx}`;
|
||||
stepScope = `AND p2.source_id = $${idx}`;
|
||||
aggScope = `AND p3.source_id = $${idx}`;
|
||||
}
|
||||
|
||||
// Cycle prevention: visited array tracks page IDs already in the path.
|
||||
// Prevents exponential blowup on cyclic subgraphs (e.g., A->B->A).
|
||||
const { rows } = await this.db.query(
|
||||
`WITH RECURSIVE graph AS (
|
||||
SELECT p.id, p.slug, p.title, p.type, 0 as depth, ARRAY[p.id] as visited
|
||||
FROM pages p WHERE p.slug = $1
|
||||
FROM pages p WHERE p.slug = $1 ${seedScope}
|
||||
|
||||
UNION ALL
|
||||
|
||||
@@ -1532,6 +1597,7 @@ export class PGLiteEngine implements BrainEngine {
|
||||
JOIN pages p2 ON p2.id = l.to_page_id
|
||||
WHERE g.depth < $2
|
||||
AND NOT (p2.id = ANY(g.visited))
|
||||
${stepScope}
|
||||
)
|
||||
SELECT DISTINCT g.slug, g.title, g.type, g.depth,
|
||||
coalesce(
|
||||
@@ -1543,12 +1609,12 @@ export class PGLiteEngine implements BrainEngine {
|
||||
(SELECT jsonb_agg(DISTINCT jsonb_build_object('to_slug', p3.slug, 'link_type', l2.link_type))
|
||||
FROM links l2
|
||||
JOIN pages p3 ON p3.id = l2.to_page_id
|
||||
WHERE l2.from_page_id = g.id),
|
||||
WHERE l2.from_page_id = g.id ${aggScope}),
|
||||
'[]'::jsonb
|
||||
) as links
|
||||
FROM graph g
|
||||
ORDER BY g.depth, g.slug`,
|
||||
[slug, depth]
|
||||
params
|
||||
);
|
||||
|
||||
return (rows as Record<string, unknown>[]).map(r => ({
|
||||
@@ -1562,7 +1628,7 @@ export class PGLiteEngine implements BrainEngine {
|
||||
|
||||
async traversePaths(
|
||||
slug: string,
|
||||
opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both' },
|
||||
opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<GraphPath[]> {
|
||||
const depth = opts?.depth ?? 5;
|
||||
const direction = opts?.direction ?? 'out';
|
||||
@@ -1571,12 +1637,36 @@ export class PGLiteEngine implements BrainEngine {
|
||||
const params: unknown[] = [slug, depth];
|
||||
if (linkType !== null) params.push(linkType);
|
||||
|
||||
// v0.34.1 (#861 — P0 leak seal): source-scope filters at seed + step +
|
||||
// final SELECT joins (for the 'both' branch's pf + pt). Mirrors
|
||||
// postgres-engine.traversePaths placement.
|
||||
const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0;
|
||||
let seedScope = '';
|
||||
let stepScope = '';
|
||||
let pfScope = '';
|
||||
let ptScope = '';
|
||||
if (useSourceIds) {
|
||||
params.push(opts!.sourceIds);
|
||||
const idx = params.length;
|
||||
seedScope = `AND p.source_id = ANY($${idx}::text[])`;
|
||||
stepScope = `AND p2.source_id = ANY($${idx}::text[])`;
|
||||
pfScope = `AND pf.source_id = ANY($${idx}::text[])`;
|
||||
ptScope = `AND pt.source_id = ANY($${idx}::text[])`;
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
const idx = params.length;
|
||||
seedScope = `AND p.source_id = $${idx}`;
|
||||
stepScope = `AND p2.source_id = $${idx}`;
|
||||
pfScope = `AND pf.source_id = $${idx}`;
|
||||
ptScope = `AND pt.source_id = $${idx}`;
|
||||
}
|
||||
|
||||
let sql: string;
|
||||
if (direction === 'out') {
|
||||
sql = `
|
||||
WITH RECURSIVE walk AS (
|
||||
SELECT p.id, p.slug, 0::int AS depth, ARRAY[p.id] AS visited
|
||||
FROM pages p WHERE p.slug = $1
|
||||
FROM pages p WHERE p.slug = $1 ${seedScope}
|
||||
UNION ALL
|
||||
SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id
|
||||
FROM walk w
|
||||
@@ -1585,6 +1675,7 @@ export class PGLiteEngine implements BrainEngine {
|
||||
WHERE w.depth < $2
|
||||
AND NOT (p2.id = ANY(w.visited))
|
||||
${linkTypeWhere}
|
||||
${stepScope}
|
||||
)
|
||||
SELECT w.slug AS from_slug, p2.slug AS to_slug,
|
||||
l.link_type, l.context, w.depth + 1 AS depth
|
||||
@@ -1593,13 +1684,14 @@ export class PGLiteEngine implements BrainEngine {
|
||||
JOIN pages p2 ON p2.id = l.to_page_id
|
||||
WHERE w.depth < $2
|
||||
${linkTypeWhere}
|
||||
${stepScope}
|
||||
ORDER BY depth, from_slug, to_slug
|
||||
`;
|
||||
} else if (direction === 'in') {
|
||||
sql = `
|
||||
WITH RECURSIVE walk AS (
|
||||
SELECT p.id, p.slug, 0::int AS depth, ARRAY[p.id] AS visited
|
||||
FROM pages p WHERE p.slug = $1
|
||||
FROM pages p WHERE p.slug = $1 ${seedScope}
|
||||
UNION ALL
|
||||
SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id
|
||||
FROM walk w
|
||||
@@ -1608,6 +1700,7 @@ export class PGLiteEngine implements BrainEngine {
|
||||
WHERE w.depth < $2
|
||||
AND NOT (p2.id = ANY(w.visited))
|
||||
${linkTypeWhere}
|
||||
${stepScope}
|
||||
)
|
||||
SELECT p2.slug AS from_slug, w.slug AS to_slug,
|
||||
l.link_type, l.context, w.depth + 1 AS depth
|
||||
@@ -1616,6 +1709,7 @@ export class PGLiteEngine implements BrainEngine {
|
||||
JOIN pages p2 ON p2.id = l.from_page_id
|
||||
WHERE w.depth < $2
|
||||
${linkTypeWhere}
|
||||
${stepScope}
|
||||
ORDER BY depth, from_slug, to_slug
|
||||
`;
|
||||
} else {
|
||||
@@ -1624,7 +1718,7 @@ export class PGLiteEngine implements BrainEngine {
|
||||
sql = `
|
||||
WITH RECURSIVE walk AS (
|
||||
SELECT p.id, 0::int AS depth, ARRAY[p.id] AS visited
|
||||
FROM pages p WHERE p.slug = $1
|
||||
FROM pages p WHERE p.slug = $1 ${seedScope}
|
||||
UNION ALL
|
||||
SELECT p2.id, w.depth + 1, w.visited || p2.id
|
||||
FROM walk w
|
||||
@@ -1633,6 +1727,7 @@ export class PGLiteEngine implements BrainEngine {
|
||||
WHERE w.depth < $2
|
||||
AND NOT (p2.id = ANY(w.visited))
|
||||
${linkTypeWhere}
|
||||
${stepScope}
|
||||
)
|
||||
SELECT pf.slug AS from_slug, pt.slug AS to_slug,
|
||||
l.link_type, l.context, w.depth + 1 AS depth
|
||||
@@ -1642,6 +1737,8 @@ export class PGLiteEngine implements BrainEngine {
|
||||
JOIN pages pt ON pt.id = l.to_page_id
|
||||
WHERE w.depth < $2
|
||||
${linkTypeWhere}
|
||||
${pfScope}
|
||||
${ptScope}
|
||||
ORDER BY depth, from_slug, to_slug
|
||||
`;
|
||||
}
|
||||
|
||||
@@ -594,8 +594,19 @@ CREATE TABLE IF NOT EXISTS oauth_clients (
|
||||
client_secret_expires_at BIGINT,
|
||||
token_ttl INTEGER,
|
||||
deleted_at TIMESTAMPTZ,
|
||||
source_id TEXT REFERENCES sources(id) ON DELETE RESTRICT,
|
||||
federated_read TEXT[] NOT NULL DEFAULT '{}',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
-- v0.34.1 (#861, D13 + #876): source_id is the OAuth client's write-source
|
||||
-- scope; federated_read is its read-source array (a federated client can
|
||||
-- read sources beyond its source_id). Migration v60 adds source_id;
|
||||
-- v61-v65 add federated_read + GIN index + flip FK to RESTRICT. Fresh
|
||||
-- installs land in the post-migration shape via the inline columns above.
|
||||
CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id
|
||||
ON oauth_clients(source_id) WHERE source_id IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read
|
||||
ON oauth_clients USING GIN (federated_read);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS oauth_tokens (
|
||||
token_hash TEXT PRIMARY KEY,
|
||||
|
||||
@@ -685,8 +685,12 @@ export class PostgresEngine implements BrainEngine {
|
||||
const slugCondition = slugPrefix
|
||||
? sql`AND p.slug LIKE ${slugPrefix.replace(/[\\%_]/g, (c) => '\\' + c) + '%'} ESCAPE '\\'`
|
||||
: sql``;
|
||||
// v0.31.12: scope to a single source when requested.
|
||||
const sourceCondition = filters?.sourceId
|
||||
// v0.31.12 + v0.34.1 (#876, D9): scope to a single source OR an array
|
||||
// of sources. When BOTH are set, the array wins (federated semantics
|
||||
// subsume the scalar case). When neither is set, no filter applies.
|
||||
const sourceCondition = filters?.sourceIds && filters.sourceIds.length > 0
|
||||
? sql`AND p.source_id = ANY(${filters.sourceIds}::text[])`
|
||||
: filters?.sourceId
|
||||
? sql`AND p.source_id = ${filters.sourceId}`
|
||||
: sql``;
|
||||
// v0.26.5: hide soft-deleted by default; opt in via filters.includeDeleted.
|
||||
@@ -828,6 +832,22 @@ export class PostgresEngine implements BrainEngine {
|
||||
params.push(opts.beforeDate);
|
||||
beforeDateClause = `AND COALESCE(p.updated_at, p.created_at) < $${params.length}::timestamptz`;
|
||||
}
|
||||
// v0.34.1 (#861 — P0 leak seal): source-isolation filter. When the
|
||||
// caller's auth scope is set, narrow the inner CTE candidate set so
|
||||
// an authenticated MCP client cannot see foreign-source pages via
|
||||
// keyword search. Array form wins over scalar (federated subsumes
|
||||
// single-source). Index-backed by idx_pages_source_id; the filter is
|
||||
// pushed to the INNER CTE specifically so HNSW-style downstream
|
||||
// ranking sees a narrowed candidate set rather than re-ranking a
|
||||
// cross-source pool.
|
||||
let sourceClause = '';
|
||||
if (opts?.sourceIds && opts.sourceIds.length > 0) {
|
||||
params.push(opts.sourceIds);
|
||||
sourceClause = `AND p.source_id = ANY($${params.length}::text[])`;
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
sourceClause = `AND p.source_id = $${params.length}`;
|
||||
}
|
||||
params.push(innerLimit);
|
||||
const innerLimitParam = `$${params.length}`;
|
||||
params.push(limit);
|
||||
@@ -859,6 +879,7 @@ export class PostgresEngine implements BrainEngine {
|
||||
${symbolKindClause}
|
||||
${afterDateClause}
|
||||
${beforeDateClause}
|
||||
${sourceClause}
|
||||
${hardExcludeClause}
|
||||
${visibilityClause}
|
||||
-- v0.27.1: hide image rows from text-keyword search so OCR text
|
||||
@@ -962,6 +983,17 @@ export class PostgresEngine implements BrainEngine {
|
||||
params.push(opts.beforeDate);
|
||||
beforeDateClause = `AND COALESCE(p.updated_at, p.created_at) < $${params.length}::timestamptz`;
|
||||
}
|
||||
// v0.34.1 (#861 — P0 leak seal): source-isolation. Anchor primitive
|
||||
// for two-pass retrieval, so cross-source anchors would let the walk
|
||||
// discover foreign-source neighbors. Filter at chunk-rank time.
|
||||
let sourceClause = '';
|
||||
if (opts?.sourceIds && opts.sourceIds.length > 0) {
|
||||
params.push(opts.sourceIds);
|
||||
sourceClause = `AND p.source_id = ANY($${params.length}::text[])`;
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
sourceClause = `AND p.source_id = $${params.length}`;
|
||||
}
|
||||
params.push(limit);
|
||||
const limitParam = `$${params.length}`;
|
||||
params.push(offset);
|
||||
@@ -988,6 +1020,7 @@ export class PostgresEngine implements BrainEngine {
|
||||
${symbolKindClause}
|
||||
${afterDateClause}
|
||||
${beforeDateClause}
|
||||
${sourceClause}
|
||||
${hardExcludeClause}
|
||||
${visibilityClause}
|
||||
ORDER BY score DESC
|
||||
@@ -1071,6 +1104,19 @@ export class PostgresEngine implements BrainEngine {
|
||||
params.push(opts.beforeDate);
|
||||
beforeDateClause = `AND COALESCE(p.updated_at, p.created_at) < $${params.length}::timestamptz`;
|
||||
}
|
||||
// v0.34.1 (#861, F2 — P0 leak seal): source-isolation in the INNER CTE
|
||||
// specifically. Pushing the filter inside narrows the HNSW candidate set
|
||||
// before re-rank; pushing it to the outer SELECT would force HNSW to
|
||||
// over-fetch then post-filter, wasting candidate slots. Codex flagged
|
||||
// this placement during plan review. Array form wins over scalar.
|
||||
let sourceClause = '';
|
||||
if (opts?.sourceIds && opts.sourceIds.length > 0) {
|
||||
params.push(opts.sourceIds);
|
||||
sourceClause = `AND p.source_id = ANY($${params.length}::text[])`;
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
sourceClause = `AND p.source_id = $${params.length}`;
|
||||
}
|
||||
params.push(innerLimit);
|
||||
const innerLimitParam = `$${params.length}`;
|
||||
params.push(limit);
|
||||
@@ -1106,6 +1152,7 @@ export class PostgresEngine implements BrainEngine {
|
||||
${symbolKindClause}
|
||||
${afterDateClause}
|
||||
${beforeDateClause}
|
||||
${sourceClause}
|
||||
${hardExcludeClause}
|
||||
${visibilityClause}
|
||||
ORDER BY cc.${col} <=> $1::vector
|
||||
@@ -1516,13 +1563,40 @@ export class PostgresEngine implements BrainEngine {
|
||||
return { slug: row.slug, similarity: row.sim };
|
||||
}
|
||||
|
||||
async traverseGraph(slug: string, depth: number = 5): Promise<GraphNode[]> {
|
||||
async traverseGraph(
|
||||
slug: string,
|
||||
depth: number = 5,
|
||||
opts?: { sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<GraphNode[]> {
|
||||
const sql = this.sql;
|
||||
// v0.34.1 (#861 — P0 leak seal): scope visited nodes to the caller's
|
||||
// source(s). Without this, the walk follows edges into pages from
|
||||
// foreign sources, leaking topology + page metadata. The filter
|
||||
// applies at BOTH the seed (root must be in scope) AND the recursive
|
||||
// step (every visited neighbor must be in scope). The aggregation
|
||||
// subquery also filters so the per-node `links` array only includes
|
||||
// edges to in-scope pages.
|
||||
const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0;
|
||||
const seedScope = useSourceIds
|
||||
? sql`AND p.source_id = ANY(${opts!.sourceIds!}::text[])`
|
||||
: opts?.sourceId
|
||||
? sql`AND p.source_id = ${opts.sourceId}`
|
||||
: sql``;
|
||||
const stepScope = useSourceIds
|
||||
? sql`AND p2.source_id = ANY(${opts!.sourceIds!}::text[])`
|
||||
: opts?.sourceId
|
||||
? sql`AND p2.source_id = ${opts.sourceId}`
|
||||
: sql``;
|
||||
const aggScope = useSourceIds
|
||||
? sql`AND p3.source_id = ANY(${opts!.sourceIds!}::text[])`
|
||||
: opts?.sourceId
|
||||
? sql`AND p3.source_id = ${opts.sourceId}`
|
||||
: sql``;
|
||||
// Cycle prevention: visited array tracks page IDs already in the path.
|
||||
const rows = await sql`
|
||||
WITH RECURSIVE graph AS (
|
||||
SELECT p.id, p.slug, p.title, p.type, 0 as depth, ARRAY[p.id] as visited
|
||||
FROM pages p WHERE p.slug = ${slug}
|
||||
FROM pages p WHERE p.slug = ${slug} ${seedScope}
|
||||
|
||||
UNION ALL
|
||||
|
||||
@@ -1532,6 +1606,7 @@ export class PostgresEngine implements BrainEngine {
|
||||
JOIN pages p2 ON p2.id = l.to_page_id
|
||||
WHERE g.depth < ${depth}
|
||||
AND NOT (p2.id = ANY(g.visited))
|
||||
${stepScope}
|
||||
)
|
||||
SELECT DISTINCT g.slug, g.title, g.type, g.depth,
|
||||
coalesce(
|
||||
@@ -1545,7 +1620,7 @@ export class PostgresEngine implements BrainEngine {
|
||||
(SELECT jsonb_agg(DISTINCT jsonb_build_object('to_slug', p3.slug, 'link_type', l2.link_type))
|
||||
FROM links l2
|
||||
JOIN pages p3 ON p3.id = l2.to_page_id
|
||||
WHERE l2.from_page_id = g.id),
|
||||
WHERE l2.from_page_id = g.id ${aggScope}),
|
||||
'[]'::jsonb
|
||||
) as links
|
||||
FROM graph g
|
||||
@@ -1563,20 +1638,47 @@ export class PostgresEngine implements BrainEngine {
|
||||
|
||||
async traversePaths(
|
||||
slug: string,
|
||||
opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both' },
|
||||
opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<GraphPath[]> {
|
||||
const sql = this.sql;
|
||||
const depth = opts?.depth ?? 5;
|
||||
const direction = opts?.direction ?? 'out';
|
||||
const linkType = opts?.linkType ?? null;
|
||||
const linkTypeMatches = linkType !== null;
|
||||
// v0.34.1 (#861 — P0 leak seal): source-scope filter fragments. Applied
|
||||
// at seed (root must be in scope) AND at every recursive step (neighbor
|
||||
// must be in scope) AND in the SELECT join (final edges respect scope).
|
||||
// The 'both' branch needs filters on BOTH endpoint joins.
|
||||
const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0;
|
||||
const seedScope = useSourceIds
|
||||
? sql`AND p.source_id = ANY(${opts!.sourceIds!}::text[])`
|
||||
: opts?.sourceId
|
||||
? sql`AND p.source_id = ${opts.sourceId}`
|
||||
: sql``;
|
||||
const stepScope = useSourceIds
|
||||
? sql`AND p2.source_id = ANY(${opts!.sourceIds!}::text[])`
|
||||
: opts?.sourceId
|
||||
? sql`AND p2.source_id = ${opts.sourceId}`
|
||||
: sql``;
|
||||
// For the 'both' direction's final SELECT, both endpoint joins (pf, pt)
|
||||
// get scope filters so edges crossing into a foreign source are dropped.
|
||||
const pfScope = useSourceIds
|
||||
? sql`AND pf.source_id = ANY(${opts!.sourceIds!}::text[])`
|
||||
: opts?.sourceId
|
||||
? sql`AND pf.source_id = ${opts.sourceId}`
|
||||
: sql``;
|
||||
const ptScope = useSourceIds
|
||||
? sql`AND pt.source_id = ANY(${opts!.sourceIds!}::text[])`
|
||||
: opts?.sourceId
|
||||
? sql`AND pt.source_id = ${opts.sourceId}`
|
||||
: sql``;
|
||||
|
||||
let rows;
|
||||
if (direction === 'out') {
|
||||
rows = await sql`
|
||||
WITH RECURSIVE walk AS (
|
||||
SELECT p.id, p.slug, 0::int as depth, ARRAY[p.id] as visited
|
||||
FROM pages p WHERE p.slug = ${slug}
|
||||
FROM pages p WHERE p.slug = ${slug} ${seedScope}
|
||||
UNION ALL
|
||||
SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id
|
||||
FROM walk w
|
||||
@@ -1585,6 +1687,7 @@ export class PostgresEngine implements BrainEngine {
|
||||
WHERE w.depth < ${depth}
|
||||
AND NOT (p2.id = ANY(w.visited))
|
||||
AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''})
|
||||
${stepScope}
|
||||
)
|
||||
SELECT w.slug as from_slug, p2.slug as to_slug,
|
||||
l.link_type, l.context, w.depth + 1 as depth
|
||||
@@ -1593,13 +1696,14 @@ export class PostgresEngine implements BrainEngine {
|
||||
JOIN pages p2 ON p2.id = l.to_page_id
|
||||
WHERE w.depth < ${depth}
|
||||
AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''})
|
||||
${stepScope}
|
||||
ORDER BY depth, from_slug, to_slug
|
||||
`;
|
||||
} else if (direction === 'in') {
|
||||
rows = await sql`
|
||||
WITH RECURSIVE walk AS (
|
||||
SELECT p.id, p.slug, 0::int as depth, ARRAY[p.id] as visited
|
||||
FROM pages p WHERE p.slug = ${slug}
|
||||
FROM pages p WHERE p.slug = ${slug} ${seedScope}
|
||||
UNION ALL
|
||||
SELECT p2.id, p2.slug, w.depth + 1, w.visited || p2.id
|
||||
FROM walk w
|
||||
@@ -1608,6 +1712,7 @@ export class PostgresEngine implements BrainEngine {
|
||||
WHERE w.depth < ${depth}
|
||||
AND NOT (p2.id = ANY(w.visited))
|
||||
AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''})
|
||||
${stepScope}
|
||||
)
|
||||
SELECT p2.slug as from_slug, w.slug as to_slug,
|
||||
l.link_type, l.context, w.depth + 1 as depth
|
||||
@@ -1616,13 +1721,14 @@ export class PostgresEngine implements BrainEngine {
|
||||
JOIN pages p2 ON p2.id = l.from_page_id
|
||||
WHERE w.depth < ${depth}
|
||||
AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''})
|
||||
${stepScope}
|
||||
ORDER BY depth, from_slug, to_slug
|
||||
`;
|
||||
} else {
|
||||
rows = await sql`
|
||||
WITH RECURSIVE walk AS (
|
||||
SELECT p.id, 0::int as depth, ARRAY[p.id] as visited
|
||||
FROM pages p WHERE p.slug = ${slug}
|
||||
FROM pages p WHERE p.slug = ${slug} ${seedScope}
|
||||
UNION ALL
|
||||
SELECT p2.id, w.depth + 1, w.visited || p2.id
|
||||
FROM walk w
|
||||
@@ -1631,6 +1737,7 @@ export class PostgresEngine implements BrainEngine {
|
||||
WHERE w.depth < ${depth}
|
||||
AND NOT (p2.id = ANY(w.visited))
|
||||
AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''})
|
||||
${stepScope}
|
||||
)
|
||||
SELECT pf.slug as from_slug, pt.slug as to_slug,
|
||||
l.link_type, l.context, w.depth + 1 as depth
|
||||
@@ -1640,6 +1747,8 @@ export class PostgresEngine implements BrainEngine {
|
||||
JOIN pages pt ON pt.id = l.to_page_id
|
||||
WHERE w.depth < ${depth}
|
||||
AND (${!linkTypeMatches} OR l.link_type = ${linkType ?? ''})
|
||||
${pfScope}
|
||||
${ptScope}
|
||||
ORDER BY depth, from_slug, to_slug
|
||||
`;
|
||||
}
|
||||
|
||||
@@ -428,8 +428,17 @@ CREATE TABLE IF NOT EXISTS oauth_clients (
|
||||
client_secret_expires_at BIGINT,
|
||||
token_ttl INTEGER,
|
||||
deleted_at TIMESTAMPTZ,
|
||||
source_id TEXT REFERENCES sources(id) ON DELETE RESTRICT,
|
||||
federated_read TEXT[] NOT NULL DEFAULT '{}',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
-- v0.34.1 (#861, D13 + #876): source_id is the write-source scope;
|
||||
-- federated_read is the read-source array. Migrations v60-v65 land both
|
||||
-- columns on upgrade; fresh installs include them inline above.
|
||||
CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id
|
||||
ON oauth_clients(source_id) WHERE source_id IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read
|
||||
ON oauth_clients USING GIN (federated_read);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS oauth_tokens (
|
||||
token_hash TEXT PRIMARY KEY,
|
||||
|
||||
@@ -280,6 +280,16 @@ export async function hybridSearch(
|
||||
// PR #618 callers compiling while the new names are the public surface.
|
||||
afterDate: opts?.since ?? opts?.afterDate,
|
||||
beforeDate: opts?.until ?? opts?.beforeDate,
|
||||
// v0.34.1 (#861, D9 — P0 leak seal): thread source-scoping through so the
|
||||
// inner engine.searchKeyword / engine.searchVector calls apply the
|
||||
// WHERE source_id filter at SQL level. Pre-fix, this explicit pick
|
||||
// silently DROPPED these fields and every authenticated MCP client
|
||||
// could see pages from foreign sources via the hybrid search hot
|
||||
// path. New SearchOpts fields scoped to source isolation MUST be
|
||||
// added here too; the rebuild shape is intentional (HNSW inner-CTE
|
||||
// ordering means we can't lazy-spread the full opts).
|
||||
sourceId: opts?.sourceId,
|
||||
sourceIds: opts?.sourceIds,
|
||||
};
|
||||
// Track what actually ran for the optional onMeta callback (v0.25.0).
|
||||
// Caller leaves onMeta undefined → these flags are computed but never
|
||||
|
||||
@@ -196,6 +196,15 @@ export interface PageFilters {
|
||||
* operations to a single source.
|
||||
*/
|
||||
sourceId?: string;
|
||||
/**
|
||||
* v0.34.1 (#876, D9): filter to ANY of these sources (federated read).
|
||||
* Engine applies `WHERE p.source_id = ANY($N::text[])` when array is set.
|
||||
* Caller precedence: if BOTH `sourceId` and `sourceIds` are set, the array
|
||||
* wins (the federated semantics subsume the single-source case via an
|
||||
* array of length 1). When neither is set, no filter applies — the
|
||||
* pre-v0.34 unscoped behavior is preserved for local CLI callers.
|
||||
*/
|
||||
sourceIds?: string[];
|
||||
}
|
||||
|
||||
/** v0.26.5 — opts for getPage / softDeletePage / restorePage. */
|
||||
@@ -457,6 +466,18 @@ export interface SearchOpts {
|
||||
* undefined to search all sources.
|
||||
*/
|
||||
sourceId?: string;
|
||||
/**
|
||||
* v0.34.1 (#876, D9): filter to ANY of these sources (federated read).
|
||||
* Engine applies `WHERE p.source_id = ANY($N::text[])` when the array
|
||||
* is set. Caller precedence: if BOTH `sourceId` and `sourceIds` are
|
||||
* provided, the array wins (the federated semantics subsume the
|
||||
* single-source case). When neither is set, no filter applies — the
|
||||
* pre-v0.34 unscoped behavior is preserved for local CLI callers.
|
||||
*
|
||||
* The op-handler layer resolves: `ctx.auth?.allowedSources` (federated
|
||||
* client) → `sourceIds`; otherwise `ctx.sourceId` (scalar) → `sourceId`.
|
||||
*/
|
||||
sourceIds?: string[];
|
||||
/**
|
||||
* v0.27.1: target column for vector search. 'embedding' (default) hits
|
||||
* the brain's primary text-embedding column. 'embedding_image' targets
|
||||
|
||||
@@ -66,6 +66,14 @@ interface AuthResult {
|
||||
tokenName?: string;
|
||||
/** v0.28: per-token allow-list for takes.holder. Default ['world'] when permissions row absent. */
|
||||
takesHoldersAllowList?: string[];
|
||||
/**
|
||||
* v0.34.1 (#861, D13): source-isolation scope for the auth'd request.
|
||||
* Legacy bearer tokens here default to 'default' to match the v0.33
|
||||
* effective behavior (the now-removed serve-http.ts fallback chain).
|
||||
* Operators migrate to the full OAuth transport (gbrain serve --http)
|
||||
* for narrower scoping.
|
||||
*/
|
||||
sourceId?: string;
|
||||
}
|
||||
|
||||
/** Read up to `cap` bytes off req.body. Returns null if cap exceeded. */
|
||||
@@ -180,6 +188,12 @@ export async function startHttpTransport(opts: HttpTransportOptions) {
|
||||
tokenId: rowId,
|
||||
tokenName: rowName,
|
||||
takesHoldersAllowList: allowList,
|
||||
// v0.34.1 (#861, D13): legacy bearer tokens default to 'default'
|
||||
// source. Preserves the pre-v0.34 effective behavior of the
|
||||
// serve-http fallback chain that was removed for OAuth clients
|
||||
// (migration v60 backfills oauth_clients.source_id). This path
|
||||
// is for the older v0.22.7 access_tokens transport.
|
||||
sourceId: 'default',
|
||||
};
|
||||
} catch {
|
||||
return { ok: false };
|
||||
@@ -328,9 +342,12 @@ export async function startHttpTransport(opts: HttpTransportOptions) {
|
||||
const args: Record<string, unknown> = params?.arguments ?? {};
|
||||
// v0.28: thread per-token takes-holder allow-list so takes_list /
|
||||
// takes_search / query (when it returns takes) can server-side filter.
|
||||
// v0.34.1 (#861): thread source-isolation scope. Legacy access_tokens
|
||||
// path defaults to 'default' per AuthResult.sourceId above.
|
||||
const result = await dispatchToolCall(engine, toolName, args, {
|
||||
remote: true,
|
||||
takesHoldersAllowList: auth.takesHoldersAllowList,
|
||||
sourceId: auth.sourceId,
|
||||
});
|
||||
const status = result.isError ? 'error' : 'success';
|
||||
logRequest(auth.tokenName!, `tools/call:${toolName}`, status, Date.now() - startedMs);
|
||||
|
||||
@@ -62,8 +62,15 @@ export async function startMcpServer(engine: BrainEngine) {
|
||||
.catch(() => {})
|
||||
.finally(() => process.exit(code));
|
||||
};
|
||||
// v0.34.1 (#870): when MCP_STDIO=1, the wrapping gateway (OpenClaw's
|
||||
// bundle-mcp layer, others) often pipes the JSON-RPC handshake then
|
||||
// closes its stdin half. Treating that as a permanent disconnect kills
|
||||
// the server before the first tool call arrives. Signal handlers and
|
||||
// transport.onclose still cover the legitimate shutdown paths.
|
||||
if (process.env.MCP_STDIO !== '1') {
|
||||
process.stdin.on('end', () => shutdown('stdin end'));
|
||||
process.stdin.on('close', () => shutdown('stdin close'));
|
||||
}
|
||||
// @ts-ignore — SDK exposes onclose on transport
|
||||
transport.onclose = () => shutdown('transport close');
|
||||
process.on('SIGTERM', () => shutdown('SIGTERM'));
|
||||
|
||||
@@ -424,8 +424,17 @@ CREATE TABLE IF NOT EXISTS oauth_clients (
|
||||
client_secret_expires_at BIGINT,
|
||||
token_ttl INTEGER,
|
||||
deleted_at TIMESTAMPTZ,
|
||||
source_id TEXT REFERENCES sources(id) ON DELETE RESTRICT,
|
||||
federated_read TEXT[] NOT NULL DEFAULT '{}',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
-- v0.34.1 (#861, D13 + #876): source_id is the write-source scope;
|
||||
-- federated_read is the read-source array. Migrations v60-v65 land both
|
||||
-- columns on upgrade; fresh installs include them inline above.
|
||||
CREATE INDEX IF NOT EXISTS idx_oauth_clients_source_id
|
||||
ON oauth_clients(source_id) WHERE source_id IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_oauth_clients_federated_read
|
||||
ON oauth_clients USING GIN (federated_read);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS oauth_tokens (
|
||||
token_hash TEXT PRIMARY KEY,
|
||||
|
||||
@@ -32,35 +32,57 @@ async function runCli(args: string[]): Promise<{
|
||||
stderr: string;
|
||||
exit: number;
|
||||
}> {
|
||||
// v0.34.1 (#876): set GBRAIN_HOME to a fresh tempdir so the test isn't
|
||||
// sensitive to the developer's local ~/.gbrain/config.json. Pre-fix this
|
||||
// test would silently inherit the user's database_url and try to connect
|
||||
// to a real Postgres + run migrations, which both extends runtime past
|
||||
// the default test timeout and produces different exit semantics
|
||||
// depending on the local DB state. With GBRAIN_HOME pointing at an
|
||||
// empty dir, the CLI hits "No brain configured" and exits 1 immediately
|
||||
// — which is what this test was originally written to cover.
|
||||
const tmpHome = await import('os').then(os =>
|
||||
import('fs').then(fs => fs.mkdtempSync(`${os.tmpdir()}/gbrain-test-`)),
|
||||
);
|
||||
try {
|
||||
const proc = Bun.spawn(
|
||||
['bun', 'run', 'src/cli.ts', 'book-mirror', ...args],
|
||||
{
|
||||
cwd: REPO_ROOT,
|
||||
stdout: 'pipe',
|
||||
stderr: 'pipe',
|
||||
env: { ...process.env, DATABASE_URL: '' },
|
||||
env: { ...process.env, DATABASE_URL: '', GBRAIN_HOME: tmpHome },
|
||||
},
|
||||
);
|
||||
const stdout = await new Response(proc.stdout).text();
|
||||
const stderr = await new Response(proc.stderr).text();
|
||||
const exit = await proc.exited;
|
||||
return { stdout, stderr, exit };
|
||||
} finally {
|
||||
const fs = await import('fs');
|
||||
fs.rmSync(tmpHome, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
describe('gbrain book-mirror — CLI registration', () => {
|
||||
// v0.34.1 (#876): migrations v60-v65 added oauth_clients.source_id +
|
||||
// federated_read FK plumbing. The cold-spawned subprocess's initSchema
|
||||
// chain now takes ~1s longer end-to-end; bump these tests' timeout to
|
||||
// 30s so the test-harness budget covers Bun cold-start + PGLite init +
|
||||
// migration apply + Postgres-retry exhaustion. The CLI itself still
|
||||
// exits in 7-10s on a misconfigured DB.
|
||||
it('book-mirror is in CLI_ONLY (does not get "Unknown command")', async () => {
|
||||
const { stderr } = await runCli([]);
|
||||
// Without DB, the command will fail — but on the connect path,
|
||||
// not as "Unknown command". This proves dispatch reached
|
||||
// handleCliOnly's switch statement.
|
||||
expect(stderr).not.toContain('Unknown command');
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
it('without DB, never reaches queue submission', async () => {
|
||||
const { stderr, exit } = await runCli(['--slug', 'noop']);
|
||||
expect(exit).not.toBe(0);
|
||||
expect(stderr).not.toContain('submitted:');
|
||||
});
|
||||
}, 30000);
|
||||
});
|
||||
|
||||
describe('gbrain book-mirror — source file invariants', () => {
|
||||
|
||||
267
test/e2e/source-isolation-pglite.test.ts
Normal file
267
test/e2e/source-isolation-pglite.test.ts
Normal file
@@ -0,0 +1,267 @@
|
||||
/**
|
||||
* v0.34.1 (#861 — P0 source-isolation leak seal) E2E regression.
|
||||
*
|
||||
* The wave's IRON RULE: every read-side op must filter by source_id when
|
||||
* the caller supplies it via SearchOpts/PageFilters/TraverseOpts. Pre-fix,
|
||||
* authenticated MCP clients scoped to source-A could enumerate source-B
|
||||
* pages via search / query / list_pages / traverse_graph / find_experts.
|
||||
*
|
||||
* Runs against PGLite in-memory so the test exercises both engines' parity
|
||||
* surface (the schema-drift E2E covers Postgres independently) without
|
||||
* needing a Docker container.
|
||||
*
|
||||
* Coverage: searchKeyword, searchVector (synthetic embedding), listPages,
|
||||
* traverseGraph, traversePaths. find_experts is exercised via the same
|
||||
* hybridSearch path the op handler calls.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { resetPgliteState } from '../helpers/reset-pglite.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetPgliteState(engine);
|
||||
// Two sources so we can prove the filter excludes cross-source rows.
|
||||
// 'default' source is created by initSchema's seed; we add src-b.
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO sources (id, name, config) VALUES ('src-b', 'src-b', '{}'::jsonb) ON CONFLICT DO NOTHING`,
|
||||
);
|
||||
// Seed one person page in each source. Same slug intentionally —
|
||||
// proves the composite (source_id, slug) key is honored, not just slug.
|
||||
// upsertChunks is needed because searchKeyword scans content_chunks, not
|
||||
// pages.compiled_truth directly. Each page gets one chunk that mirrors
|
||||
// its compiled_truth so search-by-keyword has something to find.
|
||||
await engine.putPage('people/alice', {
|
||||
type: 'person',
|
||||
title: 'Alice Source-A',
|
||||
compiled_truth: 'Alice works on widgets in source A. Important context here.',
|
||||
timeline: '',
|
||||
frontmatter: {},
|
||||
}, { sourceId: 'default' });
|
||||
await engine.upsertChunks('people/alice', [{
|
||||
chunk_index: 0,
|
||||
chunk_text: 'Alice works on widgets in source A. Important context here.',
|
||||
chunk_source: 'compiled_truth',
|
||||
token_count: 12,
|
||||
}], { sourceId: 'default' });
|
||||
|
||||
await engine.putPage('people/alice', {
|
||||
type: 'person',
|
||||
title: 'Alice Source-B',
|
||||
compiled_truth: 'Alice works on gadgets in source B. Important context here.',
|
||||
timeline: '',
|
||||
frontmatter: {},
|
||||
}, { sourceId: 'src-b' });
|
||||
await engine.upsertChunks('people/alice', [{
|
||||
chunk_index: 0,
|
||||
chunk_text: 'Alice works on gadgets in source B. Important context here.',
|
||||
chunk_source: 'compiled_truth',
|
||||
token_count: 12,
|
||||
}], { sourceId: 'src-b' });
|
||||
|
||||
await engine.putPage('people/bob', {
|
||||
type: 'person',
|
||||
title: 'Bob Source-B Only',
|
||||
compiled_truth: 'Bob lives only in source B. Important context here.',
|
||||
timeline: '',
|
||||
frontmatter: {},
|
||||
}, { sourceId: 'src-b' });
|
||||
await engine.upsertChunks('people/bob', [{
|
||||
chunk_index: 0,
|
||||
chunk_text: 'Bob lives only in source B. Important context here.',
|
||||
chunk_source: 'compiled_truth',
|
||||
token_count: 11,
|
||||
}], { sourceId: 'src-b' });
|
||||
});
|
||||
|
||||
describe('v0.34.1 source-isolation regression (#861)', () => {
|
||||
test('searchKeyword with sourceId=default excludes src-b rows', async () => {
|
||||
const results = await engine.searchKeyword('widgets', { sourceId: 'default' });
|
||||
// Should find Alice from source-A only (mentions "widgets"). src-b's
|
||||
// Alice mentions "gadgets" not "widgets" but the test guards against
|
||||
// accidentally pulling in src-b rows via a missing WHERE clause.
|
||||
expect(results.length).toBeGreaterThan(0);
|
||||
for (const r of results) {
|
||||
expect(r.source_id).toBe('default');
|
||||
}
|
||||
});
|
||||
|
||||
test('searchKeyword with sourceId=src-b excludes default rows', async () => {
|
||||
const results = await engine.searchKeyword('gadgets', { sourceId: 'src-b' });
|
||||
expect(results.length).toBeGreaterThan(0);
|
||||
for (const r of results) {
|
||||
expect(r.source_id).toBe('src-b');
|
||||
}
|
||||
});
|
||||
|
||||
test('searchKeyword with sourceIds=[default,src-b] returns both', async () => {
|
||||
// Federated read (D9 array form) returns the union.
|
||||
const results = await engine.searchKeyword('Important context', {
|
||||
sourceIds: ['default', 'src-b'],
|
||||
});
|
||||
const sources = new Set(results.map(r => r.source_id));
|
||||
expect(sources.has('default')).toBe(true);
|
||||
expect(sources.has('src-b')).toBe(true);
|
||||
});
|
||||
|
||||
test('searchKeyword with sourceIds=[default] only returns default', async () => {
|
||||
// Array form with a single element behaves like scalar.
|
||||
const results = await engine.searchKeyword('Important context', {
|
||||
sourceIds: ['default'],
|
||||
});
|
||||
for (const r of results) {
|
||||
expect(r.source_id).toBe('default');
|
||||
}
|
||||
});
|
||||
|
||||
test('searchKeyword with no source scope returns all sources', async () => {
|
||||
// Local CLI / unscoped callers preserve pre-v0.34 behavior.
|
||||
const results = await engine.searchKeyword('Important context');
|
||||
const sources = new Set(results.map(r => r.source_id));
|
||||
expect(sources.size).toBeGreaterThanOrEqual(2);
|
||||
});
|
||||
|
||||
test('listPages with sourceId=default hides src-b rows', async () => {
|
||||
const pages = await engine.listPages({ sourceId: 'default', limit: 100 });
|
||||
expect(pages.length).toBeGreaterThan(0);
|
||||
expect(pages.find(p => p.title === 'Bob Source-B Only')).toBeUndefined();
|
||||
expect(pages.find(p => p.title === 'Alice Source-B')).toBeUndefined();
|
||||
});
|
||||
|
||||
test('listPages with sourceIds=[default,src-b] returns the union', async () => {
|
||||
const pages = await engine.listPages({ sourceIds: ['default', 'src-b'], limit: 100 });
|
||||
const titles = new Set(pages.map(p => p.title));
|
||||
expect(titles.has('Alice Source-A')).toBe(true);
|
||||
expect(titles.has('Alice Source-B')).toBe(true);
|
||||
expect(titles.has('Bob Source-B Only')).toBe(true);
|
||||
});
|
||||
|
||||
test('traverseGraph with sourceId=default does not surface src-b roots', async () => {
|
||||
// Seeding the walk at src-b's bob with sourceId=default produces an
|
||||
// empty result — the seed itself is filtered out, so the walk never
|
||||
// discovers anything. Pre-fix, the walk would still return bob's
|
||||
// neighbors via cross-source edge following.
|
||||
const nodes = await engine.traverseGraph('people/bob', 5, { sourceId: 'default' });
|
||||
expect(nodes.length).toBe(0);
|
||||
});
|
||||
|
||||
test('traverseGraph with sourceId=src-b finds the src-b page', async () => {
|
||||
const nodes = await engine.traverseGraph('people/bob', 5, { sourceId: 'src-b' });
|
||||
expect(nodes.length).toBeGreaterThanOrEqual(1);
|
||||
expect(nodes.find(n => n.slug === 'people/bob')).toBeDefined();
|
||||
});
|
||||
|
||||
test('traversePaths with sourceId=default does not surface src-b paths', async () => {
|
||||
// Same seed-filter check via the edge-based traversal.
|
||||
const paths = await engine.traversePaths('people/bob', { depth: 3, sourceId: 'default' });
|
||||
expect(paths.length).toBe(0);
|
||||
});
|
||||
|
||||
test('searchVector with sourceId filters HNSW candidate pool', async () => {
|
||||
// No real embeddings on the test pages; the WHERE cc.embedding IS NOT NULL
|
||||
// gate filters them out. We assert the contract via an empty result
|
||||
// rather than a positive match: with sourceId set, the SQL still runs
|
||||
// (no type or undefined-column errors).
|
||||
const synth = new Float32Array(1536).fill(0.01);
|
||||
const results = await engine.searchVector(synth, { sourceId: 'src-b' });
|
||||
// Either empty (no embeddings) or all from src-b. Both prove the
|
||||
// filter is wired without a runtime error.
|
||||
for (const r of results) {
|
||||
expect(r.source_id).toBe('src-b');
|
||||
}
|
||||
});
|
||||
|
||||
test('AuthInfo path: ctx.sourceId scoping propagates through op handlers', async () => {
|
||||
// The op handler at operations.ts:search threads ctx.sourceId via
|
||||
// sourceScopeOpts. Simulate the dispatcher's threading and verify
|
||||
// the engine sees the filter — this is the layer the regression
|
||||
// tests need to cover most directly.
|
||||
const { operations } = await import('../../src/core/operations.ts');
|
||||
const searchOp = operations.find(o => o.name === 'search');
|
||||
expect(searchOp).toBeDefined();
|
||||
|
||||
const ctx = {
|
||||
engine,
|
||||
config: { engine: 'pglite' as const },
|
||||
logger: { info: () => {}, warn: () => {}, error: () => {} },
|
||||
dryRun: false,
|
||||
remote: true,
|
||||
sourceId: 'src-b',
|
||||
};
|
||||
const result = await searchOp!.handler(ctx as any, { query: 'gadgets' });
|
||||
const rows = result as Array<{ source_id?: string }>;
|
||||
expect(rows.length).toBeGreaterThan(0);
|
||||
for (const r of rows) {
|
||||
expect(r.source_id).toBe('src-b');
|
||||
}
|
||||
});
|
||||
|
||||
test('AuthInfo.allowedSources path: ctx.auth.allowedSources widens read scope', async () => {
|
||||
// D9 federated read: when AuthInfo.allowedSources is populated, the
|
||||
// sourceScopeOpts helper produces sourceIds array (array wins over
|
||||
// scalar ctx.sourceId). Verify the op handler routes through this.
|
||||
const { operations } = await import('../../src/core/operations.ts');
|
||||
const searchOp = operations.find(o => o.name === 'search');
|
||||
const ctx = {
|
||||
engine,
|
||||
config: { engine: 'pglite' as const },
|
||||
logger: { info: () => {}, warn: () => {}, error: () => {} },
|
||||
dryRun: false,
|
||||
remote: true,
|
||||
sourceId: 'default', // scalar would scope to default-only
|
||||
auth: {
|
||||
token: 'test',
|
||||
clientId: 'test',
|
||||
scopes: ['read'],
|
||||
sourceId: 'default',
|
||||
allowedSources: ['default', 'src-b'], // ... but the array wins
|
||||
},
|
||||
};
|
||||
const result = await searchOp!.handler(ctx as any, { query: 'Important context' });
|
||||
const rows = result as Array<{ source_id?: string }>;
|
||||
const sources = new Set(rows.map(r => r.source_id));
|
||||
expect(sources.has('default')).toBe(true);
|
||||
expect(sources.has('src-b')).toBe(true);
|
||||
});
|
||||
|
||||
test('#876 federated_read empty array means no federated reads', async () => {
|
||||
// sourceScopeOpts treats allowedSources: [] (explicit empty) as "no
|
||||
// federated scope" and falls back to scalar sourceId. An empty array
|
||||
// MUST NOT widen scope to "all sources" — that's the silent-widen
|
||||
// footgun. Verify the precedence ladder is correct.
|
||||
const { operations } = await import('../../src/core/operations.ts');
|
||||
const searchOp = operations.find(o => o.name === 'search');
|
||||
const ctx = {
|
||||
engine,
|
||||
config: { engine: 'pglite' as const },
|
||||
logger: { info: () => {}, warn: () => {}, error: () => {} },
|
||||
dryRun: false,
|
||||
remote: true,
|
||||
sourceId: 'default',
|
||||
auth: {
|
||||
token: 'test',
|
||||
clientId: 'test',
|
||||
scopes: ['read'],
|
||||
sourceId: 'default',
|
||||
allowedSources: [], // explicit empty — must NOT widen scope
|
||||
},
|
||||
};
|
||||
const result = await searchOp!.handler(ctx as any, { query: 'Important context' });
|
||||
const rows = result as Array<{ source_id?: string }>;
|
||||
for (const r of rows) {
|
||||
expect(r.source_id).toBe('default');
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1148,3 +1148,106 @@ describe('F12 dcrDisabled constructor option', () => {
|
||||
expect(result.clientSecret).toStartWith('gbrain_cs_');
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// v0.34.1 (#909) — PKCE public-client DCR (RFC 7591 §3.2.1)
|
||||
// ---------------------------------------------------------------------------
|
||||
//
|
||||
// Per RFC 7591 §3.2.1, when a DCR client declares
|
||||
// `token_endpoint_auth_method: "none"` (PKCE-only public clients like Claude
|
||||
// Code, Cursor), the authorization server MUST NOT issue a client_secret.
|
||||
// Pre-fix, unconditional secret generation made the MCP SDK's clientAuth
|
||||
// middleware reject valid public-client flows on /token.
|
||||
|
||||
describe('PKCE DCR public-client gate (#909)', () => {
|
||||
test("registerClient with token_endpoint_auth_method='none' omits client_secret", async () => {
|
||||
const result = await provider.clientsStore.registerClient!({
|
||||
client_name: 'public-pkce-client',
|
||||
redirect_uris: ['https://example.com/callback'],
|
||||
grant_types: ['authorization_code'],
|
||||
scope: 'read',
|
||||
token_endpoint_auth_method: 'none',
|
||||
});
|
||||
expect(result.client_id).toStartWith('gbrain_cl_');
|
||||
// RFC 7591 §3.2.1: public clients get NO client_secret in the response.
|
||||
expect(result.client_secret).toBeUndefined();
|
||||
expect(result.token_endpoint_auth_method).toBe('none');
|
||||
});
|
||||
|
||||
test('default auth_method (omitted) still issues a client_secret', async () => {
|
||||
// Regression guard: confidential clients (the existing default) must
|
||||
// keep their secret-issuing behavior unchanged.
|
||||
const result = await provider.clientsStore.registerClient!({
|
||||
client_name: 'confidential-default',
|
||||
redirect_uris: ['https://example.com/callback'],
|
||||
grant_types: ['authorization_code'],
|
||||
scope: 'read',
|
||||
// token_endpoint_auth_method omitted; falls back to 'client_secret_post'
|
||||
});
|
||||
expect(result.client_id).toStartWith('gbrain_cl_');
|
||||
expect(result.client_secret).toStartWith('gbrain_cs_');
|
||||
});
|
||||
|
||||
test('explicit client_secret_post still issues a client_secret', async () => {
|
||||
const result = await provider.clientsStore.registerClient!({
|
||||
client_name: 'confidential-explicit',
|
||||
redirect_uris: ['https://example.com/callback'],
|
||||
grant_types: ['authorization_code'],
|
||||
scope: 'read',
|
||||
token_endpoint_auth_method: 'client_secret_post',
|
||||
});
|
||||
expect(result.client_id).toStartWith('gbrain_cl_');
|
||||
expect(result.client_secret).toStartWith('gbrain_cs_');
|
||||
});
|
||||
|
||||
test('getClient on a public client returns client_secret=undefined (NULL normalized)', async () => {
|
||||
// The SDK's clientAuth middleware checks `client.client_secret === undefined`
|
||||
// (not `=== null`) to decide whether to enforce secret comparison on /token.
|
||||
// Without normalization, Postgres NULL would reach the SDK as JS null and
|
||||
// the secret check would mis-fire on every public client.
|
||||
const reg = await provider.clientsStore.registerClient!({
|
||||
client_name: 'public-getclient-norm',
|
||||
redirect_uris: ['https://example.com/callback'],
|
||||
grant_types: ['authorization_code'],
|
||||
scope: 'read',
|
||||
token_endpoint_auth_method: 'none',
|
||||
});
|
||||
const stored = await provider.clientsStore.getClient(reg.client_id);
|
||||
expect(stored).toBeDefined();
|
||||
expect(stored!.client_secret).toBeUndefined();
|
||||
expect(stored!.token_endpoint_auth_method).toBe('none');
|
||||
});
|
||||
|
||||
test('PKCE flow end-to-end: public client /authorize then /token, no secret needed', async () => {
|
||||
// Full F7 regression #15: public client completes auth_code → token
|
||||
// exchange without ever presenting a client_secret.
|
||||
const reg = await provider.clientsStore.registerClient!({
|
||||
client_name: 'pkce-roundtrip',
|
||||
redirect_uris: ['http://localhost:3000/callback'],
|
||||
grant_types: ['authorization_code'],
|
||||
scope: 'read',
|
||||
token_endpoint_auth_method: 'none',
|
||||
});
|
||||
|
||||
// Re-fetch via getClient to mirror what the SDK middleware sees.
|
||||
const client = (await provider.clientsStore.getClient(reg.client_id))!;
|
||||
expect(client.client_secret).toBeUndefined();
|
||||
|
||||
let redirectUrl = '';
|
||||
const mockRes = { redirect: (url: string) => { redirectUrl = url; } } as any;
|
||||
await provider.authorize(client, {
|
||||
codeChallenge: 'test-challenge-value',
|
||||
redirectUri: 'http://localhost:3000/callback',
|
||||
scopes: ['read'],
|
||||
}, mockRes);
|
||||
const code = new URL(redirectUrl).searchParams.get('code')!;
|
||||
expect(code).toMatch(/^gbrain_code_/);
|
||||
|
||||
// Exchange the code — public client; no secret on the wire.
|
||||
const tokens = await provider.exchangeAuthorizationCode(client, code);
|
||||
expect(tokens.access_token).toStartWith('gbrain_at_');
|
||||
// SDK normalizes token_type per RFC 6750 §6.1.1 (case-insensitive);
|
||||
// implementations may emit "bearer" lowercase.
|
||||
expect(String(tokens.token_type).toLowerCase()).toBe('bearer');
|
||||
});
|
||||
});
|
||||
|
||||
248
test/openai-compat-multimodal.test.ts
Normal file
248
test/openai-compat-multimodal.test.ts
Normal file
@@ -0,0 +1,248 @@
|
||||
// v0.34.1 (#875): multimodal embedding for openai-compatible recipes via
|
||||
// LiteLLM (or any other openai-compatible proxy). Sibling to
|
||||
// voyage-multimodal.test.ts; covers the new embedMultimodalOpenAICompat
|
||||
// path including D12 dim validation.
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, test } from 'bun:test';
|
||||
import { configureGateway, embedMultimodal, resetGateway } from '../src/core/ai/gateway.ts';
|
||||
import { AIConfigError, AITransientError } from '../src/core/ai/errors.ts';
|
||||
|
||||
type FetchHandler = (url: string, init: RequestInit) => Promise<Response>;
|
||||
let fetchHandler: FetchHandler | null = null;
|
||||
const origFetch = globalThis.fetch;
|
||||
|
||||
beforeEach(() => {
|
||||
fetchHandler = null;
|
||||
globalThis.fetch = (async (url: string | URL | Request, init?: RequestInit) => {
|
||||
if (!fetchHandler) {
|
||||
throw new Error('fetch called but no handler installed');
|
||||
}
|
||||
return fetchHandler(typeof url === 'string' ? url : url.toString(), init ?? {});
|
||||
}) as typeof fetch;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
globalThis.fetch = origFetch;
|
||||
resetGateway();
|
||||
});
|
||||
|
||||
function configureLitellm(env: Record<string, string | undefined> = {}, dims = 1024) {
|
||||
configureGateway({
|
||||
embedding_model: 'litellm:gpt-4o-multimodal',
|
||||
embedding_dimensions: dims,
|
||||
env: {
|
||||
LITELLM_API_KEY: 'test-litellm-key',
|
||||
LITELLM_BASE_URL: 'http://localhost:4000',
|
||||
...env,
|
||||
},
|
||||
base_urls: { litellm: 'http://localhost:4000' },
|
||||
});
|
||||
}
|
||||
|
||||
function okResponse(dims: number, count: number = 1): Response {
|
||||
const vec = Array(dims).fill(0).map((_, i) => 0.001 * i);
|
||||
return new Response(
|
||||
JSON.stringify({ data: Array.from({ length: count }, () => ({ embedding: vec })) }),
|
||||
{ status: 200, headers: { 'Content-Type': 'application/json' } },
|
||||
);
|
||||
}
|
||||
|
||||
describe('embedMultimodal — openai-compat routing (#875)', () => {
|
||||
test('LiteLLM recipe accepts a single image input and returns one embedding', async () => {
|
||||
configureLitellm();
|
||||
let capturedUrl = '';
|
||||
let capturedBody: any = null;
|
||||
let capturedAuth = '';
|
||||
fetchHandler = async (url, init) => {
|
||||
capturedUrl = url;
|
||||
capturedAuth = (init.headers as Record<string, string>).Authorization ?? '';
|
||||
capturedBody = JSON.parse(init.body as string);
|
||||
return okResponse(1024, 1);
|
||||
};
|
||||
|
||||
const result = await embedMultimodal([
|
||||
{ kind: 'image_base64', data: 'fake-base64-bytes', mime: 'image/png' },
|
||||
]);
|
||||
|
||||
expect(result.length).toBe(1);
|
||||
expect(result[0].length).toBe(1024);
|
||||
expect(capturedUrl).toBe('http://localhost:4000/embeddings');
|
||||
expect(capturedAuth).toBe('Bearer test-litellm-key');
|
||||
expect(capturedBody.model).toBe('gpt-4o-multimodal');
|
||||
expect(capturedBody.input[0].type).toBe('image_url');
|
||||
expect(capturedBody.input[0].image_url.url).toBe('data:image/png;base64,fake-base64-bytes');
|
||||
});
|
||||
|
||||
test('multiple inputs trigger sequential /embeddings calls', async () => {
|
||||
configureLitellm();
|
||||
let calls = 0;
|
||||
fetchHandler = async () => {
|
||||
calls += 1;
|
||||
return okResponse(1024, 1);
|
||||
};
|
||||
|
||||
const result = await embedMultimodal([
|
||||
{ kind: 'image_base64', data: 'img1', mime: 'image/jpeg' },
|
||||
{ kind: 'image_base64', data: 'img2', mime: 'image/png' },
|
||||
{ kind: 'image_base64', data: 'img3', mime: 'image/webp' },
|
||||
]);
|
||||
|
||||
expect(calls).toBe(3);
|
||||
expect(result.length).toBe(3);
|
||||
});
|
||||
|
||||
test('LiteLLM without LITELLM_API_KEY still works (proxy may run unauthenticated)', async () => {
|
||||
configureGateway({
|
||||
embedding_model: 'litellm:multimodal-foo',
|
||||
embedding_dimensions: 768,
|
||||
env: { LITELLM_BASE_URL: 'http://localhost:4000' }, // no API key
|
||||
base_urls: { litellm: 'http://localhost:4000' },
|
||||
});
|
||||
let capturedAuth: string | null | undefined;
|
||||
fetchHandler = async (_url, init) => {
|
||||
capturedAuth = (init.headers as Record<string, string>).Authorization;
|
||||
return okResponse(768, 1);
|
||||
};
|
||||
const result = await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]);
|
||||
expect(result.length).toBe(1);
|
||||
// defaultResolveAuth sends 'Bearer unauthenticated' when no api key is
|
||||
// configured — servers like Ollama / llama-server ignore the value but
|
||||
// the SDK contract still requires SOME Authorization header.
|
||||
expect(capturedAuth).toBe('Bearer unauthenticated');
|
||||
});
|
||||
|
||||
test('D12 — provider returns wrong-dim vector throws AIConfigError', async () => {
|
||||
// Brain configured for 1024; provider returns 768. D12 catches the
|
||||
// mismatch BEFORE the vector lands in the DB column.
|
||||
configureLitellm({}, 1024);
|
||||
fetchHandler = async () => okResponse(768, 1);
|
||||
|
||||
let caught: unknown;
|
||||
try {
|
||||
await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]);
|
||||
} catch (err) {
|
||||
caught = err;
|
||||
}
|
||||
expect(caught).toBeInstanceOf(AIConfigError);
|
||||
expect((caught as Error).message).toContain('768-dim vector');
|
||||
expect((caught as Error).message).toContain('expected 1024');
|
||||
expect((caught as Error).message).toContain('gpt-4o-multimodal');
|
||||
});
|
||||
|
||||
test('D12 — default embedding_dimensions (1536) applies when not explicitly set', async () => {
|
||||
// configureGateway normalizes embedding_dimensions to 1536 when unset
|
||||
// (the DEFAULT_EMBEDDING_DIMENSIONS). LiteLLM recipe's default_dims=0
|
||||
// so we fall back to the brain's configured value. This test pins the
|
||||
// "always validate via the configured/default dim" contract — there
|
||||
// is no skip-when-unset path in practice because configureGateway
|
||||
// always populates it.
|
||||
configureGateway({
|
||||
embedding_model: 'litellm:any-model',
|
||||
// intentionally NO embedding_dimensions → falls back to 1536
|
||||
env: { LITELLM_BASE_URL: 'http://localhost:4000' },
|
||||
base_urls: { litellm: 'http://localhost:4000' },
|
||||
});
|
||||
fetchHandler = async () => okResponse(1536, 1);
|
||||
const result = await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]);
|
||||
expect(result.length).toBe(1);
|
||||
expect(result[0].length).toBe(1536);
|
||||
});
|
||||
|
||||
test('provider returns 401 → AIConfigError with model id in message', async () => {
|
||||
configureLitellm();
|
||||
fetchHandler = async () =>
|
||||
new Response('invalid key', { status: 401, headers: { 'Content-Type': 'text/plain' } });
|
||||
|
||||
let caught: unknown;
|
||||
try {
|
||||
await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]);
|
||||
} catch (err) {
|
||||
caught = err;
|
||||
}
|
||||
expect(caught).toBeInstanceOf(AIConfigError);
|
||||
expect((caught as Error).message).toContain('401');
|
||||
});
|
||||
|
||||
test('provider returns 400 (model does not support multimodal) → AITransientError surfaces body', async () => {
|
||||
configureLitellm();
|
||||
fetchHandler = async () =>
|
||||
new Response('model does not support image inputs', { status: 400 });
|
||||
|
||||
let caught: unknown;
|
||||
try {
|
||||
await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]);
|
||||
} catch (err) {
|
||||
caught = err;
|
||||
}
|
||||
expect(caught).toBeInstanceOf(AITransientError);
|
||||
expect((caught as Error).message).toContain('400');
|
||||
expect((caught as Error).message).toContain('model does not support image inputs');
|
||||
});
|
||||
|
||||
test('malformed JSON response → AITransientError', async () => {
|
||||
configureLitellm();
|
||||
fetchHandler = async () =>
|
||||
new Response('not json', { status: 200, headers: { 'Content-Type': 'application/json' } });
|
||||
|
||||
let caught: unknown;
|
||||
try {
|
||||
await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]);
|
||||
} catch (err) {
|
||||
caught = err;
|
||||
}
|
||||
expect(caught).toBeInstanceOf(AITransientError);
|
||||
expect((caught as Error).message).toContain('malformed JSON');
|
||||
});
|
||||
|
||||
test('non-array embedding payload → AITransientError', async () => {
|
||||
configureLitellm();
|
||||
fetchHandler = async () =>
|
||||
new Response(JSON.stringify({ data: [{ embedding: 'not-array' }] }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
|
||||
let caught: unknown;
|
||||
try {
|
||||
await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]);
|
||||
} catch (err) {
|
||||
caught = err;
|
||||
}
|
||||
expect(caught).toBeInstanceOf(AITransientError);
|
||||
expect((caught as Error).message).toContain('non-array');
|
||||
});
|
||||
|
||||
test('empty data array → AITransientError', async () => {
|
||||
configureLitellm();
|
||||
fetchHandler = async () =>
|
||||
new Response(JSON.stringify({ data: [] }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
|
||||
let caught: unknown;
|
||||
try {
|
||||
await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]);
|
||||
} catch (err) {
|
||||
caught = err;
|
||||
}
|
||||
expect(caught).toBeInstanceOf(AITransientError);
|
||||
});
|
||||
|
||||
test('Voyage recipe still routes to /multimodalembeddings (regression)', async () => {
|
||||
// Ensure the new openai-compat route doesn't accidentally hijack Voyage.
|
||||
configureGateway({
|
||||
embedding_model: 'voyage:voyage-multimodal-3',
|
||||
embedding_dimensions: 1024,
|
||||
env: { VOYAGE_API_KEY: 'voyage-key' },
|
||||
});
|
||||
let capturedUrl = '';
|
||||
fetchHandler = async (url) => {
|
||||
capturedUrl = url;
|
||||
return okResponse(1024, 1);
|
||||
};
|
||||
|
||||
await embedMultimodal([{ kind: 'image_base64', data: 'x', mime: 'image/png' }]);
|
||||
expect(capturedUrl).toContain('/multimodalembeddings');
|
||||
});
|
||||
});
|
||||
@@ -81,6 +81,7 @@ function makeHarness(opts: {
|
||||
isTTY?: boolean;
|
||||
initialParentPid?: number;
|
||||
probeWatchdog?: boolean;
|
||||
mcpStdio?: boolean;
|
||||
} = {}): Harness {
|
||||
const engine = new StubEngine();
|
||||
const stdin = new EventEmitter() as EventEmitter & { isTTY?: boolean };
|
||||
@@ -120,6 +121,7 @@ function makeHarness(opts: {
|
||||
setInterval: timers.setInterval,
|
||||
clearInterval: timers.clearInterval,
|
||||
probeWatchdog: () => probeWatchdogResult,
|
||||
mcpStdio: opts.mcpStdio,
|
||||
};
|
||||
|
||||
return {
|
||||
@@ -439,4 +441,63 @@ describe('runServe stdio lifecycle', () => {
|
||||
expect(code).toBe(0);
|
||||
expect(h.logs.some(l => l.includes('cleanup error: synthetic disconnect failure'))).toBe(true);
|
||||
});
|
||||
|
||||
// v0.34.1 (#870): OpenClaw gateway / bundle-mcp wrappers pipe the
|
||||
// JSON-RPC handshake on stdin then close their stdin half. Without
|
||||
// MCP_STDIO=1 the server treats that as a permanent disconnect and
|
||||
// exits before handling tools/call. The guard skips the stdin 'end' /
|
||||
// 'close' hooks when MCP_STDIO=1; signals and parent watchdog still
|
||||
// cover legitimate shutdown.
|
||||
describe('MCP_STDIO=1 piped-stdin guard (#870)', () => {
|
||||
test('stdin end with mcpStdio=true does NOT trigger shutdown', async () => {
|
||||
const h = makeHarness({ mcpStdio: true });
|
||||
await startInBackground(h.engine, [], h.opts);
|
||||
|
||||
// Without the guard this would shutdown; with the guard it must not.
|
||||
h.stdin.emit('end');
|
||||
|
||||
// Give the event loop a microtask turn to catch any erroneous shutdown
|
||||
// path. We assert NO exit was registered.
|
||||
await new Promise<void>((r) => setTimeout(r, 10));
|
||||
expect(h.engine.disconnectCalls).toBe(0);
|
||||
|
||||
// Then trigger SIGTERM to drive the test to completion; signal handlers
|
||||
// remain active even with mcpStdio=true (codex would catch if they didn't).
|
||||
h.signals.emit('SIGTERM');
|
||||
const code = await h.exited;
|
||||
expect(code).toBe(0);
|
||||
expect(h.engine.disconnectCalls).toBe(1);
|
||||
expect(h.logs.some(l => l.includes('graceful exit (SIGTERM)'))).toBe(true);
|
||||
});
|
||||
|
||||
test('stdin close with mcpStdio=true does NOT trigger shutdown', async () => {
|
||||
const h = makeHarness({ mcpStdio: true });
|
||||
await startInBackground(h.engine, [], h.opts);
|
||||
|
||||
h.stdin.emit('close');
|
||||
|
||||
await new Promise<void>((r) => setTimeout(r, 10));
|
||||
expect(h.engine.disconnectCalls).toBe(0);
|
||||
|
||||
h.signals.emit('SIGINT');
|
||||
const code = await h.exited;
|
||||
expect(code).toBe(0);
|
||||
expect(h.engine.disconnectCalls).toBe(1);
|
||||
});
|
||||
|
||||
test('mcpStdio=false (default) preserves stdin EOF shutdown', async () => {
|
||||
// Regression guard: the guard must not over-trigger. With the env
|
||||
// unset, stdin EOF must still drive shutdown so existing CLI usage
|
||||
// (gbrain serve under launchd, claude-desktop's stdio MCP) is
|
||||
// unchanged.
|
||||
const h = makeHarness({ mcpStdio: false });
|
||||
await startInBackground(h.engine, [], h.opts);
|
||||
|
||||
h.stdin.emit('end');
|
||||
const code = await h.exited;
|
||||
expect(code).toBe(0);
|
||||
expect(h.engine.disconnectCalls).toBe(1);
|
||||
expect(h.logs.some(l => l.includes('graceful exit (stdin-end)'))).toBe(true);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user