feat: v0.9.0 -- smart file storage, publish, production-grade skills (#62)
* feat: battle-tested skill patterns from production deployment Backport production-learned brain-operations patterns: - Iron Law of Back-Linking (mandatory bidirectional linking) - Brain filing rules (file by primary subject, not format) - Enrichment protocol (7-step pipeline, 3-tier system, person/company templates) - Media ingest workflows (articles, videos, podcasts, PDFs, screenshots) - Citation requirements (mandatory [Source: ...] on every fact) - Test Before Bulk operating principle - Voice recipe: unicode crash fix, PII scrub, identity-first prompt, DIY STT+LLM+TTS - X-to-Brain recipe: image OCR, Filtered Stream, tweet rating rubric, cron stagger * chore: bump version and changelog (v0.8.1) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add _brain-filing-rules.md to CLAUDE.md key files * feat: smart file upload with TUS resumable and .redirect.yaml pointers - Supabase Storage auto-selects upload method by file size: < 100 MB standard POST, >= 100 MB TUS resumable (6 MB chunks + retry) - Signed URL generation for private bucket access (1-hour expiry) - New `upload-raw` command with size routing: small text stays in git, large/media files go to cloud with .redirect.yaml pointer - New `signed-url` command for generating access links - File resolver supports both .redirect.yaml (v0.9+) and .redirect (legacy) - Redirect format upgraded: 10 fields with full metadata - All migration commands (mirror, redirect, restore, clean) handle both formats * feat: skills reference actual gbrain file commands - Filing rules document upload-raw, signed-url, and .redirect.yaml format - Ingest skill uses gbrain files upload-raw for raw source preservation - Maintain skill adds file storage health checks - Setup skill adds storage configuration phase with migration guidance - Voice recipe uses upload-raw for call audio storage - Migration v0.9.0 with complete storage setup instructions * chore: bump version and changelog (v0.9.0) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: gbrain publish -- shareable HTML with password protection First code+skill pair: deterministic code does the work (strip private data, encrypt with AES-256-GCM, generate self-contained HTML), the skill tells the agent when and how to use it. 34 new tests. See: https://x.com/garrytan/status/2042925773300908103 * feat: backlinks check/fix, page lint, and report commands Three new deterministic tools (zero LLM calls): - gbrain backlinks check/fix -- scans brain for entity mentions without back-links, creates them. Enforces the Iron Law from the skills. - gbrain lint [--fix] -- catches LLM preambles, code fence wrapping, placeholder dates, missing frontmatter, broken citations, empty sections. --fix auto-strips fixable artifacts. - gbrain report --type <name> -- saves timestamped reports to brain/reports/{type}/YYYY-MM-DD-HHMM.md for audit trails. 33 new tests (409 total, 0 fail). * feat: v0.9.0 migration tells agents to swap scripts for built-in commands Migration file now: - Lists all 5 new deterministic commands with usage examples - Includes a script-to-command replacement table (old -> new) - Tells the agent to find custom script references in AGENTS.md, skills, and cron jobs and replace with gbrain commands - Adds recommended cron jobs for daily backlink fix + weekly lint - References the Thin Harness, Fat Skills thread * fix: CLI routing bugs found during DX review - Fixed subArgs reference error in handleCliOnly (used wrong variable name) - Renamed gbrain backlinks check/fix to gbrain check-backlinks to avoid conflict with existing backlinks operation (per-page incoming links) - Added TOOLS section to --help output showing publish, check-backlinks, lint, report - Added upload-raw and signed-url to FILES section in --help - Updated all docs/migration references to use check-backlinks * fix: security hardening from adversarial review - XSS: sanitize marked.parse() output (strip script/iframe/on* attrs) - Path traversal: validate report --type against [a-z0-9-] pattern - TUS: HEAD request before retry to get server's actual offset (TUS spec) - Pointer: upload-raw now includes pointer content in JSON output - Symlinks: use lstatSync in all walkers to prevent directory escape --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
217
test/publish.test.ts
Normal file
217
test/publish.test.ts
Normal file
@@ -0,0 +1,217 @@
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import {
|
||||
makeShareable,
|
||||
extractTitle,
|
||||
encryptContent,
|
||||
generatePassword,
|
||||
generateHtml,
|
||||
} from '../src/commands/publish.ts';
|
||||
|
||||
describe('makeShareable', () => {
|
||||
test('strips YAML frontmatter', () => {
|
||||
const input = '---\ntitle: Secret\ntype: person\n---\n\n# Jane Doe\n\nPublic content.';
|
||||
const result = makeShareable(input);
|
||||
expect(result).not.toContain('title: Secret');
|
||||
expect(result).not.toContain('type: person');
|
||||
expect(result).toContain('# Jane Doe');
|
||||
expect(result).toContain('Public content.');
|
||||
});
|
||||
|
||||
test('strips [Source: ...] citations', () => {
|
||||
const input = 'Jane is CTO [Source: Crustdata enrichment, 2026-04-01] of Acme.';
|
||||
expect(makeShareable(input)).toBe('Jane is CTO of Acme.');
|
||||
});
|
||||
|
||||
test('strips multi-format citations', () => {
|
||||
const input = 'Fact one [Source: User, meeting, 2026-04-01]. Fact two [Source: compiled from timeline].';
|
||||
const result = makeShareable(input);
|
||||
expect(result).not.toContain('[Source:');
|
||||
expect(result).toContain('Fact one');
|
||||
expect(result).toContain('Fact two');
|
||||
});
|
||||
|
||||
test('redacts confirmation numbers', () => {
|
||||
const input = '**Confirmation:** ABC123DEF456';
|
||||
expect(makeShareable(input)).toContain('on file');
|
||||
expect(makeShareable(input)).not.toContain('ABC123DEF456');
|
||||
});
|
||||
|
||||
test('strips brain cross-links, keeps display text', () => {
|
||||
const input = 'Works with [Jane Doe](../people/jane-doe.md) at Acme.';
|
||||
const result = makeShareable(input);
|
||||
expect(result).toBe('Works with Jane Doe at Acme.');
|
||||
expect(result).not.toContain('../people/');
|
||||
});
|
||||
|
||||
test('preserves external URLs', () => {
|
||||
const input = 'See [their blog](https://example.com/blog) for details.';
|
||||
expect(makeShareable(input)).toContain('https://example.com/blog');
|
||||
});
|
||||
|
||||
test('removes See also lines', () => {
|
||||
const input = '# Title\n\nContent.\n\n- See also: ../companies/acme.md\n\nMore content.';
|
||||
const result = makeShareable(input);
|
||||
expect(result).not.toContain('See also');
|
||||
expect(result).toContain('More content');
|
||||
});
|
||||
|
||||
test('removes Timeline section', () => {
|
||||
const input = '# Title\n\nPublic content.\n\n---\n\n## Timeline\n\n- 2026-04-01 | Secret event';
|
||||
const result = makeShareable(input);
|
||||
expect(result).toContain('Public content.');
|
||||
expect(result).not.toContain('Timeline');
|
||||
expect(result).not.toContain('Secret event');
|
||||
});
|
||||
|
||||
test('collapses excessive blank lines', () => {
|
||||
const input = '# Title\n\n\n\n\nContent.';
|
||||
expect(makeShareable(input)).toBe('# Title\n\nContent.');
|
||||
});
|
||||
|
||||
test('handles empty input', () => {
|
||||
expect(makeShareable('')).toBe('');
|
||||
});
|
||||
|
||||
test('handles frontmatter-only input', () => {
|
||||
const input = '---\ntitle: Test\n---\n';
|
||||
expect(makeShareable(input)).toBe('');
|
||||
});
|
||||
|
||||
test('strips .raw/ relative links', () => {
|
||||
const input = 'See [raw data](.raw/crustdata.json) for source.';
|
||||
const result = makeShareable(input);
|
||||
expect(result).toBe('See raw data for source.');
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractTitle', () => {
|
||||
test('extracts H1 title', () => {
|
||||
expect(extractTitle('# Jane Doe\n\nContent.')).toBe('Jane Doe');
|
||||
});
|
||||
|
||||
test('extracts title with formatting', () => {
|
||||
expect(extractTitle('# **Bold** Title\n\nContent.')).toBe('**Bold** Title');
|
||||
});
|
||||
|
||||
test('returns "Document" when no H1', () => {
|
||||
expect(extractTitle('No heading here.')).toBe('Document');
|
||||
});
|
||||
|
||||
test('ignores H2 and lower', () => {
|
||||
expect(extractTitle('## Not H1\n\nContent.')).toBe('Document');
|
||||
});
|
||||
|
||||
test('picks first H1 when multiple exist', () => {
|
||||
expect(extractTitle('# First\n\n# Second')).toBe('First');
|
||||
});
|
||||
});
|
||||
|
||||
describe('encryptContent', () => {
|
||||
test('returns salt, iv, and ciphertext', () => {
|
||||
const result = encryptContent('hello world', 'password123');
|
||||
expect(result.salt).toBeTruthy();
|
||||
expect(result.iv).toBeTruthy();
|
||||
expect(result.ciphertext).toBeTruthy();
|
||||
});
|
||||
|
||||
test('produces valid base64', () => {
|
||||
const result = encryptContent('test content', 'pw');
|
||||
expect(() => Buffer.from(result.salt, 'base64')).not.toThrow();
|
||||
expect(() => Buffer.from(result.iv, 'base64')).not.toThrow();
|
||||
expect(() => Buffer.from(result.ciphertext, 'base64')).not.toThrow();
|
||||
});
|
||||
|
||||
test('different passwords produce different ciphertext', () => {
|
||||
const a = encryptContent('same text', 'password1');
|
||||
const b = encryptContent('same text', 'password2');
|
||||
expect(a.ciphertext).not.toBe(b.ciphertext);
|
||||
});
|
||||
|
||||
test('same password produces different output (random salt/iv)', () => {
|
||||
const a = encryptContent('same text', 'same password');
|
||||
const b = encryptContent('same text', 'same password');
|
||||
expect(a.salt).not.toBe(b.salt);
|
||||
expect(a.iv).not.toBe(b.iv);
|
||||
});
|
||||
|
||||
test('handles unicode content', () => {
|
||||
const result = encryptContent('Hello -- arrows -> and quotes "test"', 'pw');
|
||||
expect(result.ciphertext).toBeTruthy();
|
||||
});
|
||||
|
||||
test('handles empty string', () => {
|
||||
const result = encryptContent('', 'pw');
|
||||
expect(result.ciphertext).toBeTruthy();
|
||||
});
|
||||
});
|
||||
|
||||
describe('generatePassword', () => {
|
||||
test('default length is 16', () => {
|
||||
expect(generatePassword()).toHaveLength(16);
|
||||
});
|
||||
|
||||
test('custom length', () => {
|
||||
expect(generatePassword(8)).toHaveLength(8);
|
||||
expect(generatePassword(32)).toHaveLength(32);
|
||||
});
|
||||
|
||||
test('excludes ambiguous characters', () => {
|
||||
// No 0, O, l, 1, I (all excluded from the charset)
|
||||
for (let i = 0; i < 50; i++) {
|
||||
const pw = generatePassword(32);
|
||||
expect(pw).not.toMatch(/[0OlI1]/);
|
||||
}
|
||||
});
|
||||
|
||||
test('generates unique passwords', () => {
|
||||
const passwords = new Set(Array.from({ length: 20 }, () => generatePassword()));
|
||||
expect(passwords.size).toBe(20);
|
||||
});
|
||||
});
|
||||
|
||||
describe('generateHtml', () => {
|
||||
test('generates valid HTML with title', () => {
|
||||
const html = generateHtml({ title: 'Test Page', markdown: '# Hello\n\nWorld.' });
|
||||
expect(html).toContain('<!DOCTYPE html>');
|
||||
expect(html).toContain('<title>Test Page</title>');
|
||||
expect(html).toContain('marked.parse');
|
||||
});
|
||||
|
||||
test('includes markdown content as JSON', () => {
|
||||
const html = generateHtml({ title: 'T', markdown: '# Test Content' });
|
||||
expect(html).toContain('# Test Content');
|
||||
});
|
||||
|
||||
test('escapes HTML in title', () => {
|
||||
const html = generateHtml({ title: '<script>alert("xss")</script>', markdown: 'x' });
|
||||
expect(html).not.toContain('<script>alert("xss")</script>');
|
||||
expect(html).toContain('<script>');
|
||||
});
|
||||
|
||||
test('includes password UI when encrypted', () => {
|
||||
const encrypted = encryptContent('secret', 'pw');
|
||||
const html = generateHtml({ title: 'T', markdown: 'x', encrypted });
|
||||
expect(html).toContain('pw-overlay');
|
||||
expect(html).toContain('pw-form');
|
||||
expect(html).toContain('Enter password');
|
||||
expect(html).toContain('window.__SALT');
|
||||
expect(html).toContain('window.__IV');
|
||||
expect(html).toContain('window.__CT');
|
||||
});
|
||||
|
||||
test('no password UI when unencrypted', () => {
|
||||
const html = generateHtml({ title: 'T', markdown: 'x' });
|
||||
expect(html).not.toContain('pw-overlay');
|
||||
expect(html).not.toContain('window.__SALT');
|
||||
});
|
||||
|
||||
test('includes dark mode CSS', () => {
|
||||
const html = generateHtml({ title: 'T', markdown: 'x' });
|
||||
expect(html).toContain('prefers-color-scheme: dark');
|
||||
});
|
||||
|
||||
test('includes marked.js CDN', () => {
|
||||
const html = generateHtml({ title: 'T', markdown: 'x' });
|
||||
expect(html).toContain('cdn.jsdelivr.net/npm/marked');
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user