13773be071
* fix: security hardening — search DoS, slug hijack, symlink traversal, content bombs, stdin guard 4 security vulnerabilities closed: - Search limit clamped to 100 (MAX_SEARCH_LIMIT) with statement_timeout 8s - Frontmatter slug authority enforced (path-derived, mismatch rejected) - Symlink traversal blocked (lstatSync in walker + importFromFile) - Content size guard on importFromContent (Buffer.byteLength, 5MB) - Stdin size guard in parseOpArgs (5MB cap) Search pagination added (--offset param on search + query operations). Clamp warning emitted when limit is capped. Co-Authored-By: garagon <garagon@users.noreply.github.com> * fix: PGLite concurrent access lock — prevent Aborted() crash File-based advisory lock using atomic mkdir with PID tracking and 5-minute stale detection. Clear error messages show which process holds the lock and how to recover. Co-Authored-By: danbr <danbr@users.noreply.github.com> * fix: 12 data integrity fixes + stale embedding prevention CTE searchKeyword rewrite (SQL-level LIMIT, not JS splice). Write validation on addLink/addTag/addTimelineEntry/putRawData/createVersion. Health metrics now measure real problems (stale_pages, orphan_pages, dead_links). Orphan chunk cleanup on empty pages. Embedding error logging. contentHash now covers all PageInput fields. Stale embedding NULL'd when chunk_text changes (prevents wrong vector on new text). hybridSearch stops double-embedding query. MCP param validation. type/exclude_slugs search filters now work. pgcrypto extension for Postgres <13. Co-Authored-By: win4r <win4r@users.noreply.github.com> * perf: 30x embedAll speedup + O(n²) fix + ask alias Sliding worker pool (concurrency 20, tunable via GBRAIN_EMBED_CONCURRENCY). O(n²) chunk lookup in embedPage replaced with Map. gbrain ask alias for query (CLI-only, not in MCP tools-json). .idea added to .gitignore. Co-Authored-By: stephenhungg <stephenhungg@users.noreply.github.com> Co-Authored-By: sharziki <sharziki@users.noreply.github.com> Co-Authored-By: hnshah <hnshah@users.noreply.github.com> Co-Authored-By: doguabaris <doguabaris@users.noreply.github.com> * chore: bump version and changelog (v0.9.1) Community fix wave: 10 PRs, 7 contributors. 4 security fixes, PGLite crash fix, 12 data integrity fixes, 30x embed speedup, search pagination, ask alias. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: garagon <garagon@users.noreply.github.com> Co-authored-by: danbr <danbr@users.noreply.github.com> Co-authored-by: win4r <win4r@users.noreply.github.com> Co-authored-by: stephenhungg <stephenhungg@users.noreply.github.com> Co-authored-by: sharziki <sharziki@users.noreply.github.com> Co-authored-by: hnshah <hnshah@users.noreply.github.com> Co-authored-by: doguabaris <doguabaris@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
69 lines
2.1 KiB
Markdown
69 lines
2.1 KiB
Markdown
---
|
|
version: 0.9.1
|
|
feature_pitch:
|
|
headline: "Security hardening + 30x faster embeddings"
|
|
description: "4 security fixes (search DoS, slug hijack, symlink traversal, content bombs), PGLite crash fix, 12 data integrity fixes, 30x embed speedup, and search pagination. Community fix wave: 10 PRs, 7 contributors."
|
|
recipe: null
|
|
tiers: null
|
|
---
|
|
|
|
# v0.9.1 Migration: Security + Integrity + Performance
|
|
|
|
This migration includes security hardening, data integrity fixes, and a
|
|
performance improvement that changes how hashes are computed. Existing users
|
|
will see a one-time full re-import on next sync.
|
|
|
|
## What Changed
|
|
|
|
### Content hash algorithm
|
|
The content hash now includes all page fields (title, type, frontmatter), not
|
|
just compiled_truth + timeline. This means every existing page's hash will differ
|
|
on next sync, triggering a full re-import. This is expected and correct.
|
|
|
|
### Search limit ceiling
|
|
Search results are now capped at 100. If your agent requests more, it gets a
|
|
warning and 100 results. Use the new `--offset` flag to paginate.
|
|
|
|
### Embedding performance
|
|
`gbrain embed --all` now runs 30x faster (sliding worker pool, 20 concurrent
|
|
by default). A 20,000-chunk corpus that took 2.5 hours now takes ~8 minutes.
|
|
|
|
## Agent Instructions
|
|
|
|
### Step 1: Run sync (expect full re-import)
|
|
|
|
The content hash change will cause all pages to re-import. This is expected.
|
|
|
|
```bash
|
|
gbrain sync
|
|
```
|
|
|
|
You will see all pages reported as "imported" instead of the usual few "modified".
|
|
This is a one-time event.
|
|
|
|
### Step 2: Re-embed all pages
|
|
|
|
After the re-import, embeddings need refreshing:
|
|
|
|
```bash
|
|
gbrain embed --all
|
|
```
|
|
|
|
With the new parallel worker pool, this should complete in under 10 minutes
|
|
for most brains. Set `GBRAIN_EMBED_CONCURRENCY=10` if you hit API rate limits.
|
|
|
|
### Step 3: Verify search works
|
|
|
|
```bash
|
|
gbrain search "test query" --limit 5
|
|
```
|
|
|
|
If search returns results, the re-import and re-embed worked correctly.
|
|
|
|
### Step 4: Done
|
|
|
|
```bash
|
|
mkdir -p ~/.gbrain/migrations
|
|
echo '{"version":"0.9.1","ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","status":"complete"}' >> ~/.gbrain/migrations/completed.jsonl
|
|
```
|