Files
gbrain/skills/maintain/SKILL.md
Garry Tan b7e3005b5b fix: sync pipeline, extract, features, autopilot (v0.10.1) (#129)
* feat: migrate 8 existing skills to conformance format

Add YAML frontmatter (name, version, description, triggers, tools, mutating),
Contract, Anti-Patterns, and Output Format sections to all existing skills.
Rename Workflow to Phases. Ingest becomes thin router delegating to specialized
ingestion skills (Phase 2).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add RESOLVER.md, conventions directory, and output rules

RESOLVER.md is the skill dispatcher modeled on Wintermute's AGENTS.md.
Categorized routing table: Always-on, Brain ops, Ingestion, Thinking,
Operational, Setup, Identity. Conventions directory extracts cross-cutting
rules (quality, brain-first lookup, model routing, test-before-bulk).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add skills conformance and resolver validation tests

skills-conformance.test.ts validates every skill has YAML frontmatter with
required fields, Contract, Anti-Patterns, and Output Format sections, and
manifest.json coverage. resolver.test.ts validates routing table categories,
skill path existence, and manifest-to-resolver coverage. 50 new tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add 9 brain skills from Wintermute (Phase 2)

Generalized from Wintermute's battle-tested skills:
- signal-detector: always-on idea+entity capture on every message
- brain-ops: brain-first lookup, read-enrich-write loop, source attribution
- idea-ingest: links/articles/tweets with author people page mandatory
- media-ingest: video/audio/PDF/book with entity extraction (absorbs video/youtube/book)
- meeting-ingestion: transcripts with attendee enrichment chaining
- citation-fixer: audit and fix citation formatting
- repo-architecture: filing rules by primary subject
- skill-creator: create skills with conformance standard + MECE check
- daily-task-manager: task lifecycle with priority levels

All Garry-specific references generalized. Core workflows preserved.
Updated RESOLVER.md and manifest.json.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add operational infrastructure + identity layer (Phase 3)

Operational skills:
- daily-task-prep: morning prep with calendar context and open threads
- cross-modal-review: quality gate via second model with refusal routing
- cron-scheduler: schedule staggering, quiet hours, wake-up override, idempotency
- reports: timestamped reports with keyword routing
- testing: skill validation framework (conformance checks)
- soul-audit: 6-phase interview generating SOUL.md, USER.md, ACCESS_POLICY.md, HEARTBEAT.md
- webhook-transforms: external events to brain signals with dead-letter queue

Identity layer:
- SOUL.md template (agent identity, generated by soul-audit)
- USER.md template (user profile, generated by soul-audit)
- ACCESS_POLICY.md template (4-tier access control)
- HEARTBEAT.md template (operational cadence)
- cross-modal.yaml convention (review pairs, refusal routing chain)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update CLAUDE.md with 24 skills, RESOLVER.md, conventions, templates

GBrain is now a GStack mod for agent platforms. Updated architecture description,
key files listing (16 new skill files, RESOLVER.md, conventions, templates), skills
section (24 skills organized by resolver categories), and testing section (new
conformance and resolver tests).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add GStack detection + mod status to gbrain init (Phase 4)

After brain initialization, gbrain init now reports:
- Number of skills loaded (from manifest.json)
- GStack detection (checks known host paths, uses gstack-global-discover if available)
- GStack install instructions if not found
- Resolver and soul-audit pointers

Also adds installDefaultTemplates() for SOUL.md/USER.md/ACCESS_POLICY.md/HEARTBEAT.md
deployment, and detectGStack() using gstack-global-discover with fallback to known paths
(DRY: doesn't reimplement GStack's host detection logic).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: v0.10.0 release documentation

- CHANGELOG: 24 skills, signal detector, RESOLVER.md, soul-audit, access control,
  conventions, conformance standard, GStack detection in init
- README: updated skill section with 24 skills, resolver, conventions
- TODOS: added runtime MCP access control (P1)
- VERSION: 0.9.2 → 0.10.0
- package.json + manifest.json version bumped

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add skill table to CHANGELOG v0.10.0

16-row table detailing every new skill, what it does, and why it matters.
Written to sell the upgrade, not document the implementation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: restore package.json version after merge conflict resolution

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: zero-based README rewrite for GStackBrain v0.10.0

Lead with GStack mod identity. 24 skills table organized by category.
Install block references RESOLVER.md and soul-audit. GBrain+GStack
relationship explained. Removed redundancy (733 -> 406 lines).
All essential content preserved: install, recipes, architecture,
search, commands, engines, voice, knowledge model.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: extract install block to INSTALL_FOR_AGENTS.md, simplify README

The 30-line copy-paste install block becomes one line:
"Retrieve and follow INSTALL_FOR_AGENTS.md"

Benefits: agent always gets latest instructions (no stale copy-paste),
README stays clean, install details live where agents read them.

README now leads with what GBrain does ("gives your agent a brain")
instead of GStack relationship. Removed "requires frontier model" note.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: 3 bugs in init.ts from merge conflict resolution

1. llstatSync typo (merge corruption) → lstatSync
2. __dirname undefined in ESM module → fileURLToPath polyfill
3. require('fs') in ESM → use imported readFileSync

All three would crash gbrain init at runtime. Caught by /review.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add checkResolvable shared core function for resolver validation

Shared function at src/core/check-resolvable.ts validates that all skills
are reachable from RESOLVER.md, detects MECE overlaps (with whitelist for
always-on/router skills), finds gaps in frontmatter triggers, and scans
for DRY violations. Returns structured ResolvableIssue objects with
machine-parseable fix objects alongside human-readable action strings.

Three call sites: bun test, gbrain doctor, skill-creator skill.

Cleans up test/resolver.test.ts: removes stale 9-line skip list, imports
from production check-resolvable.ts instead of reimplementing parsing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: expand doctor with resolver validation, filesystem-first architecture

Doctor now runs filesystem checks (resolver health, skill conformance) before
connecting to DB. New --fast flag skips DB checks. Falls back to filesystem-only
when DB is unavailable. Adds schema_version: 2 to JSON output, composite health
score (0-100), and structured issues array with action strings for agent parsing.

Resolver health check calls checkResolvable() and surfaces actionable fix
instructions. Link integrity check uses engine.getHealth() dead_links count.

CLI routing split: doctor dispatched before connectEngine() so filesystem
checks always run. Fixes Codex-identified blocker where doctor required DB.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add adaptive load-aware throttling and fail-improve loop

backoff.ts: System load checking (CPU via os.loadavg, memory via os.freemem),
exponential backoff with 20-attempt max guard, active hours multiplier (2x
slower during waking hours), concurrent process limit (max 2). Windows-safe:
defaults to "proceed" when os.loadavg returns zeros.

fail-improve.ts: Deterministic-first, LLM-fallback pattern with JSONL failure
logging. Cascade failure handling: when both paths fail, throws LLM error and
logs both. Log rotation at 1000 entries. Call count tracking for deterministic
hit rate metrics. Auto-generates test cases from successful LLM fallbacks.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add transcription service and enrichment-as-a-service

transcription.ts: Groq Whisper (default) with OpenAI fallback. Files >25MB
segmented via ffmpeg. Provider auto-detection from env vars. Clear error
messages for missing API keys and unsupported formats.

enrichment-service.ts: Global enrichment service callable from any ingest
pathway. Entity slug generation (people/jane-doe, companies/acme-corp),
mention counting via searchKeyword, tier auto-escalation (Tier 3→2→1 based
on mention frequency and source diversity), batch enrichment with backoff
throttling, regex-based entity extraction from text.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add data-research skill with recipe system, extraction, dedup, tracker

New skill: data-research — one parameterized pipeline for any email-to-
structured-data workflow (investor updates, donations, company metrics).
7-phase pipeline: define recipe, search, classify, extract (with extraction
integrity rule), archive, deduplicate, update tracker.

data-research.ts: Recipe validation, MRR/ARR/runway/headcount regex
extraction (battle-tested patterns), dedup with configurable tolerance,
markdown tracker parsing/appending, quarterly/monthly date windowing,
6-phase HTML email stripping with 500KB ReDoS cap.

Registers data-research in manifest.json (25th skill) and RESOLVER.md.
Fixes backoff test robustness for high-load systems.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update project documentation for v0.10.0 infrastructure additions

CLAUDE.md: added 6 new core files (check-resolvable, backoff, fail-improve,
transcription, enrichment-service, data-research), 6 new test files, updated
skill count to 25, test file count to 34.

README.md: updated skill count to 25, added data-research to skills table.

CHANGELOG.md: added Infrastructure section documenting resolver validation,
doctor expansion, adaptive throttling, fail-improve loop, voice transcription,
enrichment service, and data-research skill.

TODOS.md: anonymized personal references.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: doctor.ts use ES module imports, harden backoff test

Replace require('fs') with ES module import in doctor.ts for consistency
with the rest of the file. Backoff test made resilient to parallel test
execution leaking module-level state.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: sync --watch routing, dead_links parity, doctor command, embed --slugs

- Move sync to CLI_ONLY so --watch flag reaches runSync() (was routed through
  operation layer which only calls performSync single-pass)
- Hide sync_brain from CLI help (MCP still exposes it)
- Fix performFullSync missing sync state persistence (C1)
- Align Postgres dead_links query to match PGLite (count dangling links, not
  empty-content chunks) (C3)
- Fix doctor recommending nonexistent 'gbrain embed refresh' (C4)
- Refactor doctor outputResults to not call process.exit directly
- Add --slugs flag to embed for targeted page embedding
- Add sync auto-extract + auto-embed after performSync
- Add noExtract to SyncOpts
- Route extract, features, autopilot in CLI_ONLY
- Update help text with new commands

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: extract, features, and autopilot commands

- gbrain extract <links|timeline|all> — batch extraction of links and timeline
  entries from brain markdown files. Broad regex for all .md links (C7: filters
  external URLs). Frontmatter field parsing (company, investors, attendees).
  Directory-based link type inference. JSONL progress on stderr for agents.
  Sync integration hooks (extractLinksForSlugs, extractTimelineForSlugs).

- gbrain features [--json] [--auto-fix] — scan brain usage, pitch unused features
  with the user's own numbers. Priority 1 (data quality): missing embeddings,
  dead links. Priority 2 (unused features): zero links, zero timeline, low
  coverage, unconfigured integrations, no sync. Embedded recipe metadata for
  binary-safe integration detection. Persistence in ~/.gbrain/feature-offers.json.
  Doctor teaser hook. Upgrade hook.

- gbrain autopilot [--repo] [--interval N] — self-maintaining brain daemon.
  Pipeline: sync → extract → embed. Health-based adaptive scheduling
  (brain_score >= 90 doubles interval, < 70 halves it). --install/--uninstall
  for launchd (macOS) and crontab (Linux). Signal handling. Consecutive error
  tracking (stops at 5). Log to ~/.gbrain/autopilot.log.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: hook features scan into post-upgrade flow

After gbrain post-upgrade completes, automatically run gbrain features to show
the user what's new and what to fix. Best-effort (doesn't fail the upgrade).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: brain_score (0-100) in BrainHealth

Weighted composite score computed in getHealth() for both Postgres and PGLite:
  embed_coverage: 0.35, link_density: 0.25, timeline_coverage: 0.15,
  no_orphans: 0.15, no_dead_links: 0.10

Returns 0 for empty brains. Agents use brain_score as a health gate.
Autopilot uses it for adaptive scheduling (>=90 slows down, <70 speeds up).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: extract and features unit tests

25 tests covering:
- extractMarkdownLinks: relative links, external URL filtering, edge cases
- extractLinksFromFile: slug resolution, frontmatter parsing, directory-based
  type inference (works_at, deal_for, invested_in)
- extractTimelineFromContent: bullet format, header format with detail,
  em/en dash handling, empty content
- features: module exports, brain_score calculation weights, CLI routing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: instruction layer for extract, features, autopilot

Agent-facing tools are invisible without instruction-layer coverage.
- RESOLVER.md: add routing for extract, features, autopilot
- maintain/SKILL.md: add link graph extraction, timeline extraction,
  autopilot check sections

Without these, agents reading skills/ will never discover or run the
new commands. This is the #1 DX finding from the devex review.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.10.1)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: sync CLAUDE.md with v0.10.1 additions

Add extract.ts, features.ts, autopilot.ts to key files.
Add extract.test.ts, features.test.ts to test list.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: adversarial review fixes — 7 issues

- #3: autopilot extract step was a no-op (imported but never called)
- #6: PGLite orphan_pages query aligned with Postgres (check both inbound+outbound)
- #8: embedPage throws instead of process.exit (was killing sync/autopilot)
- #9: dead-links set auto_fixable=false (needs repo path we may not have)
- #10: JSON auto-fix output was dead code (unreachable !jsonMode check)
- #14: autopilot lock file prevents concurrent instances
- #20: --dir without value no longer crashes extract

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* security: fix command injection + plaintext API key in daemon install

- #1: Crontab install used echo pipe with shell-interpolated values.
  Now uses a temp file via crontab(1) and single-quote escaping on all
  interpolated paths. No shell expansion possible.

- #2: OPENAI_API_KEY was baked as plaintext into the launchd plist
  (readable by any local process, backed up by Time Machine). Now uses
  a wrapper script (~/.gbrain/autopilot-run.sh) that sources ~/.zshrc
  at runtime. No secrets in plist or crontab.

- #16: extract.ts used a custom 20-line YAML parser that only handled
  single-line key:value pairs. Multi-line arrays (attendees list with
  - items) were silently ignored. Now uses the project's gray-matter
  parser via parseMarkdown() from src/core/markdown.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 21:40:48 -10:00

9.4 KiB

name, version, description, triggers, tools, mutating
name version description triggers tools mutating
maintain 1.0.0 Brain health checks: back-link enforcement, citation audit, filing validation, stale info detection, orphan pages, and benchmarks. Use when asked to check brain health, run maintenance, or audit quality.
brain health
check backlinks
citation audit
maintenance
orphan pages
stale pages
get_health
get_page
put_page
list_pages
get_backlinks
add_link
search
true

Maintain Skill

Periodic brain health checks and cleanup.

Contract

This skill guarantees:

  • All health dimensions are checked (stale, orphan, dead links, cross-refs, backlinks, citations, filing, tags)
  • Each issue found has a specific fix action
  • Back-link iron law is enforced
  • Citation format is validated against the standard
  • Results are reported with counts per dimension

Phases

  1. Run health check. Check gbrain health to get the dashboard.
  2. Check each dimension:

Stale pages

Pages where compiled_truth is older than the latest timeline entry. The assessment hasn't been updated to reflect recent evidence.

  • Check the health output for stale page count
  • For each stale page: read the page from gbrain, review timeline, determine if compiled_truth needs rewriting

Orphan pages

Pages with zero inbound links. Nobody references them.

  • Review orphans: are they genuinely isolated or just missing links?
  • Add links in gbrain from related pages or flag for deletion

Links pointing to pages that don't exist.

  • Remove dead links in gbrain

Missing cross-references

Pages that mention entity names but don't have formal links.

  • Read compiled_truth from gbrain, extract entity mentions, create links in gbrain

If link_count is 0 or low relative to page_count, run batch extraction:

gbrain extract links --dir ~/brain

This scans all markdown files for entity references, See Also sections, and frontmatter fields, then creates typed links in the database.

Timeline extraction

If timeline_entry_count is 0, extract structured timeline from markdown:

gbrain extract timeline --dir ~/brain

Parses - **YYYY-MM-DD** | Source — Summary and ### YYYY-MM-DD — Title formats. Note: extracted entries improve structured queries (gbrain timeline), not vector search.

Autopilot check

Verify autopilot is running:

gbrain autopilot --status

If not running, install it:

gbrain autopilot --install --repo ~/brain

Autopilot runs sync, extract, and embed in a continuous loop with adaptive scheduling.

Check that the back-linking iron law is being followed:

  • For each recently updated page, check if entities mentioned in it have corresponding back-links FROM those entity pages
  • A mention without a back-link is a broken brain
  • Fix: add the missing back-link to the entity's Timeline or See Also section
  • Format: - **YYYY-MM-DD** | Referenced in [page title](path) -- brief context

Filing rule violations

Check for common misfiling patterns (see skills/_brain-filing-rules.md):

  • Content with clear primary subjects filed in sources/ instead of the appropriate directory (people/, companies/, concepts/, etc.)
  • Use gbrain search to find pages in sources/ that reference specific people, companies, or concepts -- these may be misfiled
  • Flag misfiled pages for review or re-filing

Citation audit

Spot-check pages for missing [Source: ...] citations:

  • Read 5-10 recently updated pages
  • Check that compiled truth (above the line) has inline citations
  • Check that timeline entries have source attribution
  • Flag pages where facts appear without provenance

Tag consistency

Inconsistent tagging (e.g., "vc" vs "venture-capital", "ai" vs "artificial-intelligence").

  • Standardize to the most common variant using gbrain tag operations

Embedding freshness

Chunks without embeddings, or chunks embedded with an old model.

  • For large embedding refreshes (>1000 chunks), use nohup: nohup gbrain embed refresh > /tmp/gbrain-embed.log 2>&1 &
  • Then check progress: tail -1 /tmp/gbrain-embed.log

Security (RLS verification)

Run gbrain doctor --json and check the RLS status. All tables should show RLS enabled. If not, run gbrain init again.

Schema health

Check that the schema version is up to date. gbrain doctor --json reports the current version vs expected. If behind, gbrain init runs migrations automatically.

File storage health

Check the integrity of stored files and redirect pointers:

  • Run gbrain files verify to check all DB records have valid data
  • Run gbrain files status to see migration state (local, mirrored, redirected)
  • Check for orphan .redirect.yaml pointers that reference missing storage files
  • Check for large binary files (>= 100 MB) still in git that should be in cloud storage
  • If storage backend is configured: verify redirect pointers resolve (download test)

Open threads

Timeline items older than 30 days with unresolved action items.

  • Flag for review

Benchmark Testing

Periodically verify search quality hasn't regressed. Run a battery of test queries across difficulty tiers:

  • Tier 1 (entity lookup): known names -- should always resolve
  • Tier 2 (topic recall): concepts, topics -- keyword search should handle
  • Tier 3 (semantic): queries with no exact keyword match -- needs embeddings
  • Tier 4 (cross-domain): relational/connection queries -- only semantic handles

Compare results from gbrain search (keyword) vs gbrain query (hybrid). Quality matters more than speed (2.5s right > 200ms wrong).

When to run benchmarks:

  • After major brain imports or re-imports
  • After gbrain version upgrades
  • After embedding regeneration
  • Monthly to track quality drift

Heartbeat Integration

For production agents running on a schedule, integrate gbrain health checks into your operational heartbeat.

On every heartbeat (hourly or per-session)

Run gbrain doctor --json and check for degradation. Report any failing checks to the user. Key signals: connection health, schema version, RLS status, embedding staleness.

Weekly maintenance

Run gbrain embed --stale to refresh embeddings for pages that have changed since their last embedding. For large brains (>5000 pages), run this with nohup:

nohup gbrain embed --stale > /tmp/gbrain-embed.log 2>&1 &

Daily verification

Verify sync is running: check gbrain stats and confirm last_sync is within the last 24 hours. If sync has stopped, the brain is drifting from the repo.

Stale compiled truth detection

Flag pages where compiled truth is >30 days old but the timeline has recent entries. This means new evidence exists that hasn't been synthesized. These pages need a compiled truth rewrite (see the maintain workflow above).

Report Storage

After maintenance runs, save a report:

  • Health check results (before/after scores for each dimension)
  • Back-link violations found and fixed
  • Filing rule violations found
  • Citation gaps flagged
  • Benchmark results (if run)
  • Outstanding issues requiring user attention

This creates an audit trail for brain health over time.

Quality Rules

  • Never delete pages without confirmation
  • Log all changes via timeline entries
  • Check gbrain health before and after to show improvement

Anti-Patterns

  • Fixing pages without reading them first -- you must understand context before editing
  • Silently skipping dimensions -- every dimension must be checked and reported, even if clean
  • Deleting orphan pages without checking if they should be linked instead
  • Running embedding refresh during peak usage hours
  • Batch-fixing back-links without verifying the relationship is real
  • Marking a dimension "clean" without actually querying it
  • Rewriting compiled truth without reading the full timeline first
  • Removing tags without checking if other pages use the same tag consistently

Output Format

The maintenance report follows this structure:

## Brain Health Report — YYYY-MM-DD

| Dimension           | Issues Found | Fixed | Remaining |
|----------------------|-------------|-------|-----------|
| Stale pages          | N           | N     | N         |
| Orphan pages         | N           | N     | N         |
| Dead links           | N           | N     | N         |
| Missing cross-refs   | N           | N     | N         |
| Back-link violations | N           | N     | N         |
| Citation gaps        | N           | N     | N         |
| Filing violations    | N           | N     | N         |
| Tag inconsistencies  | N           | N     | N         |
| Embedding staleness  | N           | N     | N         |
| Security (RLS)       | N           | N     | N         |
| Schema health        | N           | N     | N         |
| File storage         | N           | N     | N         |
| Open threads         | N           | N     | N         |

### Details
[Per-dimension breakdown with specific pages and actions taken]

### Benchmark Results (if run)
[Tier 1-4 query results with pass/fail]

### Outstanding Issues
[Items requiring user attention or confirmation]

Tools Used

  • Check gbrain health (get_health)
  • List pages in gbrain with filters (list_pages)
  • Read a page from gbrain (get_page)
  • Check backlinks in gbrain (get_backlinks)
  • Link entities in gbrain (add_link)
  • Remove links in gbrain (remove_link)
  • Tag a page in gbrain (add_tag)
  • Remove a tag in gbrain (remove_tag)
  • View timeline in gbrain (get_timeline)