diff --git a/recipes/twilio-voice-brain.md b/recipes/twilio-voice-brain.md index dd33bb9..91e855c 100644 --- a/recipes/twilio-voice-brain.md +++ b/recipes/twilio-voice-brain.md @@ -1,8 +1,8 @@ --- id: twilio-voice-brain name: Voice-to-Brain -version: 0.8.0 -description: Phone calls create brain pages via Twilio + OpenAI Realtime + GBrain MCP. Callers talk, brain pages appear. +version: 0.8.1 +description: Phone calls create brain pages via Twilio + voice pipeline + GBrain MCP. Two architectures -- OpenAI Realtime (turnkey) or DIY STT+LLM+TTS (full control). Callers talk, brain pages appear. category: sense requires: [ngrok-tunnel] secrets: @@ -52,6 +52,9 @@ auth token is incorrect. Let's re-enter it." ## Architecture +Two pipeline options: + +### Option A: OpenAI Realtime (turnkey, simpler) ``` Caller (phone) ↓ Twilio (WebSocket, g711_ulaw audio — no transcoding) @@ -64,6 +67,33 @@ Brain page created (meetings/YYYY-MM-DD-call-{caller}.md) Summary posted to messaging app (Telegram/Slack/Discord) ``` +### Option B: DIY STT+LLM+TTS (full control, production-grade) +``` +Caller (phone or WebRTC browser) + ↓ Twilio WebSocket OR WebRTC +Voice Server (Node.js) + ↓ Deepgram STT (streaming speech-to-text, speaker diarization) + ↓ Claude API (streaming SSE, sentence-boundary dispatch) + ↓ Cartesia / OpenAI TTS (text-to-speech, low latency) + ↓ Function calls during conversation +GBrain MCP (semantic search, page reads, page writes) + ↓ Post-call +Brain page + audio upload + transcript storage +``` + +**Why v2 (Option B)?** OpenAI Realtime is a black box — you can't control STT +quality, swap LLMs, or debug audio issues. The DIY stack gives you transparent +Deepgram+Claude+TTS with full control over each stage. Trade-off: more integration +work, but you own the pipeline. + +**Production-tested v2 architecture (pipeline.mjs, ~250 lines):** +- Streaming SSE from Claude with sentence-boundary TTS dispatch +- 20-turn conversation history cap (prevents context bloat) +- Reconnect logic with exponential backoff on STT/TTS disconnects +- Periodic keepalives to prevent WebSocket timeout +- Audio endpointing for natural turn-taking +- Smart VAD (Silero) as default with push-to-talk fallback + ## Opinionated Defaults These are production-tested defaults from a real deployment. Customize after setup. @@ -428,7 +458,7 @@ fi ```bash mkdir -p ~/.gbrain/integrations/twilio-voice-brain -echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","event":"setup_complete","source_version":"0.7.0","status":"ok","details":{"phone":"TWILIO_NUMBER","deployment":"local+ngrok"}}' >> ~/.gbrain/integrations/twilio-voice-brain/heartbeat.jsonl +echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","event":"setup_complete","source_version":"0.8.1","status":"ok","details":{"phone":"TWILIO_NUMBER","deployment":"local+ngrok"}}' >> ~/.gbrain/integrations/twilio-voice-brain/heartbeat.jsonl ``` Tell the user: "Voice-to-brain is fully set up. Your number is [NUMBER]. Here's @@ -472,6 +502,95 @@ The watchdog restarts the server if it crashes." - The watchdog (Step 9) handles this automatically - For a permanent URL: upgrade to ngrok paid ($8/mo) for a static domain, or deploy to Fly.io/Railway instead +**Note on Option B credentials:** If using the DIY pipeline (Option B), you will +also need API keys for your chosen STT provider (e.g., Deepgram) and TTS provider +(e.g., Cartesia, OpenAI TTS). Collect and validate these during Step 2 alongside +the Twilio and OpenAI credentials listed above. + +## Critical Production Fixes (v0.8.1) + +These are NOT optional. They prevent real production failures discovered in a +deployment handling daily calls. + +### Unicode Crash Fix (CRITICAL) + +**Problem:** Em dashes (--), arrows (->), and other non-ASCII characters in the +prompt context cause broken surrogate pairs that crash the Twilio WebSocket +connection. Phone calls drop silently. + +**Fix:** Replace ALL non-ASCII characters with ASCII equivalents throughout the +entire prompt file before sending to Twilio. This is invisible in development +(browsers handle unicode fine) and catastrophic in production. + +```javascript +function sanitizeForTwilio(text) { + return text + .replace(/[\u2014\u2013]/g, '--') // em/en dash + .replace(/[\u2018\u2019]/g, "'") // smart quotes + .replace(/[\u201C\u201D]/g, '"') // smart double quotes + .replace(/\u2192/g, '->') // right arrow + .replace(/\u2190/g, '<-') // left arrow + .replace(/[\u2026]/g, '...') // ellipsis + .replace(/[^\x00-\x7F]/g, '') // strip remaining non-ASCII +} +``` + +### PII Scrub from Voice Context (CRITICAL) + +**Problem:** Brain context loaded into the voice prompt may contain phone numbers, +email addresses, and other PII. The voice agent reads these aloud to callers. + +**Fix:** Regex-strip PII from all voice context before injecting into the prompt: +- Phone numbers: `/\+?\d[\d\s\-().]{7,}\d/g` +- Email addresses: `/[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g` +- URLs with auth tokens or API keys +- Any string matching common credential patterns + +### Identity-First Prompt (IMPORTANT) + +**Problem:** Voice agents lose their identity mid-conversation. Saying "You are NOT +Claude" doesn't stick. The model reverts to its base persona. + +**Fix:** Put identity FIRST in the system prompt, before any context or rules: +``` +# You ARE [Agent Name] +You are [Name], a voice assistant who works with [Brain Name]. +You are NOT Claude. You are NOT a general AI assistant. +[Name] has their own personality: [traits]. + +# Context +[... brain context, calendar, tasks ...] + +# Rules +[... behavioral rules ...] +``` + +Positioning identity before context ensures the model sees it first and +maintains it throughout the conversation. + +### Auto-Upload Call Audio (RECOMMENDED) + +**Problem:** If post-call processing fails, the call audio is lost forever. + +**Fix:** Auto-upload ALL call audio immediately on call end: +- Twilio calls: download the MP3 recording URL from Twilio +- WebRTC calls: capture via MediaRecorder (webm/opus format) +- Upload to cloud storage (Supabase Storage, S3, etc.) +- Store `.redirect.yaml` pointer in brain repo if >= 100 MB +- This ensures every call has a recoverable audio source regardless + of whether the transcript or brain page was created successfully + +### Smart VAD as Default + +**Problem:** Push-to-talk is unnatural on phone calls. Server-side VAD has +variable quality. + +**Fix:** Default to Smart VAD (Silero VAD) for voice activity detection: +- Better endpointing than server-side VAD +- Fewer false triggers in noisy environments +- PTT available as fallback (UI toggle for WebRTC clients) +- Presets: quiet (0.7 threshold), normal (0.85), noisy (0.95), very_noisy (0.98) + ## Production Patterns (Recommended) These patterns come from a production voice deployment handling real calls daily. @@ -488,13 +607,13 @@ AI brain. "I work with [Brain], [Owner]'s AI." Lighter, more playful, more curio #### Pre-Computed Bid System **Problem:** Dead air kills engagement. Voice agents wait passively. **Pattern:** At call start, scan live context and pre-compute up to 10 engagement bids. -Two types: informative (tasks, calendar, social radar) and relational (curiosity templates). +Two types: informative (tasks, calendar, social monitoring) and relational (curiosity templates). Bids go INTO the prompt so the agent picks from a list. Use bids #1 and #2 for greeting, cycle the rest during conversation. Never ask "anything else?" — bring up the next bid. #### Context-First Prompt **Problem:** Voice agent greets generically because it doesn't know what's happening today. -**Pattern:** Load live context at call start: tasks, calendar, location, social radar, +**Pattern:** Load live context at call start: tasks, calendar, location, social monitoring, morning briefing. Position context FIRST in the prompt (before rules) so the model sees it immediately and uses it in the greeting. Try/catch per section. Cap 500-1000 chars each. @@ -658,7 +777,7 @@ over WebRTC data channel — use Whisper post-call instead. | Keyword | Report Loaded | |---------|--------------| | email, inbox, mail | inbox sweep report | -| social, twitter, mentions | social radar report | +| social, twitter, mentions | social engagement report | | briefing, morning | morning briefing | | meeting | meeting sync report | | slack | slack scan report | diff --git a/recipes/x-to-brain.md b/recipes/x-to-brain.md index e1a5306..afe113c 100644 --- a/recipes/x-to-brain.md +++ b/recipes/x-to-brain.md @@ -1,8 +1,8 @@ --- id: x-to-brain name: X-to-Brain -version: 0.7.0 -description: Twitter timeline, mentions, and keyword monitoring flow into brain pages. Tracks deletions and engagement velocity. +version: 0.8.1 +description: Twitter timeline, mentions, and keyword monitoring flow into brain pages. Tracks deletions, engagement velocity, OCR on images, and real-time alerts. category: sense requires: [] secrets: @@ -201,7 +201,99 @@ The agent should review collected data 2-3x daily and run enrichment. ```bash mkdir -p ~/.gbrain/integrations/x-to-brain -echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","event":"setup_complete","source_version":"0.7.0","status":"ok","details":{"user_id":"X_USER_ID"}}' >> ~/.gbrain/integrations/x-to-brain/heartbeat.jsonl +echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","event":"setup_complete","source_version":"0.8.1","status":"ok","details":{"user_id":"X_USER_ID"}}' >> ~/.gbrain/integrations/x-to-brain/heartbeat.jsonl +``` + +## Production Patterns (v0.8.1) + +These patterns come from a production deployment tracking 19+ accounts with +real-time monitoring. + +### Image OCR (NEW) + +**Problem:** Text-only collection misses visual context in tweet images -- +screenshots, charts, memes with text overlay, quote screenshots. + +**Fix:** Run OCR on tweet images via a vision model (Claude Sonnet or equivalent): +- For every tweet with images, extract full text content via vision API +- Store OCR output alongside the tweet data +- Include extracted text in entity detection and brain page updates +- Charts/data visualizations: extract data points, describe findings + +This catches signal that text-only collectors miss entirely. + +### Real-Time Monitoring via Filtered Stream (NEW) + +**Problem:** 30-minute polling means you find out about things 30 minutes late. +For time-sensitive content (engagement spikes, deletions, breaking threads), +that's too slow. + +**Fix:** Use Twitter's Filtered Stream API (`GET /2/tweets/search/stream`) for +near-real-time monitoring. Catches outbound tweets within seconds. + +**Setup:** +1. Add filter rules: `POST /2/tweets/search/stream/rules` with your tracking terms +2. Open persistent connection: `GET /2/tweets/search/stream` +3. Process tweets as they arrive (no polling delay) + +**Requirements:** Basic tier ($200/mo) minimum for Filtered Stream access. + +**Use alongside polling:** Stream for real-time alerts, polling for completeness +(stream can drop tweets during disconnects). + +### Tweet Rating Rubric (NEW) + +**Problem:** Not all tweets deserve the same attention. Without scoring, every +tweet gets equal weight. + +**Fix:** Rate tweets on a 6-dimension rubric: +1. **Reach** -- follower count, engagement rate +2. **Relevance** -- connection to your interests/work +3. **Sentiment** -- positive/negative/neutral toward you +4. **Novelty** -- new information vs rehash +5. **Actionability** -- does this require a response? +6. **Virality potential** -- engagement velocity, quote-tweet ratio + +Re-rate after 60 minutes to track engagement trajectory. A tweet at 50 likes +that hits 500 in an hour is a different signal than one that stays at 50. + +### Outbound Tweet Monitoring (NEW) + +**Problem:** You tweet something and don't notice engagement patterns until +hours later. + +**Fix:** 60-second monitoring window after every outbound tweet: +- Check engagement velocity (likes, replies, quotes) +- Flag unusual reply-to-like ratios (high reply ratios signal controversy) +- Flag if quote-tweet ratio > retweet ratio (commentary, not sharing) +- Cross-reference mentioned accounts against brain for context + +### X-to-Brain Pipeline (NEW) + +Every tweet interaction can automatically create/update brain pages: +- Mentioned person has a brain page? Append to their timeline +- New person mentioned? Check notability gate, create page if notable +- Article URL in tweet? Fetch and ingest via article workflow +- Video URL in tweet? Queue for transcription pipeline +- Images? OCR and extract text content + +Follow `skills/_brain-filing-rules.md` for filing decisions. + +### Cron Staggering (IMPORTANT) + +**Problem:** Multiple cron jobs firing simultaneously causes resource contention +and timeouts. + +**Fix:** Stagger all collection schedules so max 1 runs per minute: +``` +# Good: staggered +*/30 * * * * x-collector # :00, :30 +5,35 * * * * x-bundle-ingest # :05, :35 +10 */3 * * * social-monitor # :10 every 3h + +# Bad: overlapping +*/30 * * * * x-collector +*/30 * * * * x-bundle-ingest # fires at same time! ``` ## Implementation Guide diff --git a/skills/_brain-filing-rules.md b/skills/_brain-filing-rules.md new file mode 100644 index 0000000..b8fbf0e --- /dev/null +++ b/skills/_brain-filing-rules.md @@ -0,0 +1,84 @@ +# Brain Filing Rules -- MANDATORY for all skills that write to the brain + +## The Rule + +The PRIMARY SUBJECT of the content determines where it goes. Not the format, +not the source, not the skill that's running. + +## Decision Protocol + +1. Identify the primary subject (a person? company? concept? policy issue?) +2. File in the directory that matches the subject +3. Cross-link from related directories +4. When in doubt: what would you search for to find this page again? + +## Common Misfiling Patterns -- DO NOT DO THESE + +| Wrong | Right | Why | +|-------|-------|-----| +| Analysis of a topic -> `sources/` | -> appropriate subject directory | sources/ is for raw data only | +| Article about a person -> `sources/` | -> `people/` | Primary subject is a person | +| Meeting-derived company info -> `meetings/` only | -> ALSO update `companies/` | Entity propagation is mandatory | +| Research about a company -> `sources/` | -> `companies/` | Primary subject is a company | +| Reusable framework/thesis -> `sources/` | -> `concepts/` | It's a mental model | +| Tweet thread about policy -> `media/` | -> `civic/` or `concepts/` | media/ is for content ops | + +## What `sources/` Is Actually For + +`sources/` is ONLY for: +- Bulk data imports (API dumps, CSV exports, snapshots) +- Raw data that feeds multiple brain pages (e.g., a guest export, contact sync) +- Periodic captures (quarterly snapshots, sync exports) + +If the content has a clear primary subject (a person, company, concept, policy +issue), it does NOT go in sources/. Period. + +## Notability Gate + +Not everything deserves a brain page. Before creating a new entity page: +- **People:** Will you interact with them again? Are they relevant to your work? +- **Companies:** Are they relevant to your work or interests? +- **Concepts:** Is this a reusable mental model worth referencing later? +- **When in doubt, DON'T create.** A missing page can be created later. + A junk page wastes attention and degrades search quality. + +## Iron Law: Back-Linking (MANDATORY) + +Every mention of a person or company with a brain page MUST create a back-link +FROM that entity's page TO the page mentioning them. This is bidirectional: +the new page links to the entity, AND the entity's page links back. + +Format for back-links (append to Timeline or See Also): +``` +- **YYYY-MM-DD** | Referenced in [page title](path/to/page.md) -- brief context +``` + +An unlinked mention is a broken brain. The graph is the intelligence. + +## Citation Requirements (MANDATORY) + +Every fact written to a brain page must carry an inline `[Source: ...]` citation. + +Three formats: +- **Direct attribution:** `[Source: User, {context}, YYYY-MM-DD]` +- **API/external:** `[Source: {provider}, YYYY-MM-DD]` or `[Source: {publication}, {URL}]` +- **Synthesis:** `[Source: compiled from {list of sources}]` + +Source precedence (highest to lowest): +1. User's direct statements (highest authority) +2. Compiled truth (pre-existing brain synthesis) +3. Timeline entries (raw evidence) +4. External sources (API enrichment, web search -- lowest) + +When sources conflict, note the contradiction with both citations. Don't +silently pick one. + +## Raw Source Preservation + +Every ingested item should have its raw source preserved for provenance: +- **< 100 MB** (text, PDFs, transcripts): store in the brain repo (git-tracked) + in a `.raw/` sidecar directory alongside the brain page +- **>= 100 MB** (video, audio, large datasets): upload to cloud storage and + store a `.redirect.yaml` pointer in the brain repo + +This ensures any derived brain page can be traced back to its original source. diff --git a/skills/briefing/SKILL.md b/skills/briefing/SKILL.md index 5ed1b33..bac8b5a 100644 --- a/skills/briefing/SKILL.md +++ b/skills/briefing/SKILL.md @@ -2,6 +2,9 @@ Compile a daily briefing from brain context. +> **Filing rule:** When the briefing creates or updates brain pages, +> follow `skills/_brain-filing-rules.md`. + ## Workflow 1. **Today's meetings.** For each meeting on the calendar: @@ -72,6 +75,18 @@ PEOPLE IN PLAY - [name] -- [why they're active] ``` +## Back-Linking During Briefing + +If the briefing creates or updates any brain pages (e.g., new meeting prep +pages, updated entity pages), the back-linking iron law applies: every entity +mentioned must have a back-link from their page. See `skills/_brain-filing-rules.md`. + +## Citation in Briefings + +When presenting facts from brain pages, include inline citations: +- "Jane is CTO of Acme [Source: people/jane-doe, updated 2026-04-01]" +- This lets the user trace any claim back to the brain page and assess freshness + ## Tools Used - Search gbrain by name (query) diff --git a/skills/enrich/SKILL.md b/skills/enrich/SKILL.md index 9acf0b9..6ec490f 100644 --- a/skills/enrich/SKILL.md +++ b/skills/enrich/SKILL.md @@ -1,39 +1,281 @@ # Enrich Skill -Enrich person and company pages from external APIs. +Enrich person and company pages from external sources. Scale effort to importance. -## Sources +> **Filing rule:** Read `skills/_brain-filing-rules.md` before creating any new page. -| Source | Data | API | -|--------|------|-----| -| Crustdata | LinkedIn profiles, company data | REST API | -| Happenstance | Career history, connections | REST API | -| Exa | Web mentions, articles | REST API | +## Iron Law: Back-Linking (MANDATORY) -Note: enrichment requires separate API credentials for each service. No client -integrations ship in v1. This skill guides the agent to make API calls directly. +Every mention of a person or company with a brain page MUST create a back-link +FROM that entity's page TO the page mentioning them. An unlinked mention is a +broken brain. See `skills/_brain-filing-rules.md` for format. -## Workflow +## Philosophy -1. **Select target pages.** List person or company pages in gbrain. -2. **For each page:** - - Read the page from gbrain to understand what we already know - - Call external APIs for fresh data - - Store raw API responses in gbrain (put_raw_data) to preserve provenance - - Distill highlights into compiled_truth updates - - Store the updated page in gbrain -3. **Validation rules:** - - Connection count < 20 on LinkedIn = likely wrong person, skip - - Name mismatch between brain and API = skip, flag for manual review - - Don't overwrite human-written assessments with API boilerplate +A brain page should read like an intelligence dossier, not a LinkedIn scrape. +Facts are table stakes. Texture is the value -- what do they believe, what are +they building, what makes them tick, where are they headed. -## Quality Rules +## Citation Requirements (MANDATORY) -- Raw data goes to gbrain's raw_data store (preserves provenance) -- Only distilled, useful info goes to compiled_truth -- Always add a timeline entry in gbrain: "Enriched from [source] on [date]" -- Don't enrich the same page more than once per week unless requested -- Rate limit: respect API rate limits, use exponential backoff +Every fact must carry an inline `[Source: ...]` citation. + +Three formats: +- **Direct attribution:** `[Source: User, {context}, YYYY-MM-DD]` +- **API/external:** `[Source: {provider} enrichment, YYYY-MM-DD]` +- **Synthesis:** `[Source: compiled from {list of sources}]` + +Source precedence (highest to lowest): +1. User's direct statements +2. Compiled truth (pre-existing brain synthesis) +3. Timeline entries (raw evidence) +4. External sources (API enrichment, web search) + +When sources conflict, note the contradiction with both citations. + +## When To Enrich + +### Primary triggers +- User mentions an entity in conversation +- Entity appears in a meeting transcript or email +- New contact appears with significant context +- Entity makes news or has a major event +- Any ingest pipeline encounters a notable entity + +### Do NOT enrich +- Random mentions with no relationship signal +- Bot/spam accounts +- Entities with no substantive connection to the user's work +- Same page enriched within the past week (unless new signal warrants it) + +## Enrichment Tiers + +Scale enrichment to importance. Don't waste API calls on low-value entities. + +| Tier | Who | Effort | Sources | +|------|-----|--------|---------| +| 1 (key) | Inner circle, close collaborators, key contacts | Full pipeline | All available APIs + deep web research | +| 2 (notable) | Occasional interactions, industry figures | Moderate | Web research + social + brain cross-ref | +| 3 (minor) | Worth tracking, not critical | Light | Brain cross-ref + social lookup if handle known | + +## The Enrichment Protocol (7 Steps) + +### Step 1: Identify entities + +Extract people, companies, concepts from the incoming signal. + +### Step 2: Check brain state + +For each entity: +- `gbrain search "name"` -- does a page already exist? +- **If yes:** UPDATE path (add new signal, update compiled truth if material) +- **If no:** CREATE path (check notability gate first, then create) + +### Step 3: Extract signal from source + +Don't just capture facts. Capture texture: + +| Signal Type | What to Extract | +|-------------|----------------| +| Opinions, beliefs | What They Believe section | +| Current projects, features shipped | What They're Building section | +| Ambition, career arc, motivation | What Motivates Them section | +| Topics they return to obsessively | Hobby Horses section | +| Who they amplify, argue with, respect | Network / Relationships | +| Ascending, plateauing, pivoting? | Trajectory section | +| Role, company, funding, location | State section (hard facts) | + +### Step 4: External data source lookups + +Priority order -- stop when you have enough signal for the entity's tier. + +**4a. Brain cross-reference (always, all tiers)** +- `gbrain search "name"` and `gbrain query "what do we know about name"` +- Check related pages: company pages for person enrichment and vice versa +- This is free and often the richest source + +**4b. Web research (Tier 1 and 2)** +- Use Perplexity, Brave Search, Exa, or equivalent web research tool +- **Key pattern:** Send existing brain knowledge as context so the search + returns DELTA (what's new vs what you already know), not a rehash +- Opus-class models for Tier 1 deep research, lighter models for Tier 2 + +**4c. Social media lookup (all tiers when handle known)** +- Pull recent posts/tweets for tone, interests, current focus +- Social media is the highest-texture signal for what someone actually thinks + +**4d. People enrichment APIs (Tier 1)** +- LinkedIn data, career history, connections, education + +**4e. Company enrichment APIs (Tier 1)** +- Company data, financials, headcount, key hires, recent news + +| Data Need | Example Sources | Tier | +|-----------|----------------|------| +| Web research | Perplexity, Brave, Exa | 1-2 | +| LinkedIn / career | Crustdata, Proxycurl, People Data Labs | 1 | +| Career history | Happenstance, LinkedIn | 1 | +| Funding / company data | Crunchbase, PitchBook, Clearbit | 1 | +| Social media | Platform APIs, web scraping | 1-3 | +| Meeting history | Calendar/meeting transcript tools | 1-2 | + +### Step 5: Save raw data (preserves provenance) + +Store raw API responses via `put_raw_data` in gbrain: +```json +{ + "source": "crustdata", + "fetched_at": "2026-04-11T...", + "query": "jane doe", + "data": { ... } +} +``` + +Raw data preserves provenance. If the compiled truth is ever questioned, +the raw data shows exactly what the API returned. + +### Step 6: Write to brain + +#### CREATE path + +1. Check notability gate (see `skills/_brain-filing-rules.md`) +2. Check filing rules -- where does this entity go? +3. Create page with the appropriate template (below) +4. Fill compiled truth with citations +5. Add first timeline entry +6. Leave empty sections as `[No data yet]` (don't fill with boilerplate) + +#### UPDATE path + +1. Add new timeline entries (reverse-chronological, append-only) +2. Update compiled truth ONLY if the new signal materially changes the picture +3. Update State section with new facts +4. Flag contradictions between new signal and existing compiled truth +5. Don't overwrite user-written assessments with API boilerplate + +#### Person page template + +```markdown +--- +title: Full Name +type: person +created: YYYY-MM-DD +updated: YYYY-MM-DD +tags: [] +company: Current Company +relationship: How the user knows them +email: +linkedin: +twitter: +location: +--- + +# Full Name + +> 1-paragraph executive summary: HOW do you know them, WHY do they matter, +> what's the current state of the relationship. + +## State +Role, company, key context. Hard facts only. + +## What They Believe +Ideology, first principles, worldview. What hills do they die on? + +## What They're Building +Current projects, recent launches, what they're focused on. + +## What Motivates Them +Ambition, career arc, what drives them. + +## Hobby Horses +Topics they return to obsessively. Recurring themes in their work/posts. + +## Assessment +Your read on this person. Strengths, gaps, trajectory. + +## Trajectory +Ascending, plateauing, pivoting, declining? Where are they headed? + +## Relationship +History of interactions, shared context, relationship quality. + +## Contact +Email, social handles, preferred communication channel. + +## Network +Key connections, mutual contacts, organizational relationships. + +## Open Threads +Active conversations, pending items, things to follow up on. + +--- + +## Timeline +Reverse chronological. Every entry has a date and [Source: ...] citation. +- **YYYY-MM-DD** | Event description [Source: ...] +``` + +#### Company page template + +```markdown +--- +title: Company Name +type: company +created: YYYY-MM-DD +updated: YYYY-MM-DD +tags: [] +--- + +# Company Name + +> 1-paragraph executive summary. + +## State +What they do, stage, key people, key metrics, your connection. + +## Open Threads +Active items, pending decisions, things to track. + +--- + +## Timeline +- **YYYY-MM-DD** | Event description [Source: ...] +``` + +### Step 7: Cross-reference + +- Update company pages from person enrichment (and vice versa) +- Update related project/deal pages if relevant context surfaced +- Add back-links from every entity mentioned (MANDATORY) +- Check index files if the brain uses them + +## Bulk Enrichment Rules + +- **Test on 3-5 entities first.** Read actual output. Check quality. +- Only proceed to bulk after test shots pass your quality bar. +- 3+ entities from one source -> batch process or spawn sub-agent +- Throttle API calls. Respect rate limits. +- Commit every 5-10 entities during bulk runs. +- Save a report after bulk enrichment (see Report Storage below). + +## Validation Rules + +- Connection count < 20 on LinkedIn = likely wrong person, skip +- Name mismatch between brain and API = skip, flag for review +- Joke profiles or obviously wrong data = save to raw, don't update page +- Don't overwrite user-written assessments with API boilerplate +- When in doubt: save raw data but don't update brain page + +## Report Storage + +After enrichment sweeps, save a report: +- Number of entities processed +- New pages created vs existing updated +- Data sources called and results quality +- Notable discoveries or contradictions +- Validation flags or API failures + +This creates an audit trail for brain enrichment over time. ## Tools Used @@ -43,3 +285,5 @@ integrations ship in v1. This skill guides the agent to make API calls directly. - List pages in gbrain by type (list_pages) - Store raw API data in gbrain (put_raw_data) - Retrieve raw data from gbrain (get_raw_data) +- Link entities in gbrain (add_link) +- Check backlinks in gbrain (get_backlinks) diff --git a/skills/ingest/SKILL.md b/skills/ingest/SKILL.md index 48a31be..7c56f8c 100644 --- a/skills/ingest/SKILL.md +++ b/skills/ingest/SKILL.md @@ -1,6 +1,25 @@ # Ingest Skill -Ingest meetings, articles, documents, and conversations into the brain. +Ingest meetings, articles, media, documents, and conversations into the brain. + +> **Filing rule:** Read `skills/_brain-filing-rules.md` before creating any new page. + +## Iron Law: Back-Linking (MANDATORY) + +Every mention of a person or company with a brain page MUST create a back-link +FROM that entity's page TO the page mentioning them. An unlinked mention is a +broken brain. See `skills/_brain-filing-rules.md` for format. + +## Citation Requirements (MANDATORY) + +Every fact written to a brain page must carry an inline `[Source: ...]` citation. + +- **User's statements:** `[Source: User, {context}, YYYY-MM-DD]` +- **Meeting data:** `[Source: Meeting "{title}", YYYY-MM-DD]` +- **Email/message:** `[Source: email from {name} re: {subject}, YYYY-MM-DD]` +- **Web content:** `[Source: {publication}, {URL}, YYYY-MM-DD]` +- **Social media:** `[Source: X/@handle, YYYY-MM-DD](URL)` (include link) +- **Synthesis:** `[Source: compiled from {sources}]` ## Workflow @@ -8,10 +27,11 @@ Ingest meetings, articles, documents, and conversations into the brain. 2. **For each entity mentioned:** - Read the entity's page from gbrain to check if it exists - If exists: update compiled_truth (rewrite State section with new info, don't append) - - If new: store the page in gbrain with the appropriate type and slug -3. **Append to timeline.** Add a timeline entry in gbrain for each event, with date, summary, and source. + - If new: check notability gate, then store the page in gbrain with the appropriate type and slug +3. **Append to timeline.** Add a timeline entry in gbrain for each event, with date, summary, and source citation. 4. **Create cross-reference links.** Link entities in gbrain for every entity pair mentioned together, using the appropriate relationship type. -5. **Timeline merge.** The same event appears on ALL mentioned entities' timelines. If Alice met Bob at Acme Corp, the event goes on Alice's page, Bob's page, and Acme Corp's page. +5. **Back-link all entities.** Update EVERY mentioned entity's page with a back-link to this page (Iron Law). +6. **Timeline merge.** The same event appears on ALL mentioned entities' timelines. If Alice met Bob at Acme Corp, the event goes on Alice's page, Bob's page, and Acme Corp's page. ## Entity Detection on Every Message @@ -26,13 +46,11 @@ the signal detection loop that makes the brain compound over time. - `gbrain search "name"` -- does a page already exist? - **If yes:** load context with `gbrain get `. Use the compiled truth to inform your response. Update the page if the message contains new information. - - **If no:** assess notability. If the entity is worth tracking (will come up - again, is relevant to the user's world), create a new page with - `gbrain put ` and populate with what you know. -3. **After creating or updating pages:** commit changes to the brain repo, then - sync to gbrain: + - **If no:** assess notability (see `skills/_brain-filing-rules.md`). If the entity + is worth tracking, create a new page with `gbrain put ` and populate + with what you know. +3. **After creating or updating pages:** sync to gbrain: ```bash - git add brain/ && git commit -m "update entity pages" gbrain sync --no-pull --no-embed ``` 4. **Don't block the conversation.** Entity detection and enrichment should happen @@ -42,18 +60,172 @@ the signal detection loop that makes the brain compound over time. ### What counts as notable - People the user interacts with or discusses (not random mentions) -- Companies relevant to the user's work, investments, or interests +- Companies relevant to the user's work or interests - Concepts or frameworks the user references or creates - The user's own original thinking (ideas, theses, observations) -- highest value +- See `skills/_brain-filing-rules.md` for the full notability gate + +### What to capture from the user's own thinking + +Original thinking is the most valuable signal. Capture exact phrasing -- the user's +language IS the insight. Don't paraphrase. + +- Novel observations or theses +- Frameworks, mental models, heuristics +- Connections between ideas that others miss +- Contrarian positions with reasoning +- Strong reactions to external stimuli (what triggered it and why) + +## Media Workflows + +Content the user encounters should be captured in the brain. File by PRIMARY +SUBJECT, not by format (see `skills/_brain-filing-rules.md`). + +### Articles & Web Content + +**Input:** URL shared by user, or article mentioned in conversation. + +**Process:** +1. Fetch content (`web_fetch` or equivalent) +2. Extract: title, author, publication, date, full text +3. Summarize: executive summary + key arguments (not a rehash) +4. Extract entities: people, companies, concepts mentioned +5. **Save raw source** for provenance (see Raw Source Preservation below) +6. Analyze for the user: don't just summarize. What's interesting given what you + know about them? Flag connections, contradictions, content opportunities. + +**Write to:** appropriate directory per filing rules (about a person -> `people/`, +about a company -> `companies/`, reusable framework -> `concepts/`, raw data -> `sources/`) + +### Videos & Podcasts + +**Input:** URL (YouTube, podcast, etc.) or local audio/video file. + +**Process:** +1. Get transcript -- speaker-diarized if possible (services like Diarize.io provide + speaker-labeled, word-level timing) +2. **Save raw transcript** (both JSON and human-readable TXT) +3. Analyze: executive summary, key ideas, key quotes with speaker attribution, + notable stories/anecdotes, people and companies mentioned +4. Extract and cross-reference all entities mentioned +5. **HARD RULE:** every video/podcast brain page MUST link to the raw diarized + transcript. A page without transcript links is incomplete. + +**Write to:** `media/videos/` or `media/podcasts/` with back-links to all entities. + +**Quality bar:** +- Compelling headline (not "This video discusses...") +- Executive summary that makes you want to watch/listen +- Key Ideas as actual insights, not topic labels +- Verbatim quotes with real speaker names (not "speaker_0") +- All entities extracted with context and back-linked + +### PDFs & Documents + +**Input:** File path or URL. + +**Process:** +1. Extract text (OCR if scanned/image PDF) +2. **Save raw source** for provenance +3. Summarize: executive summary + key sections + notable data +4. Extract entities +5. Cross-reference from entity pages + +**Write to:** per filing rules (file by primary subject, not format). + +### Screenshots & Images + +**Input:** Image file. + +**Process:** +1. Analyze content (OCR for text-heavy images, description for photos) +2. If tweet screenshot: extract text, author, date, route to social media workflow +3. If article screenshot: extract text, route to article workflow +4. If data/chart: extract data points, describe findings + +**Write to:** depends on content -- route to the appropriate workflow above. + +### Meeting Transcripts + +**Input:** Transcript from meeting recording service, or manual notes. + +**Process:** +1. Pull full transcript (source of truth -- AI summaries are medium-low trust) +2. **Save raw transcript** for provenance +3. Write meeting page with YOUR analysis above the line, raw transcript below +4. **Entity propagation (MANDATORY):** for each attendee and company discussed: + - Update their brain page State section if new info surfaced + - Append to their Timeline with link to the meeting page + - Create page if person/company is notable and has no page yet +5. A meeting is NOT fully ingested until all entity pages are updated + +**Write to:** `meetings/YYYY-MM-DD-short-description.md` + +**What makes a good meeting page:** +- Reveals the real crux, not a bullet dump +- Connects to existing brain pages (people, companies, deals) +- Flags what changed (status, decisions, new info) +- Names tension or what was left unsaid +- Captures actual dynamic, not performative summary + +### Social Media Content + +**Input:** Tweet, thread, or social media post. + +**Process:** +1. Fetch full content (thread, quote tweets, context) +2. If images present: OCR via vision model for full text extraction +3. Summarize: what's being said, why it matters, who's involved +4. Extract entities and update brain pages +5. Include direct link to the original post (MANDATORY for citations) + +**Write to:** `media/x/` for daily aggregation, or entity-specific directories +if the post is primarily about a person/company. + +## Raw Source Preservation + +Every ingested item must have its raw source preserved for provenance. + +- **< 100 MB** (text, PDFs, transcripts): store in brain repo `.raw/` sidecar + directories alongside the brain page +- **>= 100 MB** (video, audio, large datasets): upload to cloud storage (Supabase + Storage, S3, etc.) and store a `.redirect.yaml` pointer in the brain repo + +The `.redirect.yaml` format: +```yaml +storage: supabase +bucket: brain-files +path: media/videos/raw/video-id.mp4 +uploaded: 2026-04-11T... +size_bytes: 524288000 +``` + +Use `put_raw_data` in gbrain to store raw API responses and metadata. + +## Test Before Bulk + +When processing multiple items (batch video ingestion, bulk meeting processing, etc.): + +1. **Test on 3-5 items first.** Run in test mode if available. +2. **Read the actual output.** Is the quality good? Are titles compelling (not + "This video discusses...")? Are entities extracted and back-linked? Is the + format clean? +3. **Fix what's wrong** in the approach/skill, not via one-off patches. +4. **Only then: bulk execute** with throttling, commits every 5-10 items. + +The marginal cost of testing 3 items first is near zero. The cost of cleaning +up 100 bad pages is enormous. ## Quality Rules - Executive summary in compiled_truth must be updated, not just timeline appended - State section is REWRITTEN, not appended to. Current best understanding only. - Timeline entries are reverse-chronological (newest first) -- Every person/company mentioned gets a page if one doesn't exist +- Every person/company mentioned gets a page if notable (see filing rules) - Link types: knows, works_at, invested_in, founded, met_at, discussed -- Source attribution: every timeline entry includes the source (meeting, article, email, etc.) +- Source attribution: every timeline entry includes [Source: ...] citation +- Back-links: every entity mention creates a back-link (Iron Law) +- Filing: file by primary subject, not format or source (see filing rules) ## Tools Used @@ -63,3 +235,5 @@ the signal detection loop that makes the brain compound over time. - Link entities in gbrain (add_link) - List tags for a page (get_tags) - Tag a page in gbrain (add_tag) +- Store raw data in gbrain (put_raw_data) +- Check backlinks in gbrain (get_backlinks) diff --git a/skills/maintain/SKILL.md b/skills/maintain/SKILL.md index 4b6f8c4..5156fd8 100644 --- a/skills/maintain/SKILL.md +++ b/skills/maintain/SKILL.md @@ -25,6 +25,29 @@ Links pointing to pages that don't exist. Pages that mention entity names but don't have formal links. - Read compiled_truth from gbrain, extract entity mentions, create links in gbrain +### Back-link enforcement +Check that the back-linking iron law is being followed: +- For each recently updated page, check if entities mentioned in it have + corresponding back-links FROM those entity pages +- A mention without a back-link is a broken brain +- Fix: add the missing back-link to the entity's Timeline or See Also section +- Format: `- **YYYY-MM-DD** | Referenced in [page title](path) -- brief context` + +### Filing rule violations +Check for common misfiling patterns (see `skills/_brain-filing-rules.md`): +- Content with clear primary subjects filed in `sources/` instead of the + appropriate directory (people/, companies/, concepts/, etc.) +- Use gbrain search to find pages in `sources/` that reference specific + people, companies, or concepts -- these may be misfiled +- Flag misfiled pages for review or re-filing + +### Citation audit +Spot-check pages for missing `[Source: ...]` citations: +- Read 5-10 recently updated pages +- Check that compiled truth (above the line) has inline citations +- Check that timeline entries have source attribution +- Flag pages where facts appear without provenance + ### Tag consistency Inconsistent tagging (e.g., "vc" vs "venture-capital", "ai" vs "artificial-intelligence"). - Standardize to the most common variant using gbrain tag operations @@ -48,6 +71,25 @@ automatically. Timeline items older than 30 days with unresolved action items. - Flag for review +## Benchmark Testing + +Periodically verify search quality hasn't regressed. Run a battery of test +queries across difficulty tiers: + +- **Tier 1 (entity lookup):** known names -- should always resolve +- **Tier 2 (topic recall):** concepts, topics -- keyword search should handle +- **Tier 3 (semantic):** queries with no exact keyword match -- needs embeddings +- **Tier 4 (cross-domain):** relational/connection queries -- only semantic handles + +Compare results from `gbrain search` (keyword) vs `gbrain query` (hybrid). +Quality matters more than speed (2.5s right > 200ms wrong). + +When to run benchmarks: +- After major brain imports or re-imports +- After gbrain version upgrades +- After embedding regeneration +- Monthly to track quality drift + ## Heartbeat Integration For production agents running on a schedule, integrate gbrain health checks into @@ -78,6 +120,18 @@ Flag pages where compiled truth is >30 days old but the timeline has recent entr This means new evidence exists that hasn't been synthesized. These pages need a compiled truth rewrite (see the maintain workflow above). +## Report Storage + +After maintenance runs, save a report: +- Health check results (before/after scores for each dimension) +- Back-link violations found and fixed +- Filing rule violations found +- Citation gaps flagged +- Benchmark results (if run) +- Outstanding issues requiring user attention + +This creates an audit trail for brain health over time. + ## Quality Rules - Never delete pages without confirmation diff --git a/skills/migrations/v0.8.1.md b/skills/migrations/v0.8.1.md new file mode 100644 index 0000000..9f37433 --- /dev/null +++ b/skills/migrations/v0.8.1.md @@ -0,0 +1,103 @@ +--- +version: 0.8.1 +feature_pitch: + headline: "Your brain skills learned from production" + description: "Back-linking iron law, filing rules, enrichment protocol, media ingest, citation requirements, voice crash fixes -- all battle-tested from real production deployments." +--- + +# v0.8.1 Migration: Battle-Tested Skill Patterns + +This migration updates your agent's brain-writing patterns and voice recipe. +No schema changes, no database migrations. Skills and recipes only. + +## What's New + +### Back-Linking Iron Law +Every skill that writes to the brain now enforces bidirectional linking. If a +page mentions a person or company, that entity's page MUST link back. This was +the single biggest knowledge graph integrity rule missing from v0.8.0. + +### Brain Filing Rules (`skills/_brain-filing-rules.md`) +Cross-cutting rules for where brain pages go. File by PRIMARY SUBJECT, not +format or source. Includes common misfiling patterns table and notability gate. + +### Enrichment Protocol (rewritten) +The enrich skill is now a full 7-step pipeline with 3-tier system (scale effort +to entity importance), person page section template (What They Believe, Building, +Motivates, Hobby Horses, Trajectory), pluggable data source table, and validation. + +### Media Ingestion Workflows +The ingest skill now handles articles, videos, podcasts, PDFs, screenshots, and +social media content. Each with a complete input->process->write workflow. + +### Citation Requirements +Every fact needs inline `[Source: ...]` citations across all skills. Three formats +defined (direct, API/external, synthesis) with source precedence hierarchy. + +### Voice Recipe Production Fixes +Critical fixes from a production voice deployment: unicode crash fix (non-ASCII +breaks Twilio WebSocket), PII scrub from voice context, identity-first prompt +architecture, Smart VAD default, auto-upload call audio, and the DIY STT+LLM+TTS +pipeline (Deepgram+Claude+Cartesia) as an alternative to OpenAI Realtime. + +### X-to-Brain Recipe Enhancements +Image OCR via vision models, real-time monitoring via Filtered Stream API, +6-dimension tweet rating rubric, outbound tweet monitoring, X-to-Brain automatic +pipeline, and cron staggering for resource management. + +## Agent Instructions + +### Step 1: Verify skills are updated + +Check that your skills directory has the new files: +```bash +ls skills/_brain-filing-rules.md && echo "Filing rules: OK" +grep -q '"version": "0.8.1"' skills/manifest.json && echo "Manifest: OK" +``` + +Files that should be updated: +- `skills/_brain-filing-rules.md` (new) +- `skills/enrich/SKILL.md` (rewritten) +- `skills/ingest/SKILL.md` (expanded) +- `skills/maintain/SKILL.md` (expanded) +- `skills/query/SKILL.md` (updated) +- `skills/briefing/SKILL.md` (updated) +- `recipes/twilio-voice-brain.md` (v0.8.1 production fixes) +- `recipes/x-to-brain.md` (v0.8.1 enhancements) + +### Step 2: Review existing brain pages for back-link gaps + +Spot-check: pick 5 recently created pages and verify that every entity mentioned +has a back-link from the entity's page to the mentioning page. + +If back-links are missing, add them: +``` +- **YYYY-MM-DD** | Referenced in [page title](path/to/page.md) -- brief context +``` + +### Step 3: Review filing + +Check for pages in `sources/` that should be in a subject-specific directory +(people/, companies/, concepts/, civic/). See `skills/_brain-filing-rules.md`. + +### Step 4: Voice recipe (if using) + +If you have voice-to-brain set up: +1. Apply the unicode sanitization fix to your voice server +2. Add PII scrub to your prompt builder +3. Move identity to the TOP of your system prompt +4. Consider switching to Smart VAD (Silero) as default + +### Step 5: X-to-brain recipe (if using) + +If you have X-to-brain set up: +1. Add image OCR to your tweet processing pipeline +2. Consider Filtered Stream API for real-time monitoring +3. Stagger your cron schedules (max 1 per minute) + +### Step 6: Done + +```bash +mkdir -p ~/.gbrain/migrations +echo '{"version":"0.8.1","ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","status":"complete"}' >> ~/.gbrain/migrations/completed.jsonl +``` diff --git a/skills/query/SKILL.md b/skills/query/SKILL.md index 48b6958..5e7c5db 100644 --- a/skills/query/SKILL.md +++ b/skills/query/SKILL.md @@ -49,6 +49,23 @@ When multiple sources provide conflicting information, follow this precedence: When sources conflict, note the contradiction with both citations. Don't silently pick one. +## Citation in Answers + +When referencing brain pages in your answer, propagate inline citations: +- Cite the page: "According to [Source: people/jane-doe, compiled truth]..." +- When brain pages have inline `[Source: ...]` citations, propagate them so + the user can trace facts to their origin +- When you synthesize across multiple pages, cite all sources + +## Search Quality Awareness + +If search results seem off (wrong results, missing known pages, irrelevant hits): +- Run `gbrain doctor --json` to check index health +- Check embedding coverage -- partial embeddings degrade hybrid search +- Compare keyword search (`gbrain search`) vs hybrid search (`gbrain query`) + for the same query to isolate whether the issue is embedding-related +- Report search quality issues in the maintain workflow (see maintain skill) + ## Tools Used - Keyword search gbrain (search)