triton6564685/gbrain
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: battle-tested skill patterns from production deployment

Browse Source 
Backport production-learned brain-operations patterns:
- Iron Law of Back-Linking (mandatory bidirectional linking)
- Brain filing rules (file by primary subject, not format)
- Enrichment protocol (7-step pipeline, 3-tier system, person/company templates)
- Media ingest workflows (articles, videos, podcasts, PDFs, screenshots)
- Citation requirements (mandatory [Source: ...] on every fact)
- Test Before Bulk operating principle
- Voice recipe: unicode crash fix, PII scrub, identity-first prompt, DIY STT+LLM+TTS
- X-to-Brain recipe: image OCR, Filtered Stream, tweet rating rubric, cron stagger
This commit is contained in:
Garry Tan
2026-04-11 17:07:19 -10:00
parent 91ced664b6
commit 1e6d7e3737
9 changed files with 951 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files

131
recipes/twilio-voice-brain.md
View File

@@ -1,8 +1,8 @@
--- ---
id: twilio-voice-brain id: twilio-voice-brain
name: Voice-to-Brain name: Voice-to-Brain
version: 0.8.0 version: 0.8.1
description: Phone calls create brain pages via Twilio + OpenAI Realtime + GBrain MCP. Callers talk, brain pages appear. description: Phone calls create brain pages via Twilio + voice pipeline + GBrain MCP. Two architectures -- OpenAI Realtime (turnkey) or DIY STT+LLM+TTS (full control). Callers talk, brain pages appear.
category: sense category: sense
requires: [ngrok-tunnel] requires: [ngrok-tunnel]
secrets: secrets:
@@ -52,6 +52,9 @@ auth token is incorrect. Let's re-enter it."
## Architecture ## Architecture
Two pipeline options:
### Option A: OpenAI Realtime (turnkey, simpler)
``` ```
Caller (phone) Caller (phone)
↓ Twilio (WebSocket, g711_ulaw audio — no transcoding) ↓ Twilio (WebSocket, g711_ulaw audio — no transcoding)
@@ -64,6 +67,33 @@ Brain page created (meetings/YYYY-MM-DD-call-{caller}.md)
Summary posted to messaging app (Telegram/Slack/Discord) Summary posted to messaging app (Telegram/Slack/Discord)
``` ```
### Option B: DIY STT+LLM+TTS (full control, production-grade)
```
Caller (phone or WebRTC browser)
↓ Twilio WebSocket OR WebRTC
Voice Server (Node.js)
↓ Deepgram STT (streaming speech-to-text, speaker diarization)
↓ Claude API (streaming SSE, sentence-boundary dispatch)
↓ Cartesia / OpenAI TTS (text-to-speech, low latency)
↓ Function calls during conversation
GBrain MCP (semantic search, page reads, page writes)
↓ Post-call
Brain page + audio upload + transcript storage
```
**Why v2 (Option B)?** OpenAI Realtime is a black box — you can't control STT
quality, swap LLMs, or debug audio issues. The DIY stack gives you transparent
Deepgram+Claude+TTS with full control over each stage. Trade-off: more integration
work, but you own the pipeline.
**Production-tested v2 architecture (pipeline.mjs, ~250 lines):**
- Streaming SSE from Claude with sentence-boundary TTS dispatch
- 20-turn conversation history cap (prevents context bloat)
- Reconnect logic with exponential backoff on STT/TTS disconnects
- Periodic keepalives to prevent WebSocket timeout
- Audio endpointing for natural turn-taking
- Smart VAD (Silero) as default with push-to-talk fallback
## Opinionated Defaults ## Opinionated Defaults
These are production-tested defaults from a real deployment. Customize after setup. These are production-tested defaults from a real deployment. Customize after setup.
@@ -428,7 +458,7 @@ fi
```bash ```bash
mkdir -p ~/.gbrain/integrations/twilio-voice-brain mkdir -p ~/.gbrain/integrations/twilio-voice-brain
echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","event":"setup_complete","source_version":"0.7.0","status":"ok","details":{"phone":"TWILIO_NUMBER","deployment":"local+ngrok"}}' >> ~/.gbrain/integrations/twilio-voice-brain/heartbeat.jsonl echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","event":"setup_complete","source_version":"0.8.1","status":"ok","details":{"phone":"TWILIO_NUMBER","deployment":"local+ngrok"}}' >> ~/.gbrain/integrations/twilio-voice-brain/heartbeat.jsonl
``` ```
Tell the user: "Voice-to-brain is fully set up. Your number is [NUMBER]. Here's Tell the user: "Voice-to-brain is fully set up. Your number is [NUMBER]. Here's
@@ -472,6 +502,95 @@ The watchdog restarts the server if it crashes."
- The watchdog (Step 9) handles this automatically - The watchdog (Step 9) handles this automatically
- For a permanent URL: upgrade to ngrok paid ($8/mo) for a static domain, or deploy to Fly.io/Railway instead - For a permanent URL: upgrade to ngrok paid ($8/mo) for a static domain, or deploy to Fly.io/Railway instead
**Note on Option B credentials:** If using the DIY pipeline (Option B), you will
also need API keys for your chosen STT provider (e.g., Deepgram) and TTS provider
(e.g., Cartesia, OpenAI TTS). Collect and validate these during Step 2 alongside
the Twilio and OpenAI credentials listed above.
## Critical Production Fixes (v0.8.1)
These are NOT optional. They prevent real production failures discovered in a
deployment handling daily calls.
### Unicode Crash Fix (CRITICAL)
**Problem:** Em dashes (--), arrows (->), and other non-ASCII characters in the
prompt context cause broken surrogate pairs that crash the Twilio WebSocket
connection. Phone calls drop silently.
**Fix:** Replace ALL non-ASCII characters with ASCII equivalents throughout the
entire prompt file before sending to Twilio. This is invisible in development
(browsers handle unicode fine) and catastrophic in production.
```javascript
function sanitizeForTwilio(text) {
return text
.replace(/[\u2014\u2013]/g, '--') // em/en dash
.replace(/[\u2018\u2019]/g, "'") // smart quotes
.replace(/[\u201C\u201D]/g, '"') // smart double quotes
.replace(/\u2192/g, '->') // right arrow
.replace(/\u2190/g, '<-') // left arrow
.replace(/[\u2026]/g, '...') // ellipsis
.replace(/[^\x00-\x7F]/g, '') // strip remaining non-ASCII
}
```
### PII Scrub from Voice Context (CRITICAL)
**Problem:** Brain context loaded into the voice prompt may contain phone numbers,
email addresses, and other PII. The voice agent reads these aloud to callers.
**Fix:** Regex-strip PII from all voice context before injecting into the prompt:
- Phone numbers: `/\+?\d[\d\s\-().]{7,}\d/g`
- Email addresses: `/[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g`
- URLs with auth tokens or API keys
- Any string matching common credential patterns
### Identity-First Prompt (IMPORTANT)
**Problem:** Voice agents lose their identity mid-conversation. Saying "You are NOT
Claude" doesn't stick. The model reverts to its base persona.
**Fix:** Put identity FIRST in the system prompt, before any context or rules:
```
# You ARE [Agent Name]
You are [Name], a voice assistant who works with [Brain Name].
You are NOT Claude. You are NOT a general AI assistant.
[Name] has their own personality: [traits].
# Context
[... brain context, calendar, tasks ...]
# Rules
[... behavioral rules ...]
```
Positioning identity before context ensures the model sees it first and
maintains it throughout the conversation.
### Auto-Upload Call Audio (RECOMMENDED)
**Problem:** If post-call processing fails, the call audio is lost forever.
**Fix:** Auto-upload ALL call audio immediately on call end:
- Twilio calls: download the MP3 recording URL from Twilio
- WebRTC calls: capture via MediaRecorder (webm/opus format)
- Upload to cloud storage (Supabase Storage, S3, etc.)
- Store `.redirect.yaml` pointer in brain repo if >= 100 MB
- This ensures every call has a recoverable audio source regardless
of whether the transcript or brain page was created successfully
### Smart VAD as Default
**Problem:** Push-to-talk is unnatural on phone calls. Server-side VAD has
variable quality.
**Fix:** Default to Smart VAD (Silero VAD) for voice activity detection:
- Better endpointing than server-side VAD
- Fewer false triggers in noisy environments
- PTT available as fallback (UI toggle for WebRTC clients)
- Presets: quiet (0.7 threshold), normal (0.85), noisy (0.95), very_noisy (0.98)
## Production Patterns (Recommended) ## Production Patterns (Recommended)
These patterns come from a production voice deployment handling real calls daily. These patterns come from a production voice deployment handling real calls daily.
@@ -488,13 +607,13 @@ AI brain. "I work with [Brain], [Owner]'s AI." Lighter, more playful, more curio
#### Pre-Computed Bid System #### Pre-Computed Bid System
**Problem:** Dead air kills engagement. Voice agents wait passively. **Problem:** Dead air kills engagement. Voice agents wait passively.
**Pattern:** At call start, scan live context and pre-compute up to 10 engagement bids. **Pattern:** At call start, scan live context and pre-compute up to 10 engagement bids.
Two types: informative (tasks, calendar, social radar) and relational (curiosity templates). Two types: informative (tasks, calendar, social monitoring) and relational (curiosity templates).
Bids go INTO the prompt so the agent picks from a list. Use bids #1 and #2 for greeting, Bids go INTO the prompt so the agent picks from a list. Use bids #1 and #2 for greeting,
cycle the rest during conversation. Never ask "anything else?" — bring up the next bid. cycle the rest during conversation. Never ask "anything else?" — bring up the next bid.
#### Context-First Prompt #### Context-First Prompt
**Problem:** Voice agent greets generically because it doesn't know what's happening today. **Problem:** Voice agent greets generically because it doesn't know what's happening today.
**Pattern:** Load live context at call start: tasks, calendar, location, social radar, **Pattern:** Load live context at call start: tasks, calendar, location, social monitoring,
morning briefing. Position context FIRST in the prompt (before rules) so the model sees morning briefing. Position context FIRST in the prompt (before rules) so the model sees
it immediately and uses it in the greeting. Try/catch per section. Cap 500-1000 chars each. it immediately and uses it in the greeting. Try/catch per section. Cap 500-1000 chars each.
@@ -658,7 +777,7 @@ over WebRTC data channel — use Whisper post-call instead.
| Keyword | Report Loaded | | Keyword | Report Loaded |
|---------|--------------| |---------|--------------|
| email, inbox, mail | inbox sweep report | | email, inbox, mail | inbox sweep report |
| social, twitter, mentions | social radar report | | social, twitter, mentions | social engagement report |
| briefing, morning | morning briefing | | briefing, morning | morning briefing |
| meeting | meeting sync report | | meeting | meeting sync report |
| slack | slack scan report | | slack | slack scan report |

98
recipes/x-to-brain.md
View File

@@ -1,8 +1,8 @@
--- ---
id: x-to-brain id: x-to-brain
name: X-to-Brain name: X-to-Brain
version: 0.7.0 version: 0.8.1
description: Twitter timeline, mentions, and keyword monitoring flow into brain pages. Tracks deletions and engagement velocity. description: Twitter timeline, mentions, and keyword monitoring flow into brain pages. Tracks deletions, engagement velocity, OCR on images, and real-time alerts.
category: sense category: sense
requires: [] requires: []
secrets: secrets:
@@ -201,7 +201,99 @@ The agent should review collected data 2-3x daily and run enrichment.
```bash ```bash
mkdir -p ~/.gbrain/integrations/x-to-brain mkdir -p ~/.gbrain/integrations/x-to-brain
echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","event":"setup_complete","source_version":"0.7.0","status":"ok","details":{"user_id":"X_USER_ID"}}' >> ~/.gbrain/integrations/x-to-brain/heartbeat.jsonl echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","event":"setup_complete","source_version":"0.8.1","status":"ok","details":{"user_id":"X_USER_ID"}}' >> ~/.gbrain/integrations/x-to-brain/heartbeat.jsonl
```
## Production Patterns (v0.8.1)
These patterns come from a production deployment tracking 19+ accounts with
real-time monitoring.
### Image OCR (NEW)
**Problem:** Text-only collection misses visual context in tweet images --
screenshots, charts, memes with text overlay, quote screenshots.
**Fix:** Run OCR on tweet images via a vision model (Claude Sonnet or equivalent):
- For every tweet with images, extract full text content via vision API
- Store OCR output alongside the tweet data
- Include extracted text in entity detection and brain page updates
- Charts/data visualizations: extract data points, describe findings
This catches signal that text-only collectors miss entirely.
### Real-Time Monitoring via Filtered Stream (NEW)
**Problem:** 30-minute polling means you find out about things 30 minutes late.
For time-sensitive content (engagement spikes, deletions, breaking threads),
that's too slow.
**Fix:** Use Twitter's Filtered Stream API (`GET /2/tweets/search/stream`) for
near-real-time monitoring. Catches outbound tweets within seconds.
**Setup:**
1. Add filter rules: `POST /2/tweets/search/stream/rules` with your tracking terms
2. Open persistent connection: `GET /2/tweets/search/stream`
3. Process tweets as they arrive (no polling delay)
**Requirements:** Basic tier ($200/mo) minimum for Filtered Stream access.
**Use alongside polling:** Stream for real-time alerts, polling for completeness
(stream can drop tweets during disconnects).
### Tweet Rating Rubric (NEW)
**Problem:** Not all tweets deserve the same attention. Without scoring, every
tweet gets equal weight.
**Fix:** Rate tweets on a 6-dimension rubric:
1. **Reach** -- follower count, engagement rate
2. **Relevance** -- connection to your interests/work
3. **Sentiment** -- positive/negative/neutral toward you
4. **Novelty** -- new information vs rehash
5. **Actionability** -- does this require a response?
6. **Virality potential** -- engagement velocity, quote-tweet ratio
Re-rate after 60 minutes to track engagement trajectory. A tweet at 50 likes
that hits 500 in an hour is a different signal than one that stays at 50.
### Outbound Tweet Monitoring (NEW)
**Problem:** You tweet something and don't notice engagement patterns until
hours later.
**Fix:** 60-second monitoring window after every outbound tweet:
- Check engagement velocity (likes, replies, quotes)
- Flag unusual reply-to-like ratios (high reply ratios signal controversy)
- Flag if quote-tweet ratio > retweet ratio (commentary, not sharing)
- Cross-reference mentioned accounts against brain for context
### X-to-Brain Pipeline (NEW)
Every tweet interaction can automatically create/update brain pages:
- Mentioned person has a brain page? Append to their timeline
- New person mentioned? Check notability gate, create page if notable
- Article URL in tweet? Fetch and ingest via article workflow
- Video URL in tweet? Queue for transcription pipeline
- Images? OCR and extract text content
Follow `skills/_brain-filing-rules.md` for filing decisions.
### Cron Staggering (IMPORTANT)
**Problem:** Multiple cron jobs firing simultaneously causes resource contention
and timeouts.
**Fix:** Stagger all collection schedules so max 1 runs per minute:
```
# Good: staggered
*/30 * * * * x-collector # :00, :30
5,35 * * * * x-bundle-ingest # :05, :35
10 */3 * * * social-monitor # :10 every 3h
# Bad: overlapping
*/30 * * * * x-collector
*/30 * * * * x-bundle-ingest # fires at same time!
``` ```
## Implementation Guide ## Implementation Guide

84
skills/_brain-filing-rules.md Normal file
View File

@@ -0,0 +1,84 @@
# Brain Filing Rules -- MANDATORY for all skills that write to the brain
## The Rule
The PRIMARY SUBJECT of the content determines where it goes. Not the format,
not the source, not the skill that's running.
## Decision Protocol
1. Identify the primary subject (a person? company? concept? policy issue?)
2. File in the directory that matches the subject
3. Cross-link from related directories
4. When in doubt: what would you search for to find this page again?
## Common Misfiling Patterns -- DO NOT DO THESE
| Wrong | Right | Why |
|-------|-------|-----|
| Analysis of a topic -> `sources/` | -> appropriate subject directory | sources/ is for raw data only |
| Article about a person -> `sources/` | -> `people/` | Primary subject is a person |
| Meeting-derived company info -> `meetings/` only | -> ALSO update `companies/` | Entity propagation is mandatory |
| Research about a company -> `sources/` | -> `companies/` | Primary subject is a company |
| Reusable framework/thesis -> `sources/` | -> `concepts/` | It's a mental model |
| Tweet thread about policy -> `media/` | -> `civic/` or `concepts/` | media/ is for content ops |
## What `sources/` Is Actually For
`sources/` is ONLY for:
- Bulk data imports (API dumps, CSV exports, snapshots)
- Raw data that feeds multiple brain pages (e.g., a guest export, contact sync)
- Periodic captures (quarterly snapshots, sync exports)
If the content has a clear primary subject (a person, company, concept, policy
issue), it does NOT go in sources/. Period.
## Notability Gate
Not everything deserves a brain page. Before creating a new entity page:
- **People:** Will you interact with them again? Are they relevant to your work?
- **Companies:** Are they relevant to your work or interests?
- **Concepts:** Is this a reusable mental model worth referencing later?
- **When in doubt, DON'T create.** A missing page can be created later.
A junk page wastes attention and degrades search quality.
## Iron Law: Back-Linking (MANDATORY)
Every mention of a person or company with a brain page MUST create a back-link
FROM that entity's page TO the page mentioning them. This is bidirectional:
the new page links to the entity, AND the entity's page links back.
Format for back-links (append to Timeline or See Also):
```
- **YYYY-MM-DD** | Referenced in [page title](path/to/page.md) -- brief context
```
An unlinked mention is a broken brain. The graph is the intelligence.
## Citation Requirements (MANDATORY)
Every fact written to a brain page must carry an inline `[Source: ...]` citation.
Three formats:
- **Direct attribution:** `[Source: User, {context}, YYYY-MM-DD]`
- **API/external:** `[Source: {provider}, YYYY-MM-DD]` or `[Source: {publication}, {URL}]`
- **Synthesis:** `[Source: compiled from {list of sources}]`
Source precedence (highest to lowest):
1. User's direct statements (highest authority)
2. Compiled truth (pre-existing brain synthesis)
3. Timeline entries (raw evidence)
4. External sources (API enrichment, web search -- lowest)
When sources conflict, note the contradiction with both citations. Don't
silently pick one.
## Raw Source Preservation
Every ingested item should have its raw source preserved for provenance:
- **< 100 MB** (text, PDFs, transcripts): store in the brain repo (git-tracked)
in a `.raw/` sidecar directory alongside the brain page
- **>= 100 MB** (video, audio, large datasets): upload to cloud storage and
store a `.redirect.yaml` pointer in the brain repo
This ensures any derived brain page can be traced back to its original source.

15
skills/briefing/SKILL.md
View File

@@ -2,6 +2,9 @@
Compile a daily briefing from brain context. Compile a daily briefing from brain context.
> **Filing rule:** When the briefing creates or updates brain pages,
> follow `skills/_brain-filing-rules.md`.
## Workflow ## Workflow
1. **Today's meetings.** For each meeting on the calendar: 1. **Today's meetings.** For each meeting on the calendar:
@@ -72,6 +75,18 @@ PEOPLE IN PLAY
- [name] -- [why they're active] - [name] -- [why they're active]
``` ```
## Back-Linking During Briefing
If the briefing creates or updates any brain pages (e.g., new meeting prep
pages, updated entity pages), the back-linking iron law applies: every entity
mentioned must have a back-link from their page. See `skills/_brain-filing-rules.md`.
## Citation in Briefings
When presenting facts from brain pages, include inline citations:
- "Jane is CTO of Acme [Source: people/jane-doe, updated 2026-04-01]"
- This lets the user trace any claim back to the brain page and assess freshness
## Tools Used ## Tools Used
- Search gbrain by name (query) - Search gbrain by name (query)

298
skills/enrich/SKILL.md
View File

@@ -1,39 +1,281 @@
# Enrich Skill # Enrich Skill
Enrich person and company pages from external APIs. Enrich person and company pages from external sources. Scale effort to importance.
## Sources > **Filing rule:** Read `skills/_brain-filing-rules.md` before creating any new page.
| Source | Data | API | ## Iron Law: Back-Linking (MANDATORY)
|--------|------|-----|
| Crustdata | LinkedIn profiles, company data | REST API |
| Happenstance | Career history, connections | REST API |
| Exa | Web mentions, articles | REST API |
Note: enrichment requires separate API credentials for each service. No client Every mention of a person or company with a brain page MUST create a back-link
integrations ship in v1. This skill guides the agent to make API calls directly. FROM that entity's page TO the page mentioning them. An unlinked mention is a
broken brain. See `skills/_brain-filing-rules.md` for format.
## Workflow ## Philosophy
1. **Select target pages.** List person or company pages in gbrain. A brain page should read like an intelligence dossier, not a LinkedIn scrape.
2. **For each page:** Facts are table stakes. Texture is the value -- what do they believe, what are
- Read the page from gbrain to understand what we already know they building, what makes them tick, where are they headed.
- Call external APIs for fresh data
- Store raw API responses in gbrain (put_raw_data) to preserve provenance
- Distill highlights into compiled_truth updates
- Store the updated page in gbrain
3. **Validation rules:**
- Connection count < 20 on LinkedIn = likely wrong person, skip
- Name mismatch between brain and API = skip, flag for manual review
- Don't overwrite human-written assessments with API boilerplate
## Quality Rules ## Citation Requirements (MANDATORY)
- Raw data goes to gbrain's raw_data store (preserves provenance) Every fact must carry an inline `[Source: ...]` citation.
- Only distilled, useful info goes to compiled_truth
- Always add a timeline entry in gbrain: "Enriched from [source] on [date]" Three formats:
- Don't enrich the same page more than once per week unless requested - **Direct attribution:** `[Source: User, {context}, YYYY-MM-DD]`
- Rate limit: respect API rate limits, use exponential backoff - **API/external:** `[Source: {provider} enrichment, YYYY-MM-DD]`
- **Synthesis:** `[Source: compiled from {list of sources}]`
Source precedence (highest to lowest):
1. User's direct statements
2. Compiled truth (pre-existing brain synthesis)
3. Timeline entries (raw evidence)
4. External sources (API enrichment, web search)
When sources conflict, note the contradiction with both citations.
## When To Enrich
### Primary triggers
- User mentions an entity in conversation
- Entity appears in a meeting transcript or email
- New contact appears with significant context
- Entity makes news or has a major event
- Any ingest pipeline encounters a notable entity
### Do NOT enrich
- Random mentions with no relationship signal
- Bot/spam accounts
- Entities with no substantive connection to the user's work
- Same page enriched within the past week (unless new signal warrants it)
## Enrichment Tiers
Scale enrichment to importance. Don't waste API calls on low-value entities.
| Tier | Who | Effort | Sources |
|------|-----|--------|---------|
| 1 (key) | Inner circle, close collaborators, key contacts | Full pipeline | All available APIs + deep web research |
| 2 (notable) | Occasional interactions, industry figures | Moderate | Web research + social + brain cross-ref |
| 3 (minor) | Worth tracking, not critical | Light | Brain cross-ref + social lookup if handle known |
## The Enrichment Protocol (7 Steps)
### Step 1: Identify entities
Extract people, companies, concepts from the incoming signal.
### Step 2: Check brain state
For each entity:
- `gbrain search "name"` -- does a page already exist?
- **If yes:** UPDATE path (add new signal, update compiled truth if material)
- **If no:** CREATE path (check notability gate first, then create)
### Step 3: Extract signal from source
Don't just capture facts. Capture texture:
| Signal Type | What to Extract |
|-------------|----------------|
| Opinions, beliefs | What They Believe section |
| Current projects, features shipped | What They're Building section |
| Ambition, career arc, motivation | What Motivates Them section |
| Topics they return to obsessively | Hobby Horses section |
| Who they amplify, argue with, respect | Network / Relationships |
| Ascending, plateauing, pivoting? | Trajectory section |
| Role, company, funding, location | State section (hard facts) |
### Step 4: External data source lookups
Priority order -- stop when you have enough signal for the entity's tier.
**4a. Brain cross-reference (always, all tiers)**
- `gbrain search "name"` and `gbrain query "what do we know about name"`
- Check related pages: company pages for person enrichment and vice versa
- This is free and often the richest source
**4b. Web research (Tier 1 and 2)**
- Use Perplexity, Brave Search, Exa, or equivalent web research tool
- **Key pattern:** Send existing brain knowledge as context so the search
returns DELTA (what's new vs what you already know), not a rehash
- Opus-class models for Tier 1 deep research, lighter models for Tier 2
**4c. Social media lookup (all tiers when handle known)**
- Pull recent posts/tweets for tone, interests, current focus
- Social media is the highest-texture signal for what someone actually thinks
**4d. People enrichment APIs (Tier 1)**
- LinkedIn data, career history, connections, education
**4e. Company enrichment APIs (Tier 1)**
- Company data, financials, headcount, key hires, recent news
| Data Need | Example Sources | Tier |
|-----------|----------------|------|
| Web research | Perplexity, Brave, Exa | 1-2 |
| LinkedIn / career | Crustdata, Proxycurl, People Data Labs | 1 |
| Career history | Happenstance, LinkedIn | 1 |
| Funding / company data | Crunchbase, PitchBook, Clearbit | 1 |
| Social media | Platform APIs, web scraping | 1-3 |
| Meeting history | Calendar/meeting transcript tools | 1-2 |
### Step 5: Save raw data (preserves provenance)
Store raw API responses via `put_raw_data` in gbrain:
```json
{
"source": "crustdata",
"fetched_at": "2026-04-11T...",
"query": "jane doe",
"data": { ... }
}
```
Raw data preserves provenance. If the compiled truth is ever questioned,
the raw data shows exactly what the API returned.
### Step 6: Write to brain
#### CREATE path
1. Check notability gate (see `skills/_brain-filing-rules.md`)
2. Check filing rules -- where does this entity go?
3. Create page with the appropriate template (below)
4. Fill compiled truth with citations
5. Add first timeline entry
6. Leave empty sections as `[No data yet]` (don't fill with boilerplate)
#### UPDATE path
1. Add new timeline entries (reverse-chronological, append-only)
2. Update compiled truth ONLY if the new signal materially changes the picture
3. Update State section with new facts
4. Flag contradictions between new signal and existing compiled truth
5. Don't overwrite user-written assessments with API boilerplate
#### Person page template
```markdown
---
title: Full Name
type: person
created: YYYY-MM-DD
updated: YYYY-MM-DD
tags: []
company: Current Company
relationship: How the user knows them
email:
linkedin:
twitter:
location:
---
# Full Name
> 1-paragraph executive summary: HOW do you know them, WHY do they matter,
> what's the current state of the relationship.
## State
Role, company, key context. Hard facts only.
## What They Believe
Ideology, first principles, worldview. What hills do they die on?
## What They're Building
Current projects, recent launches, what they're focused on.
## What Motivates Them
Ambition, career arc, what drives them.
## Hobby Horses
Topics they return to obsessively. Recurring themes in their work/posts.
## Assessment
Your read on this person. Strengths, gaps, trajectory.
## Trajectory
Ascending, plateauing, pivoting, declining? Where are they headed?
## Relationship
History of interactions, shared context, relationship quality.
## Contact
Email, social handles, preferred communication channel.
## Network
Key connections, mutual contacts, organizational relationships.
## Open Threads
Active conversations, pending items, things to follow up on.
---
## Timeline
Reverse chronological. Every entry has a date and [Source: ...] citation.
- **YYYY-MM-DD** | Event description [Source: ...]
```
#### Company page template
```markdown
---
title: Company Name
type: company
created: YYYY-MM-DD
updated: YYYY-MM-DD
tags: []
---
# Company Name
> 1-paragraph executive summary.
## State
What they do, stage, key people, key metrics, your connection.
## Open Threads
Active items, pending decisions, things to track.
---
## Timeline
- **YYYY-MM-DD** | Event description [Source: ...]
```
### Step 7: Cross-reference
- Update company pages from person enrichment (and vice versa)
- Update related project/deal pages if relevant context surfaced
- Add back-links from every entity mentioned (MANDATORY)
- Check index files if the brain uses them
## Bulk Enrichment Rules
- **Test on 3-5 entities first.** Read actual output. Check quality.
- Only proceed to bulk after test shots pass your quality bar.
- 3+ entities from one source -> batch process or spawn sub-agent
- Throttle API calls. Respect rate limits.
- Commit every 5-10 entities during bulk runs.
- Save a report after bulk enrichment (see Report Storage below).
## Validation Rules
- Connection count < 20 on LinkedIn = likely wrong person, skip
- Name mismatch between brain and API = skip, flag for review
- Joke profiles or obviously wrong data = save to raw, don't update page
- Don't overwrite user-written assessments with API boilerplate
- When in doubt: save raw data but don't update brain page
## Report Storage
After enrichment sweeps, save a report:
- Number of entities processed
- New pages created vs existing updated
- Data sources called and results quality
- Notable discoveries or contradictions
- Validation flags or API failures
This creates an audit trail for brain enrichment over time.
## Tools Used ## Tools Used
@@ -43,3 +285,5 @@ integrations ship in v1. This skill guides the agent to make API calls directly.
- List pages in gbrain by type (list_pages) - List pages in gbrain by type (list_pages)
- Store raw API data in gbrain (put_raw_data) - Store raw API data in gbrain (put_raw_data)
- Retrieve raw data from gbrain (get_raw_data) - Retrieve raw data from gbrain (get_raw_data)
- Link entities in gbrain (add_link)
- Check backlinks in gbrain (get_backlinks)

200
skills/ingest/SKILL.md
View File

@@ -1,6 +1,25 @@
# Ingest Skill # Ingest Skill
Ingest meetings, articles, documents, and conversations into the brain. Ingest meetings, articles, media, documents, and conversations into the brain.
> **Filing rule:** Read `skills/_brain-filing-rules.md` before creating any new page.
## Iron Law: Back-Linking (MANDATORY)
Every mention of a person or company with a brain page MUST create a back-link
FROM that entity's page TO the page mentioning them. An unlinked mention is a
broken brain. See `skills/_brain-filing-rules.md` for format.
## Citation Requirements (MANDATORY)
Every fact written to a brain page must carry an inline `[Source: ...]` citation.
- **User's statements:** `[Source: User, {context}, YYYY-MM-DD]`
- **Meeting data:** `[Source: Meeting "{title}", YYYY-MM-DD]`
- **Email/message:** `[Source: email from {name} re: {subject}, YYYY-MM-DD]`
- **Web content:** `[Source: {publication}, {URL}, YYYY-MM-DD]`
- **Social media:** `[Source: X/@handle, YYYY-MM-DD](URL)` (include link)
- **Synthesis:** `[Source: compiled from {sources}]`
## Workflow ## Workflow
@@ -8,10 +27,11 @@ Ingest meetings, articles, documents, and conversations into the brain.
2. **For each entity mentioned:** 2. **For each entity mentioned:**
- Read the entity's page from gbrain to check if it exists - Read the entity's page from gbrain to check if it exists
- If exists: update compiled_truth (rewrite State section with new info, don't append) - If exists: update compiled_truth (rewrite State section with new info, don't append)
- If new: store the page in gbrain with the appropriate type and slug - If new: check notability gate, then store the page in gbrain with the appropriate type and slug
3. **Append to timeline.** Add a timeline entry in gbrain for each event, with date, summary, and source. 3. **Append to timeline.** Add a timeline entry in gbrain for each event, with date, summary, and source citation.
4. **Create cross-reference links.** Link entities in gbrain for every entity pair mentioned together, using the appropriate relationship type. 4. **Create cross-reference links.** Link entities in gbrain for every entity pair mentioned together, using the appropriate relationship type.
5. **Timeline merge.** The same event appears on ALL mentioned entities' timelines. If Alice met Bob at Acme Corp, the event goes on Alice's page, Bob's page, and Acme Corp's page. 5. **Back-link all entities.** Update EVERY mentioned entity's page with a back-link to this page (Iron Law).
6. **Timeline merge.** The same event appears on ALL mentioned entities' timelines. If Alice met Bob at Acme Corp, the event goes on Alice's page, Bob's page, and Acme Corp's page.
## Entity Detection on Every Message ## Entity Detection on Every Message
@@ -26,13 +46,11 @@ the signal detection loop that makes the brain compound over time.
- `gbrain search "name"` -- does a page already exist? - `gbrain search "name"` -- does a page already exist?
- **If yes:** load context with `gbrain get <slug>`. Use the compiled truth to - **If yes:** load context with `gbrain get <slug>`. Use the compiled truth to
inform your response. Update the page if the message contains new information. inform your response. Update the page if the message contains new information.
- **If no:** assess notability. If the entity is worth tracking (will come up - **If no:** assess notability (see `skills/_brain-filing-rules.md`). If the entity
again, is relevant to the user's world), create a new page with is worth tracking, create a new page with `gbrain put <type/slug>` and populate
`gbrain put <type/slug>` and populate with what you know. with what you know.
3. **After creating or updating pages:** commit changes to the brain repo, then 3. **After creating or updating pages:** sync to gbrain:
sync to gbrain:
```bash ```bash
git add brain/ && git commit -m "update entity pages"
gbrain sync --no-pull --no-embed gbrain sync --no-pull --no-embed
``` ```
4. **Don't block the conversation.** Entity detection and enrichment should happen 4. **Don't block the conversation.** Entity detection and enrichment should happen
@@ -42,18 +60,172 @@ the signal detection loop that makes the brain compound over time.
### What counts as notable ### What counts as notable
- People the user interacts with or discusses (not random mentions) - People the user interacts with or discusses (not random mentions)
- Companies relevant to the user's work, investments, or interests - Companies relevant to the user's work or interests
- Concepts or frameworks the user references or creates - Concepts or frameworks the user references or creates
- The user's own original thinking (ideas, theses, observations) -- highest value - The user's own original thinking (ideas, theses, observations) -- highest value
- See `skills/_brain-filing-rules.md` for the full notability gate
### What to capture from the user's own thinking
Original thinking is the most valuable signal. Capture exact phrasing -- the user's
language IS the insight. Don't paraphrase.
- Novel observations or theses
- Frameworks, mental models, heuristics
- Connections between ideas that others miss
- Contrarian positions with reasoning
- Strong reactions to external stimuli (what triggered it and why)
## Media Workflows
Content the user encounters should be captured in the brain. File by PRIMARY
SUBJECT, not by format (see `skills/_brain-filing-rules.md`).
### Articles & Web Content
**Input:** URL shared by user, or article mentioned in conversation.
**Process:**
1. Fetch content (`web_fetch` or equivalent)
2. Extract: title, author, publication, date, full text
3. Summarize: executive summary + key arguments (not a rehash)
4. Extract entities: people, companies, concepts mentioned
5. **Save raw source** for provenance (see Raw Source Preservation below)
6. Analyze for the user: don't just summarize. What's interesting given what you
know about them? Flag connections, contradictions, content opportunities.
**Write to:** appropriate directory per filing rules (about a person -> `people/`,
about a company -> `companies/`, reusable framework -> `concepts/`, raw data -> `sources/`)
### Videos & Podcasts
**Input:** URL (YouTube, podcast, etc.) or local audio/video file.
**Process:**
1. Get transcript -- speaker-diarized if possible (services like Diarize.io provide
speaker-labeled, word-level timing)
2. **Save raw transcript** (both JSON and human-readable TXT)
3. Analyze: executive summary, key ideas, key quotes with speaker attribution,
notable stories/anecdotes, people and companies mentioned
4. Extract and cross-reference all entities mentioned
5. **HARD RULE:** every video/podcast brain page MUST link to the raw diarized
transcript. A page without transcript links is incomplete.
**Write to:** `media/videos/` or `media/podcasts/` with back-links to all entities.
**Quality bar:**
- Compelling headline (not "This video discusses...")
- Executive summary that makes you want to watch/listen
- Key Ideas as actual insights, not topic labels
- Verbatim quotes with real speaker names (not "speaker_0")
- All entities extracted with context and back-linked
### PDFs & Documents
**Input:** File path or URL.
**Process:**
1. Extract text (OCR if scanned/image PDF)
2. **Save raw source** for provenance
3. Summarize: executive summary + key sections + notable data
4. Extract entities
5. Cross-reference from entity pages
**Write to:** per filing rules (file by primary subject, not format).
### Screenshots & Images
**Input:** Image file.
**Process:**
1. Analyze content (OCR for text-heavy images, description for photos)
2. If tweet screenshot: extract text, author, date, route to social media workflow
3. If article screenshot: extract text, route to article workflow
4. If data/chart: extract data points, describe findings
**Write to:** depends on content -- route to the appropriate workflow above.
### Meeting Transcripts
**Input:** Transcript from meeting recording service, or manual notes.
**Process:**
1. Pull full transcript (source of truth -- AI summaries are medium-low trust)
2. **Save raw transcript** for provenance
3. Write meeting page with YOUR analysis above the line, raw transcript below
4. **Entity propagation (MANDATORY):** for each attendee and company discussed:
- Update their brain page State section if new info surfaced
- Append to their Timeline with link to the meeting page
- Create page if person/company is notable and has no page yet
5. A meeting is NOT fully ingested until all entity pages are updated
**Write to:** `meetings/YYYY-MM-DD-short-description.md`
**What makes a good meeting page:**
- Reveals the real crux, not a bullet dump
- Connects to existing brain pages (people, companies, deals)
- Flags what changed (status, decisions, new info)
- Names tension or what was left unsaid
- Captures actual dynamic, not performative summary
### Social Media Content
**Input:** Tweet, thread, or social media post.
**Process:**
1. Fetch full content (thread, quote tweets, context)
2. If images present: OCR via vision model for full text extraction
3. Summarize: what's being said, why it matters, who's involved
4. Extract entities and update brain pages
5. Include direct link to the original post (MANDATORY for citations)
**Write to:** `media/x/` for daily aggregation, or entity-specific directories
if the post is primarily about a person/company.
## Raw Source Preservation
Every ingested item must have its raw source preserved for provenance.
- **< 100 MB** (text, PDFs, transcripts): store in brain repo `.raw/` sidecar
directories alongside the brain page
- **>= 100 MB** (video, audio, large datasets): upload to cloud storage (Supabase
Storage, S3, etc.) and store a `.redirect.yaml` pointer in the brain repo
The `.redirect.yaml` format:
```yaml
storage: supabase
bucket: brain-files
path: media/videos/raw/video-id.mp4
uploaded: 2026-04-11T...
size_bytes: 524288000
```
Use `put_raw_data` in gbrain to store raw API responses and metadata.
## Test Before Bulk
When processing multiple items (batch video ingestion, bulk meeting processing, etc.):
1. **Test on 3-5 items first.** Run in test mode if available.
2. **Read the actual output.** Is the quality good? Are titles compelling (not
"This video discusses...")? Are entities extracted and back-linked? Is the
format clean?
3. **Fix what's wrong** in the approach/skill, not via one-off patches.
4. **Only then: bulk execute** with throttling, commits every 5-10 items.
The marginal cost of testing 3 items first is near zero. The cost of cleaning
up 100 bad pages is enormous.
## Quality Rules ## Quality Rules
- Executive summary in compiled_truth must be updated, not just timeline appended - Executive summary in compiled_truth must be updated, not just timeline appended
- State section is REWRITTEN, not appended to. Current best understanding only. - State section is REWRITTEN, not appended to. Current best understanding only.
- Timeline entries are reverse-chronological (newest first) - Timeline entries are reverse-chronological (newest first)
- Every person/company mentioned gets a page if one doesn't exist - Every person/company mentioned gets a page if notable (see filing rules)
- Link types: knows, works_at, invested_in, founded, met_at, discussed - Link types: knows, works_at, invested_in, founded, met_at, discussed
- Source attribution: every timeline entry includes the source (meeting, article, email, etc.) - Source attribution: every timeline entry includes [Source: ...] citation
- Back-links: every entity mention creates a back-link (Iron Law)
- Filing: file by primary subject, not format or source (see filing rules)
## Tools Used ## Tools Used
@@ -63,3 +235,5 @@ the signal detection loop that makes the brain compound over time.
- Link entities in gbrain (add_link) - Link entities in gbrain (add_link)
- List tags for a page (get_tags) - List tags for a page (get_tags)
- Tag a page in gbrain (add_tag) - Tag a page in gbrain (add_tag)
- Store raw data in gbrain (put_raw_data)
- Check backlinks in gbrain (get_backlinks)

54
skills/maintain/SKILL.md
View File

@@ -25,6 +25,29 @@ Links pointing to pages that don't exist.
Pages that mention entity names but don't have formal links. Pages that mention entity names but don't have formal links.
- Read compiled_truth from gbrain, extract entity mentions, create links in gbrain - Read compiled_truth from gbrain, extract entity mentions, create links in gbrain
### Back-link enforcement
Check that the back-linking iron law is being followed:
- For each recently updated page, check if entities mentioned in it have
corresponding back-links FROM those entity pages
- A mention without a back-link is a broken brain
- Fix: add the missing back-link to the entity's Timeline or See Also section
- Format: `- **YYYY-MM-DD** | Referenced in [page title](path) -- brief context`
### Filing rule violations
Check for common misfiling patterns (see `skills/_brain-filing-rules.md`):
- Content with clear primary subjects filed in `sources/` instead of the
appropriate directory (people/, companies/, concepts/, etc.)
- Use gbrain search to find pages in `sources/` that reference specific
people, companies, or concepts -- these may be misfiled
- Flag misfiled pages for review or re-filing
### Citation audit
Spot-check pages for missing `[Source: ...]` citations:
- Read 5-10 recently updated pages
- Check that compiled truth (above the line) has inline citations
- Check that timeline entries have source attribution
- Flag pages where facts appear without provenance
### Tag consistency ### Tag consistency
Inconsistent tagging (e.g., "vc" vs "venture-capital", "ai" vs "artificial-intelligence"). Inconsistent tagging (e.g., "vc" vs "venture-capital", "ai" vs "artificial-intelligence").
- Standardize to the most common variant using gbrain tag operations - Standardize to the most common variant using gbrain tag operations
@@ -48,6 +71,25 @@ automatically.
Timeline items older than 30 days with unresolved action items. Timeline items older than 30 days with unresolved action items.
- Flag for review - Flag for review
## Benchmark Testing
Periodically verify search quality hasn't regressed. Run a battery of test
queries across difficulty tiers:
- **Tier 1 (entity lookup):** known names -- should always resolve
- **Tier 2 (topic recall):** concepts, topics -- keyword search should handle
- **Tier 3 (semantic):** queries with no exact keyword match -- needs embeddings
- **Tier 4 (cross-domain):** relational/connection queries -- only semantic handles
Compare results from `gbrain search` (keyword) vs `gbrain query` (hybrid).
Quality matters more than speed (2.5s right > 200ms wrong).
When to run benchmarks:
- After major brain imports or re-imports
- After gbrain version upgrades
- After embedding regeneration
- Monthly to track quality drift
## Heartbeat Integration ## Heartbeat Integration
For production agents running on a schedule, integrate gbrain health checks into For production agents running on a schedule, integrate gbrain health checks into
@@ -78,6 +120,18 @@ Flag pages where compiled truth is >30 days old but the timeline has recent entr
This means new evidence exists that hasn't been synthesized. These pages need a This means new evidence exists that hasn't been synthesized. These pages need a
compiled truth rewrite (see the maintain workflow above). compiled truth rewrite (see the maintain workflow above).
## Report Storage
After maintenance runs, save a report:
- Health check results (before/after scores for each dimension)
- Back-link violations found and fixed
- Filing rule violations found
- Citation gaps flagged
- Benchmark results (if run)
- Outstanding issues requiring user attention
This creates an audit trail for brain health over time.
## Quality Rules ## Quality Rules
- Never delete pages without confirmation - Never delete pages without confirmation

103
skills/migrations/v0.8.1.md Normal file
View File

@@ -0,0 +1,103 @@
---
version: 0.8.1
feature_pitch:
headline: "Your brain skills learned from production"
description: "Back-linking iron law, filing rules, enrichment protocol, media ingest, citation requirements, voice crash fixes -- all battle-tested from real production deployments."
---
# v0.8.1 Migration: Battle-Tested Skill Patterns
This migration updates your agent's brain-writing patterns and voice recipe.
No schema changes, no database migrations. Skills and recipes only.
## What's New
### Back-Linking Iron Law
Every skill that writes to the brain now enforces bidirectional linking. If a
page mentions a person or company, that entity's page MUST link back. This was
the single biggest knowledge graph integrity rule missing from v0.8.0.
### Brain Filing Rules (`skills/_brain-filing-rules.md`)
Cross-cutting rules for where brain pages go. File by PRIMARY SUBJECT, not
format or source. Includes common misfiling patterns table and notability gate.
### Enrichment Protocol (rewritten)
The enrich skill is now a full 7-step pipeline with 3-tier system (scale effort
to entity importance), person page section template (What They Believe, Building,
Motivates, Hobby Horses, Trajectory), pluggable data source table, and validation.
### Media Ingestion Workflows
The ingest skill now handles articles, videos, podcasts, PDFs, screenshots, and
social media content. Each with a complete input->process->write workflow.
### Citation Requirements
Every fact needs inline `[Source: ...]` citations across all skills. Three formats
defined (direct, API/external, synthesis) with source precedence hierarchy.
### Voice Recipe Production Fixes
Critical fixes from a production voice deployment: unicode crash fix (non-ASCII
breaks Twilio WebSocket), PII scrub from voice context, identity-first prompt
architecture, Smart VAD default, auto-upload call audio, and the DIY STT+LLM+TTS
pipeline (Deepgram+Claude+Cartesia) as an alternative to OpenAI Realtime.
### X-to-Brain Recipe Enhancements
Image OCR via vision models, real-time monitoring via Filtered Stream API,
6-dimension tweet rating rubric, outbound tweet monitoring, X-to-Brain automatic
pipeline, and cron staggering for resource management.
## Agent Instructions
### Step 1: Verify skills are updated
Check that your skills directory has the new files:
```bash
ls skills/_brain-filing-rules.md && echo "Filing rules: OK"
grep -q '"version": "0.8.1"' skills/manifest.json && echo "Manifest: OK"
```
Files that should be updated:
- `skills/_brain-filing-rules.md` (new)
- `skills/enrich/SKILL.md` (rewritten)
- `skills/ingest/SKILL.md` (expanded)
- `skills/maintain/SKILL.md` (expanded)
- `skills/query/SKILL.md` (updated)
- `skills/briefing/SKILL.md` (updated)
- `recipes/twilio-voice-brain.md` (v0.8.1 production fixes)
- `recipes/x-to-brain.md` (v0.8.1 enhancements)
### Step 2: Review existing brain pages for back-link gaps
Spot-check: pick 5 recently created pages and verify that every entity mentioned
has a back-link from the entity's page to the mentioning page.
If back-links are missing, add them:
```
- **YYYY-MM-DD** | Referenced in [page title](path/to/page.md) -- brief context
```
### Step 3: Review filing
Check for pages in `sources/` that should be in a subject-specific directory
(people/, companies/, concepts/, civic/). See `skills/_brain-filing-rules.md`.
### Step 4: Voice recipe (if using)
If you have voice-to-brain set up:
1. Apply the unicode sanitization fix to your voice server
2. Add PII scrub to your prompt builder
3. Move identity to the TOP of your system prompt
4. Consider switching to Smart VAD (Silero) as default
### Step 5: X-to-brain recipe (if using)
If you have X-to-brain set up:
1. Add image OCR to your tweet processing pipeline
2. Consider Filtered Stream API for real-time monitoring
3. Stagger your cron schedules (max 1 per minute)
### Step 6: Done
```bash
mkdir -p ~/.gbrain/migrations
echo '{"version":"0.8.1","ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","status":"complete"}' >> ~/.gbrain/migrations/completed.jsonl
```

17
skills/query/SKILL.md
View File

@@ -49,6 +49,23 @@ When multiple sources provide conflicting information, follow this precedence:
When sources conflict, note the contradiction with both citations. Don't silently When sources conflict, note the contradiction with both citations. Don't silently
pick one. pick one.
## Citation in Answers
When referencing brain pages in your answer, propagate inline citations:
- Cite the page: "According to [Source: people/jane-doe, compiled truth]..."
- When brain pages have inline `[Source: ...]` citations, propagate them so
the user can trace facts to their origin
- When you synthesize across multiple pages, cite all sources
## Search Quality Awareness
If search results seem off (wrong results, missing known pages, irrelevant hits):
- Run `gbrain doctor --json` to check index health
- Check embedding coverage -- partial embeddings degrade hybrid search
- Compare keyword search (`gbrain search`) vs hybrid search (`gbrain query`)
for the same query to isolate whether the issue is embedding-related
- Report search quality issues in the maintain workflow (see maintain skill)
## Tools Used ## Tools Used
- Keyword search gbrain (search) - Keyword search gbrain (search)