v0.42.6.0 feat(enrich): gbrain enrich --thin — brain-internal grounded synthesis for stub pages (#1700) (#1757)

* feat(engine): listEnrichCandidates source-aware candidate selection (#1700)

One SQL query per engine: thin-filter + per-page source-correct inbound-link
count (to_page_id = p.id, mentions excluded) + enriched_at recency guard +
whitelisted ORDER BY (ENRICH_ORDER_SQL) + LIMIT, returning a lightweight
projection (no page bodies). EnrichCandidate/EnrichCandidatesOpts types.
pg + pglite parity, pinned by engine-parity.test.ts.

* feat(enrich): gbrain enrich --thin brain-internal grounded synthesis (#1700)

Develops stub pages at scale by consolidating scattered brain knowledge (search
+ backlinks + facts + raw_data) into one grounded gateway.chat call per page.
Resumable (op-checkpoint), budget-capped (best-effort under --workers), per-page
advisory lock, put_page write-through. CLI + thin-client refuse + Minion handler.

Includes codex-review fixes: sanitizeContext neutralizes the <context> envelope
delimiters (P1 injection escape); background fan-out idempotency key carries the
run fingerprint (P1); post-hoc budget-overage flag via new BudgetTracker.cap
getter (P1); checkpoint flush on budget exhaustion (P2). Accepts the documented
best-effort in-flight-cancel limitation (D5) with an explicit code note.

* feat(cycle): enrich_thin opt-in autopilot phase (#1700)

Default-OFF trickle around runEnrichCore: develops a few thin pages per source
per tick so the brain compounds over time. Per-source cap enforced as
min(per-source, brain-wide remaining) with brain-wide total + walltime caps
(P2 fix: per-source max_cost_usd was parsed but never enforced). Wired into
CyclePhase / ALL_PHASES / PHASE_SCOPE / NEEDS_LOCK + dispatch.

* chore: bump version and changelog (v0.42.6.0)

gbrain enrich --thin batch enrichment (#1700) + codex-review fixes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-06-01 22:13:19 -07:00
committed by GitHub
parent 766604dea0
commit 662a6e27d4
24 changed files with 2606 additions and 15 deletions

View File

@@ -47,8 +47,9 @@ import type {
EvalCaptureFailure, EvalCaptureFailureReason,
SalienceOpts, SalienceResult, AnomaliesOpts, AnomalyResult,
EmotionalWeightInputRow, EmotionalWeightWriteRow,
EnrichCandidatesOpts, EnrichCandidate,
} from './types.ts';
import { GBrainError, PAGE_SORT_SQL } from './types.ts';
import { GBrainError, PAGE_SORT_SQL, ENRICH_ORDER_SQL } from './types.ts';
import { computeAnomaliesFromBuckets } from './cycle/anomaly.ts';
import * as db from './db.ts';
import { ConnectionManager } from './connection-manager.ts';
@@ -5090,6 +5091,67 @@ export class PostgresEngine implements BrainEngine {
}));
}
async listEnrichCandidates(opts: EnrichCandidatesOpts): Promise<EnrichCandidate[]> {
// v0.41.39 (issue #1700). Empty types → no rows (no SQL).
if (!opts.types || opts.types.length === 0) return [];
const sql = this.sql;
const limit = Math.max(1, Math.min(opts.limit ?? 50, 5000));
const threshold = Math.max(0, opts.thinThreshold);
// Source scope: array wins over scalar (canonical precedence).
const sourceCondition = opts.sourceIds && opts.sourceIds.length > 0
? sql`AND p.source_id = ANY(${opts.sourceIds}::text[])`
: opts.sourceId
? sql`AND p.source_id = ${opts.sourceId}`
: sql``;
// Re-enrich recency guard. enriched_at is written as toISOString() so a
// lexical text comparison is correct AND can't throw on a malformed value
// (a ::timestamptz cast would). Pages never enriched (NULL) are eligible.
const reenrichMs = opts.reenrichAfterMs ?? 0;
const recencyCondition = reenrichMs > 0
? sql`AND NOT (
p.frontmatter ->> 'enriched_at' IS NOT NULL
AND p.frontmatter ->> 'enriched_at' > ${new Date(Date.now() - reenrichMs).toISOString()}
)`
: sql``;
// Whitelisted ORDER BY (no injection — enum maps to a literal fragment).
const orderKey = ENRICH_ORDER_SQL[opts.order] ? opts.order : 'inbound-links';
const orderBy = sql.unsafe(ENRICH_ORDER_SQL[orderKey]);
const rows = await sql`
SELECT
p.slug,
p.source_id,
p.title,
p.type,
(char_length(p.compiled_truth) + char_length(COALESCE(p.timeline, ''))) AS body_len,
COALESCE((
SELECT COUNT(*)
FROM links l
WHERE l.to_page_id = p.id
AND l.link_source IS DISTINCT FROM 'mentions'
), 0)::int AS inbound_count
FROM pages p
WHERE p.deleted_at IS NULL
AND p.type = ANY(${opts.types}::text[])
AND (char_length(p.compiled_truth) + char_length(COALESCE(p.timeline, ''))) < ${threshold}
${sourceCondition}
${recencyCondition}
ORDER BY ${orderBy}
LIMIT ${limit}
`;
return rows.map((r: Record<string, unknown>) => ({
slug: String(r.slug),
source_id: String(r.source_id),
title: String(r.title ?? ''),
type: r.type as EnrichCandidate['type'],
body_len: Number(r.body_len ?? 0),
inbound_count: Number(r.inbound_count ?? 0),
}));
}
async findAnomalies(opts: AnomaliesOpts): Promise<AnomalyResult[]> {
const sql = this.sql;
const sigma = opts.sigma ?? 3.0;