Guide

A reusable, self-updating, AI-assisted SEO engine: always-correct technical SEO (sitemap, robots, JSON-LD, canonical, meta/OG, hreflang) computed automatically from your live content, plus AI-proposed content SEO (keyword clusters, per-page meta copy, landing-page + content-gap ideas) that a human approves. It understands a project from its own docs and can be re-run anytime. Pure ESM cores, zero dependencies, no build step.

This is the full guide. For the 30-second version see README.md; for the concrete Bea's-Stories install see INSTALL-beastories.md (in the licensed distribution).

1. Honesty note (set expectations first)

No system can guarantee a #1 Google ranking. Rankings are Google's call — relevance, authority, technical health, and freshness. What this engine does is make a site maximally rankable and keep it that way with minimal human effort. It does not promise literal first place, and it deliberately does not mass-auto-publish AI content, which is the #1 cause of Google spam penalties. The AI proposes; a human approves; the technical artifacts are always correct.

2. The autonomy model (read this — it's the design)

Two clean halves, split by whether a machine can be trusted to act unattended:

The AUTO side is pure, deterministic derivation from your live content (SeoEntity[]) — it can't "go wrong", so it's served directly and stays correct as content changes. The PROPOSED side is everything the AI generates: it lands in seo_proposals as pending and mutates nothing until a human approves it (an approval promotes the payload into seo_keywords / seo_page_meta via an RPC).

3. Architecture (five parts)

CONTEXT INGESTION  →  ENGINE (Claude)  →  { AUTO artifacts | PROPOSED inbox }  →  ADMIN COCKPIT
   CLI seed (repo docs)     pure cores + 1 edge fn      reader routes             admin "SEO" area
   "Update SEO" (GitHub raw)                            + JSON-LD / meta / hreflang

1. Context ingestion. The engine learns the project from its own docs (context.md, CLAUDE.md, docs/…) — via GitHub raw for the admin "Update SEO" button, or straight from disk for the CLI seed — plus a bounded sample of live content from the adapter. 2. Engine. Pure cores (core/*.mjs) build an injection-safe prompt, call Claude, and parse/ normalize the output. The only I/O lives in one thin backend shell (backend/seo-generate.mjs). 3. Auto artifacts. sitemap.xml, robots.txt, JSON-LD, canonical, meta/OG, and hreflang are computed by the cores from listIndexableEntities() + approved seo_page_meta and served by the front-end. No human step. 4. Proposed inbox. AI-generated keywords / meta / landing-page / content-gap suggestions are written to seo_proposals (pending), for one-click approve / edit / reject. 5. Admin cockpit. A "SEO" area: Profile (+ "Update SEO" trigger), Keywords manager, Proposals inbox, Health dashboard.

Portability heart: all logic is pure ESM .mjs cores + .d.ts typedefs, node --test, zero deps. The same cores import cleanly into a Supabase edge function (Deno/TS), a Vercel Node function, and a browser/Vite/React-TS front-end. The cores never do I/O and never reference a project domain concept — all project data reaches them through the adapter as a uniform SeoEntity[].

4. The SeoEntity contract (the one shape the cores know)

The adapter is the single seam through which project data reaches every core. From core/types.d.ts:

/** One indexable page, produced by a project's adapter. */
export interface SeoEntity {
  /** Free-form project type tag, e.g. 'story' | 'author' | 'page'. */
  type: string
  /** Absolute-from-root path, e.g. '/stories/the-blue-house'. */
  url_path: string
  title: string
  /** Short excerpt used for AI context + thin-content checks. NEVER paywalled full text. */
  body_sample: string
  /** BCP-47 language tag, e.g. 'en', 'fr', 'mfe'. */
  lang: string
  /** ISO 8601 last-modified timestamp. */
  updated_at: string
  /** schema.org type to emit, e.g. 'Article' | 'Person' | 'WebPage'. */
  schema_type: string
  /** Translations of this entity, for hreflang alternates. */
  alternates?: { lang: string; url_path: string }[]
  /** Absolute image URL for OG/Twitter cards + Article schema. */
  image_url?: string
}

The SeoProfile shape (doc-derived, admin-editable, steers the engine and JSON-LD/meta defaults) is also in types.d.ts: siteName, siteUrl, description, and optional geography, languages, audience, competitors, logoUrl, sameAs, twitterCard.

The rule that keeps you safe: body_sample is an excerpt — a first line / summary, never the paywalled or full body text. It exists only for AI context and thin-content checks.

5. Authoring an adapter

Copy adapter/seo.adapter.example.mjs → seo.adapter.mjs and implement the one function the contract (adapter/adapter.d.ts) declares:

export function listIndexableEntities(): Promise<SeoEntity[]>

Rules (from the contract):

• Return only publicly-indexable, published entities.
• body_sample is an excerpt only — never expose paywalled/full body text.
• Include alternates for translated entities so hreflang is emitted.

The shipped example returns two hard-coded entities (a story/Article with a French alternate and an author/Person) so the module is runnable and testable standalone:

export async function listIndexableEntities() {
  return [
    { type: 'story', url_path: '/stories/the-blue-house', title: 'The Blue House',
      body_sample: 'A short tale about a house by the sea…', lang: 'en',
      updated_at: '2026-07-08T00:00:00Z', schema_type: 'Article',
      alternates: [{ lang: 'fr', url_path: '/fr/stories/the-blue-house' }],
      image_url: 'https://example.com/covers/blue-house.png' },
    { type: 'author', url_path: '/authors/bea', title: 'Beatrice Bellepeau',
      body_sample: 'Mauritian journalist, entrepreneur, and lifelong storyteller.',
      lang: 'en', updated_at: '2026-07-01T00:00:00Z', schema_type: 'Person' },
  ]
}

A real adapter queries your DB (using your existing gated reads), maps each published row to a SeoEntity, and derives alternates from your translation table. Because every core consumes this uniform shape, no core ever references a "story" or "author".

6. Context ingestion — docs_source, the CLI seed, and "Update SEO"

The engine derives the project profile from the project's own docs. Two paths feed it the same way:

• CLI seed (cli/seed-seo-profile.mjs) — run at setup (and any time). It reads the configured doc

files from the local repo, concatenates them, and POSTs them to the seo-generate endpoint with INTERNAL_SECRET. Config via env: SEO_ENDPOINT, INTERNAL_SECRET, SEO_DOCS (comma-separated paths), optional SEO_REPO_REF. It sends { mode: 'cli', docs, docs_source }, so the engine uses the supplied docs verbatim. --dry-run prints the assembled payload without POSTing. Zero deps (Node ≥ 18 for global fetch).

• "Update SEO" button (admin) — fires seo-generate with an admin JWT. Here the engine calls

fetchDocs(docsSource) to pull the latest docs from GitHub raw using the stored docs_source = { repo, ref, paths[] } + a server-side GITHUB_TOKEN (same fetch pattern as the github-json-cms component). This is how the profile self-updates against the latest docs.

docs_source is one of the two consumer seams: it is stored on seo_project_profile and re-used by every subsequent "Update SEO".

7. The pure cores (core/*.mjs)

All cores are pure ESM: no I/O, no DOM, no hidden clock, no project domain terms. One line each:

The engine's orchestration lives in backend/seo-generate.mjs (§9) and is the only place the cores are composed with real I/O.

8. The database (db/schema.sql)

All objects are prefixed seo_; RLS is admin-only on every table; every RPC is SECURITY DEFINER with EXECUTE revoked FROM PUBLIC (not just anon) and re-granted to authenticated, with an admin check inside. Apply via your project's normal migration flow; db/rollback.sql reverses it.

Tables:

• seo_project_profile (singleton, id=1) — profile JSONB, docs_source JSONB (`{ repo, ref,

paths[] }), generated_at, edited_by, updated_at`.

• seo_keywords — cluster, keyword (unique), intent, priority (1–5),

target_url_pattern, status (proposed|approved|active|rejected), search_volume (nullable — reserved for a future Google Search Console feed).

• seo_page_meta — url_path (unique), entity_type, title, description, og_title,

og_description, focus_keyword, source (auto|ai|manual), status (proposed|approved).

• seo_proposals (approval inbox) — type (keyword|meta|landing_page|content_gap), payload

JSONB, rationale, status (pending|approved|rejected), reviewed_by, reviewed_at.

• seo_health — url_path, issues JSONB, severity, checked_at (indexed on url_path).
• seo_settings (singleton) — model, autonomy JSONB ({auto_technical, propose_content}),

schedule.

RPCs (all admin-gated, SECURITY DEFINER, set search_path = public):

• seo_approve_proposal(uuid) — promotes a pending proposal's payload into seo_keywords /

seo_page_meta (keyword/meta) or records the decision (landing_page/content_gap), then marks it approved.

• seo_reject_proposal(uuid) — marks a pending proposal rejected.
• seo_upsert_keyword(text,text,text,int,text,text) — manual keyword add/edit (upsert on keyword).
• seo_set_page_meta(text,text,text,text,text,text) — manual per-page meta (upsert on url_path,

source='manual').

• seo_get_dashboard() — one JSONB rollup for the admin: keyword totals, pending proposals, pages

with meta, non-ok health rows, last-checked, profile generated-at.

Host requirement: seo_is_admin() + the REVOKE-FROM-PUBLIC note

The schema depends on one host predicate — public.seo_is_admin() returning true for the current admin user — used in every RLS policy and inside every RPC. Projects usually already have an admin check; alias it rather than re-implement:

create or replace function public.seo_is_admin() returns boolean
  language sql stable security definer set search_path = public as $$
    select public.is_admin();  -- alias your existing admin check
$$;

A commented reference alias sits at the bottom of schema.sql. Security note: the RPCs are SECURITY DEFINER, so their EXECUTE grant is explicitly revoked FROM PUBLIC (not merely from anon) and re-granted only to authenticated — revoking from anon alone leaves a PUBLIC grant that both anon and authenticated inherit, which would leak the functions. This is the recurring fm-supabase-rls gotcha; keep the revoke … from public block.

9. The engine backend (backend/seo-generate.mjs)

One runtime-agnostic function — all I/O injected via deps, so the same core runs on any host:

handleSeoGenerate(req, deps) => { status, body }

Flow (see the source + backend/_interface.md): 1. AuthZ — deps.auth(req) must return { ok: true } (admin JWT or valid INTERNAL_SECRET header) or it 401s. 2. Docs — CLI mode (body.mode === 'cli') uses the posted body.docs; otherwise deps.fetchDocs(docsSource) pulls the latest from GitHub raw. 3. Live content — deps.listIndexableEntities() (your adapter). 4. Health (AUTO) — auditEntities(entities, pageMeta) is always computed + saved (deps.db.saveHealth) — the deterministic side runs even if the model fails. 5. Model — buildSeoPrompt(...) → deps.callClaude(prompt); the output is fence-stripped and JSON-parsed fail-safe (invalid JSON → nothing written, returns the health + an error). 6. Profile — persisted only if the model returned a profile object (deps.db.saveProfile). 7. Proposals — parseProposals + normalizeKeywords (deduped against existing keywords), then deps.db.insertProposals as pending. It never mutates already-approved rows.

The deps contract (from _interface.md):

Required env: INTERNAL_SECRET, GITHUB_TOKEN (for fetchDocs), the Claude API key. backend/_interface.md gives ready-to-adapt Supabase Deno and Vercel Node shells; the whole engine is a single serverless function (stays under the Vercel 12-function cap).

10. The serving side (always-on, front-end)

Thin front-end additions that read approved data and the live adapter — no admin action needed:

• sitemap.xml route — buildSitemapXml(listIndexableEntities() + approved landing pages),

served with a cache header. Use buildSitemapIndex if you split by section at scale.

• robots.txt route — buildRobotsTxt({ sitemapUrl, disallow }) pointing at the sitemap.
• JSON-LD injection — on each route, pick the schema builder by entity.schema_type

(articleSchema / personSchema / websiteSchema / …), then renderJsonLd(obj) inside a <script type="application/ld+json"> (a small <JsonLd> helper). renderJsonLd escapes </ so it can't break out of the script tag.

• Meta / OG — extend each route's meta with mergeMeta({ profile, pageMeta, entity, siteUrl });

the admin's approved seo_page_meta row overrides the entity-derived and profile defaults.

• hreflang — buildHreflang(entity, siteUrl) emits an <link rel="alternate" hreflang=…> per

translation plus x-default, for every translated entity.

These stay correct automatically as content changes — that is the AUTO half of the autonomy model.

11. The admin cockpit

A new "SEO" area, RBAC-gated behind your admin auth. Reference panels ship in ui/ as self-contained vanilla-JS examples (profile.example.html, keywords.example.html, proposals.example.html); port them to your admin framework.

• Profile — view/edit the doc-derived profile; shows docs_source; the [Update SEO] button

fires seo-generate (GitHub-raw docs path) and streams status.

• Keywords manager — the cluster table; filter by status/priority/intent; add/edit manually

(seo_upsert_keyword); approve proposed.

• Proposals inbox — approve / edit-then-approve / reject each AI proposal

(seo_approve_proposal / seo_reject_proposal).

• Health dashboard — coverage %, pages missing meta/schema, warnings, last-audit time — driven by

seo_get_dashboard().

12. Testing

npm test    # → node --test

Because the cores are pure, confidence comes from unit tests over plain objects (core/*.test.mjs): sitemap XML shape + index, robots, all JSON-LD builders, hreflang incl. x-default, meta merge precedence, keyword normalize/dedupe, proposal parse fail-closed, and the health rules. The engine orchestration (backend/seo-generate.test.mjs) is tested at the interface level with injected deps — auth gating and the fail-safe (no mutation of approved rows on malformed AI output). Front-end and admin wiring are tested in the consuming project (e.g. vitest for the <JsonLd> helper, meta merge, the sitemap route, and the approve/reject panels).

13. Future: Google Search Console plug-in

The health dashboard has a documented seam for a free Google Search Console feed. Today the health signal is a deterministic audit of your own content (missing meta, thin content, no schema/image). GSC is Google's own free product that reports real search performance — impressions, clicks, average position, and the exact queries a page ranks for.

The schema is already primed for it: seo_keywords.search_volume is a reserved nullable column, and seo_health / seo_get_dashboard() can absorb per-URL rank/impression rows. A future plug-in would authenticate to the GSC API (server-side, feature-flagged by presence of a token — the same env-presence pattern the rest of the stack uses), pull the last-N-days performance per URL/query, and merge it into the health dashboard so admins see actual ranking movement next to the audit — and so the keyword manager can prioritize by real demand instead of guesswork. It stays a plug-in (not built in v1) to keep the core engine zero-dependency and free of third-party lock-in.

v2 — Autonomous Research Agent

v2 evolves the engine from a prompt-once generator into an autonomous SEO research agent. v1 learned the project from its docs and proposed content SEO in a single shot; v2 keeps doing that, and adds a governed, budget-capped loop that *perceives* multiple sources (docs, the live site, the live web, competitors, search rankings), *remembers* what it learns in a versioned knowledge base, *reasons* with tool-using researchers, and *acts* by keeping technical SEO perfect and proposing content/ranking improvements for human approval. Everything ships as an extension of this same seo/ module — same zero-dep pure cores, same two project seams (content adapter + config), plus a new admin-configurable SERP provider (Google default).

Everything in v1 still applies. v2 adds tables, cores, adapters, and researcher shells alongside it.

v2.1 The agent loop

PERCEIVE (sources) → REASON (researchers, tool-use) → REMEMBER (knowledge base) → ACT (auto artifacts + proposals)
                        └──────────── CONTROL LOOP: scheduler + budget governor ────────────┘

• Perceive — re-fetch the registered sources (project docs, own site, SERP results, competitor

pages), hash each, and diff against the stored hash. Only what changed is re-investigated (unless a full refresh is explicitly requested).

• Reason — the researchers (thin shells over pure cores + adapters) run tool-using analysis:

derive the knowledge base, monitor rank, discover + analyze competitors.

• Remember — write findings to the append-only, versioned knowledge base (seo_knowledge) and the

rank/competitor time-series.

• Act — technical SEO stays auto (unchanged from v1); content, keyword, and ranking findings land

in seo_proposals as pending, each with provenance, for one-click admin approval.

• Control loop — a scheduler (manual / change-triggered / pg_cron tiered cadence) drives the

loop; the budget governor stops it cleanly the moment the monthly cap is reached.

v2.2 The v2 data model (db/schema.v2.sql)

schema.v2.sql is additive — apply it *after* v1 schema.sql. Same rules as v1: everything is seo_*, RLS is admin-only on every table, RPCs are SECURITY DEFINER with EXECUTE revoked FROM PUBLIC and re-granted to authenticated. It is idempotent and requires the same host predicate public.seo_is_admin(). db/rollback.v2.sql reverses it.

New tables:

Extends v1 tables: seo_keywords gains is_target, tier, difficulty; seo_proposals gains provenance jsonb + a widened type check that now allows ranking; seo_settings gains serp_provider (default google), serp_key_ref, budget_cap_cents_month, cadence jsonb (per tier), and enabled_researchers jsonb.

New RPCs (all admin-gated, SECURITY DEFINER, search_path = public): seo_get_rank_dashboard(), seo_list_competitors(), seo_upsert_competitor(text,text,text), seo_set_competitor_relevance(uuid,text,boolean), seo_get_competitor_analysis(uuid), seo_set_settings(jsonb), seo_get_settings() (rolls current-month spend into the payload), seo_enqueue_run(text,jsonb), and seo_get_agent_health() (latest self-checks + open circuits + learning stats for the Agent Health panel).

v2.3 The new pure cores (core/*.mjs)

Same discipline as v1: pure ESM, no I/O, no DOM, no clock, no project terms, node --test. One line each:

v2.4 The adapter seams

v2 adds three swappable, fakeable adapters alongside the v1 content adapter. All are keyed server-side and admin-configured; Google is the default SERP/search provider.

• SerpProvider (adapter/serpProvider.*) — `search(query, {location?, num?}) → {results:

{url,title}[]}. Ships **serpProvider.google.mjs** (Google Programmable / Custom Search JSON API, fetchImpl injected, fails soft to {results:[]}) + serpProvider.fake.mjs. The interface allows other providers (serpapi/brave/bing/dataforseo) chosen via seo_settings.serp_provider; all share the serpParse` normalizer. Rankings come from a provider API, never by scraping Google directly.

• WebSearch (adapter/webSearch.*) — search(query, {num?}) → {results: {url,title,snippet?}[]},

the CompetitorResearcher's search tool. Ships webSearch.google.mjs + webSearch.fake.mjs.

• FetchPage (adapter/fetchPage.*) — fetch(url) → {ok, url, text}, reads a competitor page as

text. Ships fetchPage.fake.mjs for tests. Every fetch is SSRF-guarded via fetchGuard.isSafeUrl before it runs.

An optional SearchConsole adapter (own average positions/impressions, free) is a documented plug-in, not required by the v2 core. The v1 content adapter (listIndexableEntities()) is unchanged.

v2.5 The researchers (backend/*.mjs)

Thin, runtime-agnostic shells that compose the cores + adapters with injected I/O (deps). Each is independently toggleable (seo_settings.enabled_researchers) and unit-tested against fakes:

• researchPipeline.mjs — runContentResearch(input, deps) (Phase 2) — runs the Docs + Site

researchers, merges/tiers the keyword universe (expandKeywords), writes the versioned KB, and emits keyword + content_gap proposals with provenance.

• rankMonitor.mjs — runRankMonitor(input, deps) (Phase 3) — for each target keyword:

SerpProvider.search → serpParse → save our_position + top results to seo_rank_history, then buildRankProposals for anything not #1. Budget-aware: stops at the cap and returns capped.

• competitorDiscovery.mjs — runCompetitorDiscovery(input, deps) (Phase 3) — from rank results,

surface domains that outrank us (registrableDomain + isCompetitor, deduped/ignore-listed) and upsert them as discovered competitors.

• competitorResearcher.mjs — runCompetitorResearch(input, deps) (Phase 3, agentic) — per

enabled competitor: WebSearch → FetchPage (each SSRF-guarded) → Claude analysis → save seo_competitor_analysis + emit gap-closing proposals with provenance. Budget-aware.

• jobWorker.mjs — drainJobs(input, deps) (Phase 4) — drains a batch of claimed seo_jobs,

dispatching each to its type handler, canSpend-gated, marking each done/failed/capped.

v2.6 Autonomy — jobs worker + budget governor + cadence

• Triggers (one shared pipeline): manual ("Update SEO" / "Refresh rankings"); change-triggered

(source hash diff via diffSources); and pg_cron on a tiered cadence — priority ranks daily, the rest / analysis / discovery weekly (from seo_settings.cadence).

• Chunked jobs (edge-timeout safety): a run enqueues seo_jobs; the worker edge function drains a

few per invocation (idempotent, resumable), so each invocation stays short and the whole sweep finishes across invocations. pg_cron both enqueues scheduled runs and pokes the worker.

• Budget governor: before every costed step, budgetGovernor.canSpend(monthSpend, cap, nextCost);

if it can't, the run/job is marked capped and stops. Every costed step writes a seo_budget_ledger line, and seo_get_settings() surfaces current-month spend next to the cap. The cap is a hard stop against runaway external spend.

Deploy adds one new serverless function (seo-worker) beside seo-generate, staying within the Vercel 12-function cap.

v2.7 Phase 5 — Self-Ops & Learning

Makes the agent self-testing, operationally self-healing, and feedback-driven — without self-modifying code (humans + tests still own all logic fixes; this is operational self-healing only).

> "Self-healing" here is NOT the hosted product's self-heal. In this document the word means > *operational resilience*: retry with backoff, provider fallback, circuit-breaking and > halt-on-regression — recovering from a failing dependency, with humans and tests still owning every > logic fix. The hosted Umbodi product uses the same word for something different and larger: a write > loop that merges its own pull request once checks are green and then verifies the deploy against > the live page. Both are real, neither implies the other, and only the hosted one touches a > repository. If you are reading this to find out whether the engine can repair your codebase: it > cannot, by design.

• Self-testing (SelfCheck): on the schedule, selfCheck.runSelfChecks verifies invariants →

seo_health_checks. Checks include: KB write-then-read round-trips, proposals persist, the SERP provider responds, anon is denied on seo_* (security regression), /sitemap.xml returns 200, and the last run had no unhandled error. This is the direct antidote to the class of silent-write bug that motivated v2 (Phase 0).

• Operational self-healing (bounded): retry-with-backoff on transient failures; **provider

fallback (primary SERP fails → next configured provider); circuit-breaker (circuitBreaker + seo_circuit per researcher/provider — open after N consecutive failures, disable + alert); and halt-on-regression** — if a SelfCheck fails, haltOnRegression pauses autonomous *actions* (reads/reporting continue) until cleared, so a broken run can't compound.

• Learning loop (seo_feedback + proposalRanker): record proposal outcomes (admin

approve/reject + whether rank improved after acting — rank history is the reward). The pure proposalRanker.scoreProposals biases future proposals: down-weight repeatedly-rejected tactics, up-weight those that historically moved rank. Gets smarter over time; no model training.

• Admin "Agent Health" panel: latest self-checks (green/red), open circuit-breaker states, recent

healing events, and learning stats — driven by seo_get_agent_health().

v2.8 Honesty note (still the design)

No system guarantees a #1 Google ranking. "Always be first" is the optimization *target*, not a promise — rankings remain Google's decision. v2 makes the site maximally rankable, continuously tracks where we actually rank and who outranks us, and proposes concrete improvements; a human approves anything Google reads as content (no mass-auto-publishing — the #1 cause of spam penalties). And to be exact about "autonomous": the agent self-*tests* and self-*heals operationally (retries, provider fallback, circuit-breakers, halt-on-regression) but does not modify its own code — humans and tests own every logic fix.