Build Log · live execution

Building it, in the open.

The plan is done; this is the record of turning it into a running system. Each entry is a merged PR that passed the gate (lint, typecheck, tests). Newest first.

Repo Niftory/ai-mindshare-sift private · pnpm workspace · scheduled Actions on Ubicloud · R2 + D1 · TanStack Start
● LIVE ai-mindshare.pages.dev · real X data — 146k tracked mentions across ChatGPT, Fable 5, Claude Code, Cursor

Build-pack progress

0 · Bootstraprepo + plan import
1 · Scaffold#1
2 · Contracts#3
3 · Scores#4
4 · Collector#5
5 · Enrichment#6
6 · Rollups#7
7 · Orchestrator + Site#8
88 · R2 + Actions schedulenext
99 · Author enrichment

Log

Prototype build-out — the full roadmap surface set

2026-06-13#20#21#22#23#24all live

A focused sprint took the demo from "one rich entity + a 4-row board" to the roadmap's v1/v2 surface set. The collector + enricher are being built separately, so the data stays mocked by design; everything user-facing is real.

  • Widened the field to 14 entities (#20): 10 compact per-entity profiles expanded by a deterministic generator, so the leaderboard and every aggregate feel like a product. The 4 real-data entities keep their embeds + threads.
  • Cross-entity surfaces (#21): /good-at/<tag> (which model wins each skill), /themes/<topic> (who gets roasted on pricing), and the Gap — the benchmarks-vs-vibes scatter, the shareable artifact. Built on new themes/gap aggregate artifacts + the existing tags schema.
  • Credibility pages (#22): /methodology and /open — the rigor that survives Hacker News, including the X-data permission boundary.
  • Narrative layer (#23): a live change-feed ticker on the board, and the editorial launch read for Fable 5 (KPI band + real launch-spike chart + receipts), on the events/reads schemas.
  • Good-citizen attribution (#24): Voices link out to X profiles by id; Twitter-card meta for clean unfurls.

Also cleared: the X contract — written approval to run on the enterprise token (summary + embeds only), which the architecture already satisfies. The one gate left is data correctness (query precision + real enrichment), owned by the collector/enricher track.

Thread shape-map rows — discourse glyphs

2026-06-13PR #19live · the distinctive view

Threads went from giant stacked embeds to a tight scannable list. Each conversation is one row carrying a shape-map: a spine of dots, one per reply, sized by engagement and colored by who replied (lab / builder / media / anon). Click a row to expand the real embeds.

  • The leak rule means only the X embed may show text/handles, so the collapsed row leans on compliant aggregates — an identity bucket + an engagement weight per reply — instead of custom tweet cards.
  • fetch-threads classifies each reply + root author (lab/media/builder/anon, from handle + verified + followers — a stand-in for build-pack-9's LLM author classification) and records engagement as the dot size. Re-pulled all 12 real threads with the mix.
  • Contract carries root_identity + reply_nodes (post_id / identity / weight) — IDs + buckets only, still leak-clean.

Thread expand UX — inline accordion

2026-06-13PR #18live

Every conversation collapses to just the root tweet + a stat bar; "show N replies" reveals the real reply spine in place (the spine animates in, grid-rows 0fr→1fr). Reply embeds are now lazy on expand — a collapsed page loads 7 iframes instead of ~20, and all the thread roots stay scannable.

Full threads — real root + reply spine

2026-06-13PR #17live · 12 real threads pulled

Threads now render whole conversations, root to reply, pulled live from X — not just a root link.

  • fetch-threads seed builder hits the API raw to read referenced_tweets, rebuilds the real reply tree (true depth + an engagement-ranked spine in chronological order), and commits thread-seed.json — so there's no live API at build time. Pulled 12 real threads across the 4 entities (e.g. Fable 5's top conversation: 71 replies, 50 people, depth 5).
  • Thread component: root embed + a connected reply spine + collapse/expand (2 shown, "show N more" reveals the rest). reply_post_ids are IDs-only, so the leak rule still holds.
  • Lazy embeds: an IntersectionObserver mounts each tweet only when it scrolls near, so ~10 embeds per page stay smooth.

Real X embeds — key posts + threads

2026-06-13PR #16live · 5 iframes, 0 fallbacks

The entity page now renders real X embeds instead of bare links. Verified on the Fable 5 page: 5 live tweet iframes, no fallbacks.

  • PostEmbed: loads widgets.js once, renders theme-aware tweets (light/dark follow the active brand theme), 4.5s timeout, styled fallback card if a widget can't load — the embeds-only display path keeps us compliant (IDs + aggregates in JSON, X renders the text).
  • Key posts grid embeds the top 4 resonant posts; Threads embeds the top thread root + stat rows (score, replies / participants / depth, relationship).
  • Collector switched to relevancy sort so we surface high-engagement posts (good thread roots), not the newest/lowest-engagement set.

Hardening — CI on Ubicloud, LLM client, copy

2026-06-13#15#14#12gate ~16s
  • CI moved to Ubicloud (standard-4): the lint + typecheck + test gate runs in ~16s.
  • LLM client sets max_tokens (some providers 400 without it) + a vet-enrich dev tool that prints real tweet text next to model labels for comparison.
  • Copy pass: shorter headings, dropped the AI-ish subtitles + em-dashes.

Test suite + mobile + launch-ready

2026-06-13PR #11gate green · 79 tests
  • Data test suite: the synthesizer became a pure function; a suite validates every artifact (index + 4 summaries + launches) against the schemas + leak check, and asserts content invariants — never surfaces 'novelty', ranked leaderboard, Fable 5's full surface set populated, voice scores in range, themes carry receipts.
  • Mobile-optimized: nav wraps, leaderboard drops to #/Model/Mindshare/Vibe, KPIs 2-col, launch card + grids stack, facets/verdicts reflow. Verified at 390px.
  • Launch polish: meta description, OG tags, favicon. 79 tests green.

Full entity surfaces + complete-mock data

2026-06-13PR #10live

Filled the entity page out to the whole vision, on real volume + rich enrichment. The Fable 5 page is authored from a genuine read of the launch conversation.

  • What-people-say consensus paragraph + per-theme verdicts + ✓/✗ aspect chips; like/dislike themes with facets (token limits 70% …) and field-rate badges.
  • Voices tabs (community / official / rival) ranked by voice score with identity badges + rising; owned-vs-organic split; capability scorecard; good/bad-at tags; deep threads; a launch score card (87, live).
  • Dropped the novelty theme (too vague). All artifacts still schema-valid + leak-free.

🚀 Deployed live

2026-06-13PR #8live

From "runs in a script" to a deployed site with real data: ai-mindshare.pages.dev.

  • Orchestrator (run.ts) loops a real entity set (ChatGPT, Fable 5, Claude Code, Cursor), ranks by mindshare, emits schema-valid + leak-checked index.json + per-entity summary.json. 146k real tracked mentions.
  • TanStack Start site on Cloudflare Pages — leaderboard + entity pages, inline-SVG volume charts (Fable 5's shows the real Jun 9 launch spike), top posts linking to X.
  • Sift brand-book 4-theme token system with a header switcher (Sift / Sift Dark / Light / Dark) — flips the whole site, no hardcoded hex. 69KB gz JS.

Rollup → first real public artifact

2026-06-13PR #7gate green · 72 tests

Enriched posts now roll up into a real summary.json — the thing the site will serve.

  • Sentiment, like/dislike themes (engagement-ranked receipts), top posts, vibe score, capability scorecard, emergent tags — all from the classified sample + the counts backbone.
  • A live Fable 5 pull emits a valid artifact: 48,112 mentions, vibe 46, pricing/novelty/limits themes, schema-valid, 0 leaks (post IDs only — no tweet text or handles).
  • Author-dependent surfaces (voices, threads, owned/organic) default until author enrichment lands; the artifact is valid without them.

First real pull + runtime decided

2026-06-13live data

Ran the pipeline against real X for Fable 5 (no longer the synthetic fixture):

  • 47,998 real mentions over ~7 days, peak 3,801/hr at the Jun 9 17:00 UTC launch spike; 100 posts sampled, 39 caught by spam heuristics, 61 classified.
  • Confirmed the collector, spam filter, and enrichment work on live data end to end.

Runtime decided: the pipeline runs as a scheduled GitHub Actions workflow on Ubicloud runners (cron + compute in one, parity with the main repo's CI, dirt cheap) — not Cloudflare Workers, not a self-hosted box. Storage stays R2 (raw lake + published JSON, the CDN speed layer) + D1 over its HTTP API (the working set across runs). Secrets from Infisical. Idempotent re-runs heal anything missed; the Actions run history is the tracking.

Enrichment + first end-to-end pull

2026-06-12PR #6gate green · 64 tests

The LLM stage, and the first runnable pull that walks the whole pipeline.

  • LLM client over the OpenAI-compatible API (OpenRouter-ready, incl. free models), with fence-stripping and a one-shot repair retry. A deterministic MockLLM runs the path offline.
  • The classify.v1 prompt + schema, spam heuristics, the classifyBatch orchestrator (retweets never classified, spam filtered before the LLM), and the consensus citation validator.
  • pnpm pull --entity <slug> --max 100: registry → collect → enrich → summary. Proven in --fixtures mode on the Fable capture (16 posts → 7 RTs out → 9 classified). The same command does a real pull once the X + LLM keys are set.

Decision · site framework → TanStack Start

2026-06-12spec

Switched the site spec from Astro to TanStack Start (the house framework), deployed pure-static to Cloudflare Pages with public routes prerendered for SEO + OG cards, then client-hydrated for the live data. No SSR server in production.

Collector

2026-06-12PR #5gate green · 18 tests

The privacy boundary and the first pipeline stage — zero live API calls, all mock/fixture tested.

  • The Collector interface (counts + posts); everything downstream consumes NormalizedPost and never learns the transport.
  • XOfficialCollector: X API v2, injected fetch, relevancy/recency pagination that stops on a missing next_token (the quirk from the gotchas file).
  • FixtureCollector replays the real June-9 Fable capture through the same interface for tests and local dev.
  • Pure budget guard: per-entity daily caps + global monthly kill-switch at 90%.

Scores

2026-06-12PR #4gate green · 12 goldens

The coined numbers as pure, versioned functions.

  • Vibe Score (standing favorability): recency-weighted net sentiment minus theme drag, volume excluded, null below 30 posts.
  • Launch Score (per launch): velocity / builder / sentiment / durability, provisional weighting flipping to final at T+72h.
  • 12 golden tests pin exact outputs (60/10/30 → 88; provisional launch → 64, final → 68) for determinism.

Contracts package

2026-06-12PR #3gate green · 20 tests

The data contracts everything builds on: @avb/contracts.

  • Zod schemas for the registry, NormalizedPost, Classification, and all seven public artifacts (index, summary, tags, events, launches, reads, manifest).
  • The voice model: relationshipOf(author, entity, trackedLabs) derives owned / affiliated / rival / community per pair. Tested with the @sama case (owned on GPT-5.2, rival on Fable 5 from one record).
  • The query compiler (512-char guard) and the leak check that bans raw text, handles, names, source, and @handle values from public JSON.
  • Validated against the real 21-entity registry seed and real-Fable example artifacts.

Workspace scaffold

2026-06-12PR #1gate green

Tooling and the gate that every later PR has to pass.

  • pnpm workspace, biome (100-char, no semicolons), strict TypeScript, vitest.
  • GitHub Actions CI running lint → typecheck → test on every PR.
  • @avb/core: the structured log() helper and SCHEMA_VERSION.

Repo bootstrap

2026-06-12main

Created Niftory/ai-mindshare-sift (private) and imported the full plan + build pack into docs/plan/. Working agreement: branch per build-pack step, PR through the gate, this log updated each merge.