Anatomy of a living game engine

1 · The cast of models, and who does what

Trivia & Tunes doesn't lean on one big model. It routes each task to the model that's best (and cheapest) for it, with a fallback ready when the primary is slow or refuses. Routing is data-driven — a table of rules with sensible launch defaults baked in.

Every call routes through a LiteLLM gateway, so the provider behind a model name can change without touching game code. And every call is priced and logged (more on that below) — so the business knows the marginal cost of a question, not just the bill.

2 · One question, end to end

Here's the per-question pipeline, from a player's voice to the host's reply:

flowchart LR
  A[Player answers aloud] --> B["Whisper STT (medium)"]
  B --> C[Grade: Azure GPT-4o-mini]
  C -->|primary slow or refuses| C2[Fallback: Nova-lite]
  C --> D[Verdict + score 0-10]
  D --> E[Enrich: trivia-facts RAG]
  E --> F["Host line: Claude Haiku (Bedrock)"]
  F --> G[TTS: ElevenLabs or Piper - Finn speaks]
  C --> T[(ai_usage_events: cost telemetry)]
  F --> T
      

Two details matter. First, grading returns a numeric score and a strictness level, not just yes/no — which is what lets a host be lenient on a near-miss. Second, before the host speaks, the engine does a quick retrieval for a fun fact about the answer, so commentary can be genuinely informative, not generic.

3 · The muse — retrieval as memory

This is the part that makes the host feel alive. The muse is a set of CaveauAI corpora reached over a small REST surface (/api/v2/search and /api/v2/documents). Five corpora do distinct work:

• host-muse — the personality memory: notable moments from games, scoped per host.
• trivia-facts — curated facts for commentary enrichment.
• grading-rules — answer-variant rules and edge cases that keep grading consistent.
• qa-history — every graded Q&A pair from every finished game (the dataset, growing).
• question-bank — question metadata with enriched notes.

The muse — a curated corpus of expertly-calibrated quiz clues — underpins the question quality; the host-muse memory layers personality on top. Before the host speaks at a payoff moment, the engine recalls a memory or two; after a payoff moment, it remembers the line it just delivered:

flowchart TB
  M[Moment: round_intro / reveal / standings] --> R{recall}
  R -->|query = host id + context| S[search host-muse corpus]
  S --> F[Filter to this host, max 320 chars]
  F --> P[Inject 1-2 memories into host prompt]
  P --> L[Host delivers a callback]
  L --> W{remember}
  W -->|reveal / standings only| D[ingest: save payoff moment]
  D -. compounds .-> M
      

Memories are scoped by a stable host identity tag (e.g. host:42), because the corpus is account-wide but a host's room is their own. Recall only fires at the moments where a callback can land naturally — round intros, reveals, standings — so the host sounds intentional, not chatty.

4 · How retrieval actually works (RAG, briefly)

Each corpus document is chunked, embedded, and stored for hybrid search: dense vector similarity in Qdrant alongside keyword matching, fused and reranked so the host gets the relevant memory, not merely a lexically similar one. Queries are built from live context — the category, the round label, the correct answer, the current leader — then filtered down to this host's tagged memories. Short, on-topic, in character.

5 · Play becomes a dataset

Here's where "it learns" stops being a slogan. When a game finalizes, its graded answers are exported as a single document into qa-history. One upload per game; CaveauAI's ingestion pipeline chunks it for search. Over weeks, this becomes a substantial, real corpus of how people actually answer — the raw material for everything that comes next.

flowchart LR
  G[Game session ends] --> Q[ai_grade_log: finalized rows]
  Q --> X[exportSessionToCorpus]
  X --> Y[One qa-history document per game]
  Y --> Z[(CaveauAI corpus)]
  Z --> H[Grading recalls precedent]
  Z --> K[Question-bank enrichment]
  H -. compounds .-> G
      

6 · Knowing the unit economics

Every model call writes a row to ai_usage_events: task, model, provider, token counts, latency, retries, cache hits, success — and an estimated cost computed from a versioned rate-card table. Because grading and hosting are routed separately, the business can see the cost of a question and a night, attribute it per account, and tune routing for margin. This isn't a guess at AI spend; it's a ledger.

7 · The flywheel: the tool becomes your AI

Now stack it up. Play generates graded data and routing telemetry. That data already makes the host sharper and grading fairer today. The honest roadmap — clearly coming next, not shipped — uses the same accumulated signal for two things:

• Predictive difficulty — model the room and auto-calibrate question sets so a night is challenging but winnable.
• A sovereign fine-tune — distil a private host model from the platform's own play, so the personality and grading judgment live in weights you own, not just in prompts.

flowchart LR
  A[Accumulated qa-history + routing telemetry] --> B[Predictive difficulty model]
  A --> C[Sovereign fine-tune dataset]
  B -.coming next.-> D[Auto-calibrated question sets]
  C -.coming next.-> E[Private fine-tuned host model]
  D --> F[Better games]
  E --> F
  F -. compounds .-> A
      

The whole picture

All diagrams (editable) ← The Experience Overview