From 46a601dc7b337c76053f8bef21173a8d84a7b3e1 Mon Sep 17 00:00:00 2001 From: Markus Graf Date: Sun, 29 Mar 2026 11:41:05 +0200 Subject: [PATCH] docs(02): capture phase context (assumptions mode) --- .../02-CONTEXT.md | 98 +++++++++++++++++++ .../02-DISCUSSION-LOG.md | 40 ++++++++ 2 files changed, 138 insertions(+) create mode 100644 .planning/phases/02-gemini-integration-asset-pipeline/02-CONTEXT.md create mode 100644 .planning/phases/02-gemini-integration-asset-pipeline/02-DISCUSSION-LOG.md diff --git a/.planning/phases/02-gemini-integration-asset-pipeline/02-CONTEXT.md b/.planning/phases/02-gemini-integration-asset-pipeline/02-CONTEXT.md new file mode 100644 index 0000000..180f3e8 --- /dev/null +++ b/.planning/phases/02-gemini-integration-asset-pipeline/02-CONTEXT.md @@ -0,0 +1,98 @@ +# Phase 2: Gemini-Integration + Asset-Pipeline - Context + +**Gathered:** 2026-03-29 (assumptions mode) +**Status:** Ready for planning + + +## Phase Boundary + +Gemini-API funktioniert für Text und Bild, Character Sheets sind generiert und eingebunden. Umfasst: Config-Loader, Text-API-Client mit Retry, Bild-API-Client mit Character Consistency, Fallback-Texte und -Bilder, Rate-Limiting, Build-Script für Asset-Generierung, generierte Avatare im Welcome-Screen, Begleitfigur-Texte (Begrüssung, Buchstaben-Vorstellung, Wald-Kommentar). Kein Wald-Wachstum (Phase 3), keine Übungsphasen-Verbindung (Phase 3), kein Audio (Phase 4). + + + +## Implementation Decisions + +### Build Script Runtime +- **D-01:** `scripts/generate-assets.ts` runs in Node.js via `npx tsx` (new devDependency). Reads `public/config.json` via `fs.readFileSync`. The browser API client uses `fetch('/config.json')`. A shared isomorphic Gemini API module using only `fetch()` serves both contexts — only the config-loading layer differs. +- **D-02:** Build script generates 4 Character Sheets + 4 Avatars + 1 Style Reference, saves as PNG to `src/assets/companions/{type}/` and `src/assets/style-reference/`. These are checked into git. + +### IndexedDB Schema Extension +- **D-03:** Keep DB at version 1. API call counts stored as fields in the existing `settings` store (e.g. `apiCallsToday: number`, `lastApiCallDate: string`). No dedicated `apiLog` store — simple counting suffices for rate-limiting (GAPI-06). Detailed per-call logging is not required. +- **D-04:** Add read/write helper methods for `companionAssets` and `styleReference` stores in `db.ts`. These stores exist structurally from Phase 1 but lack accessor functions. + +### Welcome Screen Avatar Replacement +- **D-05:** Static imports in `characters.ts` for each avatar file. Each `CompanionDefinition` gets an `avatarUrl: string` field with the Vite-resolved URL. Welcome screen renders `` instead of emoji spans. Type-safe, explicit mapping. +- **D-06:** Fallback to emoji if avatar image fails to load (onerror handler on ``). + +### Gemini API Client Architecture +- **D-07:** Isomorphic API module `src/api/gemini.ts` with `generateText(prompt, systemPrompt)` and `generateImage(prompt, referenceImages[])`. Uses only `fetch()` — works in both Node.js 18+ and browser. +- **D-08:** Reference images sent as Base64-encoded `inlineData` (standard Gemini REST pattern). `response_modalities: ["IMAGE"]` for image generation. Verify exact format against current Gemini docs at implementation time. +- **D-09:** Retry logic: 3 attempts with exponential backoff (1s, 2s, 4s). On final failure, return null and let caller use fallback. +- **D-10:** Rate-limiting: Max 1 image + 1 text per exercise unit. Check counts in `settings` store before API call. Reset daily. + +### Fallback System +- **D-11:** Fallback texts stored as static arrays in `src/companion/fallbacks.ts`: 10 greetings, 10 forest comments, letter introductions for levels 1-6. All in simple German, max 25 words, no performance praise. +- **D-12:** Fallback images: 5-10 simple SVG illustrations in `src/assets/fallback/` showing generic forest elements (flower, mushroom, bird, tree, butterfly). Used when API unavailable or rate-limited. + +### Companion Text Integration +- **D-13:** Companion text module `src/companion/companion.ts` with `getGreeting(character, timeOfDay)`, `getLetterIntro(character, letter)`, `getForestComment(character)`. Each tries API first, falls back to static texts. +- **D-14:** Companion area shown on welcome/forest screens with greeting text. Companion does NOT appear during typing exercise in Phase 2 (Phase 3 adds "Entdecken" phase with companion letter introduction). + +### Claude's Discretion +- Exact prompt templates for Gemini API calls (following SPEC.md section 9.2 as guide) +- Build script CLI output format and error handling +- Asset file naming conventions within companion directories +- Config loader error handling strategy + + + +## Canonical References + +**Downstream agents MUST read these before planning or implementing.** + +- `SPEC.md` — Full specification, THE authoritative source +- `SPEC.md` section 5.4 — Visual consistency architecture (Character Sheets, Style Reference) +- `SPEC.md` section 6.4 — API key handling (config.json structure) +- `SPEC.md` section 9 — Gemini Integration (API endpoints, prompt templates, models) +- `SPEC.md` section 9.2 — Prompt templates for all companion text types +- `SPEC.md` section 9.3 — Character Sheet generation prompts and visual descriptions +- `src/types.ts` — Existing TypeScript interfaces (CompanionAssets, ForestElement, etc.) +- `src/storage/db.ts` — IndexedDB wrapper with 5 stores (companionAssets, styleReference need methods) +- `src/companion/characters.ts` — Companion definitions (needs avatarUrl field) +- `src/ui/screens.ts` — Welcome screen rendering (emoji → img replacement) +- `public/config.json` — Gemini API key and model names + + + +## Existing Code Insights + +### Reusable Assets +- `src/types.ts`: `CompanionAssets` interface (characterSheet: Blob, avatarImage: Blob, characterType, generatedAt), `ForestElement` interface, `StyleReference` interface already defined +- `src/storage/db.ts`: 5 IndexedDB stores created, `progress` and `settings` have CRUD methods, other 3 stores need method additions +- `src/companion/characters.ts`: 4 `CompanionDefinition` objects with name, type, emoji, personality — ready to extend with avatarUrl +- `public/config.json`: Already contains `geminiApiKey`, `geminiModel`, `imageModel` + +### Established Patterns +- All modules use TypeScript strict mode +- Functions export individually (no default exports) +- IndexedDB operations are Promise-wrapped with typed results +- CSS uses BEM-style class names with `--` modifiers + +### Integration Points +- Welcome screen `initWelcomeScreen()` in `screens.ts` — replace emoji cards with img cards +- `app.ts` `initApp()` — add greeting text display after detecting returning user +- `db.ts` — add `saveCompanionAssets()`, `getCompanionAssets()`, `saveStyleReference()`, `getStyleReference()`, `saveForestElement()`, `getForestElement()` methods +- `characters.ts` — add `avatarUrl` to each companion definition + + + +## Specific Ideas + +No specific requirements — open to standard approaches + + + +## Deferred Ideas + +None — analysis stayed within phase scope + diff --git a/.planning/phases/02-gemini-integration-asset-pipeline/02-DISCUSSION-LOG.md b/.planning/phases/02-gemini-integration-asset-pipeline/02-DISCUSSION-LOG.md new file mode 100644 index 0000000..a6d1e89 --- /dev/null +++ b/.planning/phases/02-gemini-integration-asset-pipeline/02-DISCUSSION-LOG.md @@ -0,0 +1,40 @@ +# Phase 2: Gemini-Integration + Asset-Pipeline - Discussion Log (Assumptions Mode) + +> **Audit trail only.** Do not use as input to planning, research, or execution agents. +> Decisions captured in CONTEXT.md — this log preserves the analysis. + +**Date:** 2026-03-29 +**Phase:** 02-Gemini-Integration + Asset-Pipeline +**Mode:** assumptions +**Areas analyzed:** Build Script Runtime, IndexedDB Schema Extension, Welcome Screen Avatar Replacement, Gemini API Client Architecture, Fallback System, Companion Text Integration + +## Assumptions Presented + +### Build Script Runtime +| Assumption | Confidence | Evidence | +|------------|-----------|----------| +| Node.js via npx tsx, isomorphic API module | Confident | SPEC.md line 392, package.json has no tsx yet | + +### IndexedDB Schema Extension +| Assumption | Confidence | Evidence | +|------------|-----------|----------| +| Keep DB version 1, API counts in settings store | Likely | db.ts version 1 with 5 stores, simple counting suffices | + +### Welcome Screen Avatar Replacement +| Assumption | Confidence | Evidence | +|------------|-----------|----------| +| Static imports in characters.ts, avatarUrl field | Likely | src/assets/companions/ dirs exist, CompanionDefinition has emoji | + +### Gemini Image API Format +| Assumption | Confidence | Evidence | +|------------|-----------|----------| +| Base64 inlineData, response_modalities: IMAGE | Likely | SPEC.md lines 636-646, types.ts Blob storage | + +## Corrections Made + +No corrections — all assumptions confirmed. + +## External Research Flagged + +- Gemini 3.1 Flash Image Preview API exact format with character consistency +- tsx devDependency version for build script