Phase 2: Gemini-Integration + Asset-Pipeline - Context

Gathered: 2026-03-29 (assumptions mode) Status: Ready for planning

## Phase Boundary

Gemini-API funktioniert für Text und Bild, Character Sheets sind generiert und eingebunden. Umfasst: Config-Loader, Text-API-Client mit Retry, Bild-API-Client mit Character Consistency, Fallback-Texte und -Bilder, Rate-Limiting, Build-Script für Asset-Generierung, generierte Avatare im Welcome-Screen, Begleitfigur-Texte (Begrüssung, Buchstaben-Vorstellung, Wald-Kommentar). Kein Wald-Wachstum (Phase 3), keine Übungsphasen-Verbindung (Phase 3), kein Audio (Phase 4).

## Implementation Decisions

Build Script Runtime

D-01: scripts/generate-assets.ts runs in Node.js via npx tsx (new devDependency). Reads public/config.json via fs.readFileSync. The browser API client uses fetch('/config.json'). A shared isomorphic Gemini API module using only fetch() serves both contexts — only the config-loading layer differs.
D-02: Build script generates 4 Character Sheets + 4 Avatars + 1 Style Reference, saves as PNG to src/assets/companions/{type}/ and src/assets/style-reference/. These are checked into git.

IndexedDB Schema Extension

D-03: Keep DB at version 1. API call counts stored as fields in the existing settings store (e.g. apiCallsToday: number, lastApiCallDate: string). No dedicated apiLog store — simple counting suffices for rate-limiting (GAPI-06). Detailed per-call logging is not required.
D-04: Add read/write helper methods for companionAssets and styleReference stores in db.ts. These stores exist structurally from Phase 1 but lack accessor functions.

Welcome Screen Avatar Replacement

D-05: Static imports in characters.ts for each avatar file. Each CompanionDefinition gets an avatarUrl: string field with the Vite-resolved URL. Welcome screen renders <img> instead of emoji spans. Type-safe, explicit mapping.
D-06: Fallback to emoji if avatar image fails to load (onerror handler on <img>).

Gemini API Client Architecture

D-07: Isomorphic API module src/api/gemini.ts with generateText(prompt, systemPrompt) and generateImage(prompt, referenceImages[]). Uses only fetch() — works in both Node.js 18+ and browser.
D-08: Reference images sent as Base64-encoded inlineData (standard Gemini REST pattern). response_modalities: ["IMAGE"] for image generation. Verify exact format against current Gemini docs at implementation time.
D-09: Retry logic: 3 attempts with exponential backoff (1s, 2s, 4s). On final failure, return null and let caller use fallback.
D-10: Rate-limiting: Max 1 image + 1 text per exercise unit. Check counts in settings store before API call. Reset daily.

Fallback System

D-11: Fallback texts stored as static arrays in src/companion/fallbacks.ts: 10 greetings, 10 forest comments, letter introductions for levels 1-6. All in simple German, max 25 words, no performance praise.
D-12: Fallback images: 5-10 simple SVG illustrations in src/assets/fallback/ showing generic forest elements (flower, mushroom, bird, tree, butterfly). Used when API unavailable or rate-limited.

Companion Text Integration

D-13: Companion text module src/companion/companion.ts with getGreeting(character, timeOfDay), getLetterIntro(character, letter), getForestComment(character). Each tries API first, falls back to static texts.
D-14: Companion area shown on welcome/forest screens with greeting text. Companion does NOT appear during typing exercise in Phase 2 (Phase 3 adds "Entdecken" phase with companion letter introduction).

Claude's Discretion

Exact prompt templates for Gemini API calls (following SPEC.md section 9.2 as guide)
Build script CLI output format and error handling
Asset file naming conventions within companion directories
Config loader error handling strategy

<canonical_refs>

Canonical References

Downstream agents MUST read these before planning or implementing.

SPEC.md — Full specification, THE authoritative source
SPEC.md section 5.4 — Visual consistency architecture (Character Sheets, Style Reference)
SPEC.md section 6.4 — API key handling (config.json structure)
SPEC.md section 9 — Gemini Integration (API endpoints, prompt templates, models)
SPEC.md section 9.2 — Prompt templates for all companion text types
SPEC.md section 9.3 — Character Sheet generation prompts and visual descriptions
src/types.ts — Existing TypeScript interfaces (CompanionAssets, ForestElement, etc.)
src/storage/db.ts — IndexedDB wrapper with 5 stores (companionAssets, styleReference need methods)
src/companion/characters.ts — Companion definitions (needs avatarUrl field)
src/ui/screens.ts — Welcome screen rendering (emoji → img replacement)
public/config.json — Gemini API key and model names </canonical_refs>

<code_context>

Existing Code Insights

Reusable Assets

src/types.ts: CompanionAssets interface (characterSheet: Blob, avatarImage: Blob, characterType, generatedAt), ForestElement interface, StyleReference interface already defined
src/storage/db.ts: 5 IndexedDB stores created, progress and settings have CRUD methods, other 3 stores need method additions
src/companion/characters.ts: 4 CompanionDefinition objects with name, type, emoji, personality — ready to extend with avatarUrl
public/config.json: Already contains geminiApiKey, geminiModel, imageModel

Established Patterns

All modules use TypeScript strict mode
Functions export individually (no default exports)
IndexedDB operations are Promise-wrapped with typed results
CSS uses BEM-style class names with -- modifiers

Integration Points

Welcome screen initWelcomeScreen() in screens.ts — replace emoji cards with img cards
app.ts initApp() — add greeting text display after detecting returning user
db.ts — add saveCompanionAssets(), getCompanionAssets(), saveStyleReference(), getStyleReference(), saveForestElement(), getForestElement() methods
characters.ts — add avatarUrl to each companion definition </code_context>

## Specific Ideas

No specific requirements — open to standard approaches

## Deferred Ideas

None — analysis stayed within phase scope

6.4 KiB Raw Permalink Blame History