diff --git a/.planning/phases/03-pipeline-integration-and-mvp/03-CONTEXT.md b/.planning/phases/03-pipeline-integration-and-mvp/03-CONTEXT.md new file mode 100644 index 0000000..6b4d1fe --- /dev/null +++ b/.planning/phases/03-pipeline-integration-and-mvp/03-CONTEXT.md @@ -0,0 +1,114 @@ +# Phase 3: Pipeline Integration and MVP - Context + +**Gathered:** 2026-03-26 +**Status:** Ready for planning + + +## Phase Boundary + +Wire the capture-classify-aggregate pipeline into the audio synthesis and MP3 encoding engine. Deliver the complete v1 MVP: `netsynth -i eth0 -o out.mp3` → capture → Ctrl+C → valid MP3. Replace the single "unknown" traffic class with 4 hash-bucketed sub-clusters, each producing a distinct dissonant drone tone. + + + + +## Implementation Decisions + +### Auto-clustering +- **D-01:** Hash-bucketing approach — deterministic hash of (dst_port, protocol) into 4 fixed buckets. No k-means, no warmup period, no external dependency. +- **D-02:** 4 buckets for unknown traffic. Total layer count becomes 14 (10 known + 4 unknown buckets). +- **D-03:** Hash-bucketing happens inside the existing classifier — classifier returns `unknown-1` through `unknown-4` instead of plain `unknown`. New TrafficClass constants added. +- **D-04:** Replace `ClassUnknown` entirely with `ClassUnknown1`-`ClassUnknown4`. `AllClasses()` returns 14 classes. Clean break, no fallback. + +### Cluster tone assignment +- **D-05:** All 4 unknown buckets live in a dedicated 850-1100 Hz dissonant range, above the known protocol range (60-800 Hz). Slightly detuned intervals between them. +- **D-06:** Same dissonant harmonic character for all 4 buckets (matching Phase 2's D-04/D-06 for the original "unknown" tone). Differentiated by pitch only, preserving the "something foreign" sonic identity. + +### Shutdown & encoding feedback +- **D-07:** Status line during encoding: print `Encoding N windows to ...` then `Saved (Xs, N KB, encoded in Xs)` on completion. Minimal but confirms encoding is happening. +- **D-08:** Protocol summary prints BEFORE encoding. Ctrl+C → protocol summary → "Encoding..." → "Saved". User sees capture stats immediately. + +### Post-run output +- **D-09:** Saved line includes path, audio duration, file size, and encoding time. e.g., `Saved out.mp3 (12.5s, 198 KB, encoded in 0.3s)`. +- **D-10:** Unknown buckets appear in the protocol summary naturally as `unknown-1: 42, unknown-2: 17`, etc. Falls out of existing summary logic since they're full TrafficClass values. + +### Claude's Discretion +- Exact hash function for port/protocol → bucket mapping (within the 4-bucket constraint) +- Exact Hz values for the 4 unknown bucket tones (within 850-1100 Hz range, detuned intervals) +- GainPerLayer recalculation for 14 layers (was 1/11 for 11 layers) +- Stereo panning positions for the 4 unknown buckets +- Encoding time measurement implementation + + + + +## Canonical References + +**Downstream agents MUST read these before planning or implementing.** + +### Project context +- `.planning/PROJECT.md` — Core value, constraints (Go, MP3 output, non-interactive) +- `.planning/REQUIREMENTS.md` — CAPT-03 (graceful flush), CLAS-02 (auto-clustering) + +### Prior phase context +- `.planning/phases/01-capture-and-classification/01-CONTEXT.md` — Phase 1 decisions; classifier design, pipeline architecture +- `.planning/phases/02-audio-synthesis-engine/02-CONTEXT.md` — Phase 2 decisions; tone mapping, mixing, encoding + +### Key source files (integration points) +- `cmd/netsynth/main.go` — CLI wiring with TODO(phase-3) at line 99; accumulator loop at lines 92-97 +- `encode/mp3.go` — `RunSynthesis()` and `EncodeMP3()` — the synthesis entry point +- `synth/bank.go` — `OscillatorBank` and `RenderWindow()` — renders audio from WindowSnapshots +- `synth/config.go` — `ClassFreqConfigs` map, `NumLayers`, `GainPerLayer` constants +- `classify/types.go` — `TrafficClass` constants, `AllClasses()`, `WindowSnapshot` struct +- `classify/rules.go` — Classification rules (first-match-wins ordered slice) +- `classify/classifier.go` — `Classify()` method where hash-bucketing will be added +- `aggregate/summary.go` — `PrintSummary()` and `PrintWindowLine()` for exit output + +### Stack decisions +- `CLAUDE.md` — Technology stack section; `muesli/kmeans` listed but NOT needed (hash-bucketing chosen instead) + + + + +## Existing Code Insights + +### Reusable Assets +- `encode.RunSynthesis([]WindowSnapshot, string) error` — already takes snapshots and produces MP3; main.go just needs to call it +- `synth.NewBank(tau)` / `bank.RenderWindow(snap)` — synthesis engine already works with WindowSnapshot +- `classify.NewClassifier(rules)` / `classifier.Classify(pkt)` — classifier where hash-bucketing logic will be added +- `aggregate.PrintSummary(io.Writer, map[TrafficClass]int64)` — exit summary; will naturally show unknown-1 through unknown-4 + +### Established Patterns +- Channel-based pipeline: capture → classify → aggregate → (synthesis goes here) +- `done <-chan struct{}` for shutdown signaling via `signal.NotifyContext` +- Buffered channels for stage decoupling (512-buffer capture, 1024-buffer classified, 8-buffer aggregate) +- `io.Writer` injection for testable output (aggregate/summary.go) +- Config-driven classifier with ordered `[]Rule` slice, first-match-wins + +### Integration Points +- `main.go:99` — Replace `_ = collectedSnapshots` with `encode.RunSynthesis(collectedSnapshots, outputPath)` call +- `main.go:92-97` — Snapshot accumulator loop already collects snapshots; wiring is mechanical +- `synth/config.go` — `ClassFreqConfigs` map needs 4 new entries for unknown buckets; `NumLayers` and `GainPerLayer` need updating from 11 to 14 +- `classify/types.go` — Remove `ClassUnknown`, add `ClassUnknown1`-`ClassUnknown4`, update `AllClasses()` + + + + +## Specific Ideas + +- The end-to-end flow should feel like tcpdump: start capturing, see activity, Ctrl+C, get your results +- Unknown bucket tones should be immediately recognizable as "something foreign" — the dissonant character is the sonic signal, pitch differentiates which bucket +- The protocol summary + saved line should give the user everything they need at a glance without running ffprobe + + + + +## Deferred Ideas + +None — discussion stayed within phase scope + + + +--- + +*Phase: 03-pipeline-integration-and-mvp* +*Context gathered: 2026-03-26* diff --git a/.planning/phases/03-pipeline-integration-and-mvp/03-DISCUSSION-LOG.md b/.planning/phases/03-pipeline-integration-and-mvp/03-DISCUSSION-LOG.md new file mode 100644 index 0000000..5f9090e --- /dev/null +++ b/.planning/phases/03-pipeline-integration-and-mvp/03-DISCUSSION-LOG.md @@ -0,0 +1,148 @@ +# Phase 3: Pipeline Integration and MVP - Discussion Log + +> **Audit trail only.** Do not use as input to planning, research, or execution agents. +> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered. + +**Date:** 2026-03-26 +**Phase:** 03-pipeline-integration-and-mvp +**Areas discussed:** Auto-clustering strategy, Cluster tone assignment, Shutdown & encoding feedback, Post-run output + +--- + +## Auto-clustering Strategy + +| Option | Description | Selected | +|--------|-------------|----------| +| Hash-bucketing | Deterministic: hash (dst_port, protocol) into N fixed buckets. Same traffic always maps to same cluster/tone. Simple, no warmup, predictable. | ✓ | +| K-means clustering | Statistical: feature vectors into muesli/kmeans. Discovers natural groupings. Needs warmup, non-deterministic, adds dependency. | | +| You decide | Claude picks based on codebase fit and simplicity. | | + +**User's choice:** Hash-bucketing +**Notes:** None + +### Bucket count + +| Option | Description | Selected | +|--------|-------------|----------| +| 4 buckets | Enough variety without overwhelming stereo field. Total 14 layers (10 known + 4 unknown). | ✓ | +| 8 buckets | More granularity. Total 18 layers — thinner per-layer gain. | | +| You decide | Claude picks balancing variety vs mixing headroom. | | + +**User's choice:** 4 buckets +**Notes:** None + +### Placement + +| Option | Description | Selected | +|--------|-------------|----------| +| Inside classifier | Classifier returns unknown-1 through unknown-4. Clean: final class in one place. | ✓ | +| Post-classification step | Separate stage re-maps unknowns. Keeps classifier untouched but adds complexity. | | +| You decide | Claude picks based on minimal code change. | | + +**User's choice:** Inside classifier +**Notes:** None + +--- + +## Cluster Tone Assignment + +### Frequency approach + +| Option | Description | Selected | +|--------|-------------|----------| +| Dissonant range | All 4 buckets in 850-1100 Hz range above known protocols. Detuned intervals. Extends Phase 2 D-04. | ✓ | +| Hash-to-frequency | Hash bucket ID to frequency. Deterministic but could collide with known tones. | | +| You decide | Claude picks frequencies fitting existing harmonic scheme. | | + +**User's choice:** Dissonant range (850-1100 Hz) +**Notes:** None + +### Timbre + +| Option | Description | Selected | +|--------|-------------|----------| +| Same dissonant character | All 4 use same detuned/beating harmonics as original unknown. Differentiated by pitch only. | ✓ | +| Varied timbres per bucket | Each gets own harmonic ratio set. More variety but harder to recognize as unknown family. | | +| You decide | Claude picks based on coherence. | | + +**User's choice:** Same dissonant character +**Notes:** None + +### Migration + +| Option | Description | Selected | +|--------|-------------|----------| +| Replace entirely | Remove ClassUnknown, add ClassUnknown1-4. AllClasses() returns 14. Clean break. | ✓ | +| Keep as fallback | ClassUnknown remains for edge cases. Buckets are ClassUnknown1-4 for hashable unknowns. | | +| You decide | Claude picks based on realistic edge cases. | | + +**User's choice:** Replace entirely +**Notes:** None + +--- + +## Shutdown & Encoding Feedback + +### Feedback level + +| Option | Description | Selected | +|--------|-------------|----------| +| Status line | Print "Encoding N windows..." then "Saved path (Xs, N KB, encoded in Xs)". Minimal but confirms activity. | ✓ | +| Silent encoding | No output during encoding. File appearing on disk is enough. | | +| Progress bar | Show encoding progress. Adds dependency or custom logic for likely <1s operation. | | +| You decide | Claude picks appropriate level. | | + +**User's choice:** Status line +**Notes:** None + +### Output order + +| Option | Description | Selected | +|--------|-------------|----------| +| Before encoding | Ctrl+C → summary → Encoding → Saved. Summary appears instantly. | ✓ | +| After encoding | Ctrl+C → Encoding → Saved → summary. File ready sooner. | | +| You decide | Claude picks best feel. | | + +**User's choice:** Before encoding (summary first) +**Notes:** None + +--- + +## Post-run Output + +### Saved line verbosity + +| Option | Description | Selected | +|--------|-------------|----------| +| Path + duration + size | e.g., "Saved out.mp3 (12.5s, 198 KB, encoded in 0.3s)". Everything at a glance. | ✓ | +| Path only | e.g., "Saved out.mp3". Minimal. | | +| Path + duration | e.g., "Saved out.mp3 (12.5s audio)". Middle ground. | | +| You decide | Claude picks verbosity level. | | + +**User's choice:** Path + duration + size +**Notes:** None + +### Cluster info in summary + +| Option | Description | Selected | +|--------|-------------|----------| +| Yes, in protocol summary | Unknown buckets appear naturally as unknown-1: 42, unknown-2: 17 etc. Falls out of existing logic. | ✓ | +| No, keep summary as-is | Only show known protocol classes. Buckets are implementation details. | | +| You decide | Claude picks based on usefulness. | | + +**User's choice:** Yes, in protocol summary +**Notes:** None + +--- + +## Claude's Discretion + +- Exact hash function for port/protocol → bucket mapping +- Exact Hz values for 4 unknown bucket tones (within 850-1100 Hz) +- GainPerLayer recalculation for 14 layers +- Stereo panning positions for unknown buckets +- Encoding time measurement implementation + +## Deferred Ideas + +None — discussion stayed within phase scope