docs(03): research phase domain for pipeline integration and MVP
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,457 @@
|
|||||||
|
# Phase 3: Pipeline Integration and MVP - Research
|
||||||
|
|
||||||
|
**Researched:** 2026-03-26
|
||||||
|
**Domain:** Go pipeline wiring, hash-bucketing classifier extension, audio configuration update, encoding feedback UX
|
||||||
|
**Confidence:** HIGH
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Phase 3 is almost entirely an integration and light-extension phase. Phases 1 and 2 produced a fully functional capture-classify-aggregate pipeline AND a fully functional audio synthesis/MP3 encoder. The only reason they are not yet connected is a single `TODO(phase-3)` stub in `main.go` at line 99. The mechanical wiring is one line: replace `_ = collectedSnapshots` with `encode.RunSynthesis(collectedSnapshots, outputPath)`. The rest of the phase extends the `TrafficClass` type system from 11 classes to 14 (replacing `ClassUnknown` with `ClassUnknown1`-`ClassUnknown4`), updates `synth/config.go` to add tone configs for the 4 new classes and recalculate `GainPerLayer`, and adds encoding-progress feedback to the UX flow in `main.go`.
|
||||||
|
|
||||||
|
The project's existing patterns — `io.Writer` injection, `AllClasses()` driving loops in `bank.go`, config-driven `ClassFreqConfigs` map — mean that all synthesis and summary code automatically picks up the new classes with zero changes beyond updating the type definitions and config map. Hash-bucketing is a deterministic `(dstPort*31 + protocolNum) % 4` style operation inserted at the end of `classifier.Classify()` as a tail clause replacing the bare `ClassUnknown` return.
|
||||||
|
|
||||||
|
**Primary recommendation:** Execute in four focused, independently testable tasks: (1) extend `classify/types.go`; (2) add hash-bucketing to `classifier.Classify()`; (3) update `synth/config.go` for 14 layers; (4) wire `main.go` and add encoding feedback. Tests for tasks 1-3 follow existing package test patterns. Task 4 confirms end-to-end correctness by building and running with a loopback interface.
|
||||||
|
|
||||||
|
<user_constraints>
|
||||||
|
## User Constraints (from CONTEXT.md)
|
||||||
|
|
||||||
|
### Locked Decisions
|
||||||
|
|
||||||
|
- **D-01:** Hash-bucketing approach — deterministic hash of (dst_port, protocol) into 4 fixed buckets. No k-means, no warmup period, no external dependency.
|
||||||
|
- **D-02:** 4 buckets for unknown traffic. Total layer count becomes 14 (10 known + 4 unknown buckets).
|
||||||
|
- **D-03:** Hash-bucketing happens inside the existing classifier — classifier returns `unknown-1` through `unknown-4` instead of plain `unknown`. New TrafficClass constants added.
|
||||||
|
- **D-04:** Replace `ClassUnknown` entirely with `ClassUnknown1`-`ClassUnknown4`. `AllClasses()` returns 14 classes. Clean break, no fallback.
|
||||||
|
- **D-05:** All 4 unknown buckets live in a dedicated 850-1100 Hz dissonant range, above the known protocol range (60-800 Hz). Slightly detuned intervals between them.
|
||||||
|
- **D-06:** Same dissonant harmonic character for all 4 buckets (matching Phase 2's D-04/D-06 for the original "unknown" tone). Differentiated by pitch only, preserving the "something foreign" sonic identity.
|
||||||
|
- **D-07:** Status line during encoding: print `Encoding N windows to <path>...` then `Saved <path> (Xs, N KB, encoded in Xs)` on completion. Minimal but confirms encoding is happening.
|
||||||
|
- **D-08:** Protocol summary prints BEFORE encoding. Ctrl+C → protocol summary → "Encoding..." → "Saved". User sees capture stats immediately.
|
||||||
|
- **D-09:** Saved line includes path, audio duration, file size, and encoding time. e.g., `Saved out.mp3 (12.5s, 198 KB, encoded in 0.3s)`.
|
||||||
|
- **D-10:** Unknown buckets appear in the protocol summary naturally as `unknown-1: 42, unknown-2: 17`, etc. Falls out of existing summary logic since they're full TrafficClass values.
|
||||||
|
|
||||||
|
### Claude's Discretion
|
||||||
|
|
||||||
|
- Exact hash function for port/protocol → bucket mapping (within the 4-bucket constraint)
|
||||||
|
- Exact Hz values for the 4 unknown bucket tones (within 850-1100 Hz range, detuned intervals)
|
||||||
|
- GainPerLayer recalculation for 14 layers (was 1/11 for 11 layers)
|
||||||
|
- Stereo panning positions for the 4 unknown buckets
|
||||||
|
- Encoding time measurement implementation
|
||||||
|
|
||||||
|
### Deferred Ideas (OUT OF SCOPE)
|
||||||
|
|
||||||
|
None — discussion stayed within phase scope
|
||||||
|
</user_constraints>
|
||||||
|
|
||||||
|
<phase_requirements>
|
||||||
|
## Phase Requirements
|
||||||
|
|
||||||
|
| ID | Description | Research Support |
|
||||||
|
|----|-------------|------------------|
|
||||||
|
| CAPT-03 | Capture runs until user presses Ctrl+C, then gracefully flushes and saves MP3 | The existing `signal.NotifyContext` shutdown path already drains all channels before the accumulator loop exits; `RunSynthesis` simply needs to be called with the collected snapshots and the output path. D-07/D-08/D-09 add the feedback messages around the call. |
|
||||||
|
| CLAS-02 | Unrecognized traffic is auto-clustered and assigned unique tones automatically | Hash-bucketing in `classifier.Classify()` tail clause maps `(dstPort, protocolNum)` deterministically to `ClassUnknown1`-`ClassUnknown4`. New constants in `classify/types.go`, new `FreqConfig` entries in `synth/config.go` covering 850-1100 Hz. |
|
||||||
|
</phase_requirements>
|
||||||
|
|
||||||
|
## Standard Stack
|
||||||
|
|
||||||
|
### Core (no changes — inherited from Phases 1 and 2)
|
||||||
|
|
||||||
|
| Library | Version | Purpose |
|
||||||
|
|---------|---------|---------|
|
||||||
|
| `github.com/gopacket/gopacket` | v1.5.0 | Packet capture/decoding — already in go.mod |
|
||||||
|
| `github.com/packetcap/go-pcap` | v0.0.0-20251215 | Live capture backend — already in go.mod |
|
||||||
|
| `github.com/sjzar/go-lame` | v0.0.9 | MP3 encoding — already in go.mod |
|
||||||
|
| `github.com/spf13/cobra` | v1.10.2 | CLI — already in go.mod |
|
||||||
|
|
||||||
|
No new dependencies required for Phase 3.
|
||||||
|
|
||||||
|
**Installation:** None — all dependencies are already in go.mod and go.sum.
|
||||||
|
|
||||||
|
## Architecture Patterns
|
||||||
|
|
||||||
|
### Existing Project Structure (Phase 3 touches these files)
|
||||||
|
|
||||||
|
```
|
||||||
|
classify/
|
||||||
|
├── types.go # ADD ClassUnknown1-4, REMOVE ClassUnknown, UPDATE AllClasses()
|
||||||
|
├── classifier.go # ADD hash-bucketing tail clause in Classify()
|
||||||
|
├── rules.go # No change
|
||||||
|
└── classifier_test.go # UPDATE: test unknown bucket routing
|
||||||
|
synth/
|
||||||
|
├── config.go # ADD 4 FreqConfig entries, UPDATE NumLayers=14, GainPerLayer=1/14
|
||||||
|
├── bank.go # No change (loops over AllClasses() — picks up new classes automatically)
|
||||||
|
└── bank_test.go # UPDATE: test 14-layer count assertion
|
||||||
|
cmd/netsynth/
|
||||||
|
└── main.go # WIRE RunSynthesis, ADD encoding feedback messages (D-07/D-08/D-09)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pattern 1: TrafficClass Extension
|
||||||
|
|
||||||
|
**What:** Add constants to `classify/types.go`, update `AllClasses()`, add config entries to `synth/config.go`. All synthesis and summary loops that iterate `AllClasses()` automatically pick up the 4 new classes.
|
||||||
|
|
||||||
|
**How it works end-to-end:**
|
||||||
|
- `NewBank(tau)` in `bank.go` calls `classify.AllClasses()` to populate the `layers` map
|
||||||
|
- `RenderWindow()` loops `classify.AllClasses()` for both amplitude updates and frame mixing
|
||||||
|
- `PrintSummary()` in `aggregate/summary.go` sorts and prints whatever keys are in `totals` — it is map-driven, not `AllClasses()`-driven, so no change needed there
|
||||||
|
|
||||||
|
**Example — types.go change:**
|
||||||
|
```go
|
||||||
|
const (
|
||||||
|
ClassICMP TrafficClass = "ICMP"
|
||||||
|
ClassDNS TrafficClass = "DNS"
|
||||||
|
ClassHTTPS TrafficClass = "HTTPS"
|
||||||
|
ClassHTTP TrafficClass = "HTTP"
|
||||||
|
ClassSSH TrafficClass = "SSH"
|
||||||
|
ClassSMTP TrafficClass = "SMTP"
|
||||||
|
ClassNTP TrafficClass = "NTP"
|
||||||
|
ClassDHCP TrafficClass = "DHCP"
|
||||||
|
ClassOtherTCP TrafficClass = "other-TCP"
|
||||||
|
ClassOtherUDP TrafficClass = "other-UDP"
|
||||||
|
// ClassUnknown REMOVED — replaced by 4 hash buckets (D-04)
|
||||||
|
ClassUnknown1 TrafficClass = "unknown-1"
|
||||||
|
ClassUnknown2 TrafficClass = "unknown-2"
|
||||||
|
ClassUnknown3 TrafficClass = "unknown-3"
|
||||||
|
ClassUnknown4 TrafficClass = "unknown-4"
|
||||||
|
)
|
||||||
|
|
||||||
|
func AllClasses() []TrafficClass {
|
||||||
|
return []TrafficClass{
|
||||||
|
ClassICMP, ClassDNS, ClassHTTPS, ClassHTTP, ClassSSH,
|
||||||
|
ClassSMTP, ClassNTP, ClassDHCP, ClassOtherTCP, ClassOtherUDP,
|
||||||
|
ClassUnknown1, ClassUnknown2, ClassUnknown3, ClassUnknown4,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pattern 2: Hash-Bucketing Tail Clause
|
||||||
|
|
||||||
|
**What:** The existing `Classify()` method returns `ClassUnknown` as a bare fallback in three places: after the ICMP rule loop, at the end of the TCP block (no matching port rule), at the end of the UDP block (no matching port rule), and in the final fall-through (no recognized transport layer). Each of these becomes a `hashBucket(result.DstPort, result.Protocol)` call.
|
||||||
|
|
||||||
|
**Hash function design (Claude's Discretion):**
|
||||||
|
|
||||||
|
A deterministic hash of `(dstPort, protocolNum)` into 4 buckets:
|
||||||
|
|
||||||
|
```go
|
||||||
|
// hashBucket maps an unrecognized packet to one of 4 unknown classes.
|
||||||
|
// Uses a simple polynomial hash to spread common port ranges across buckets.
|
||||||
|
func hashBucket(dstPort uint16, protocol string) TrafficClass {
|
||||||
|
var protoNum uint16
|
||||||
|
switch protocol {
|
||||||
|
case "tcp":
|
||||||
|
protoNum = 6
|
||||||
|
case "udp":
|
||||||
|
protoNum = 17
|
||||||
|
case "icmp":
|
||||||
|
protoNum = 1
|
||||||
|
default:
|
||||||
|
protoNum = 0
|
||||||
|
}
|
||||||
|
// Polynomial mix — prime multipliers prevent trivially adjacent ports
|
||||||
|
// landing in the same bucket.
|
||||||
|
h := uint32(dstPort)*31 + uint32(protoNum)*7
|
||||||
|
switch h % 4 {
|
||||||
|
case 0:
|
||||||
|
return ClassUnknown1
|
||||||
|
case 1:
|
||||||
|
return ClassUnknown2
|
||||||
|
case 2:
|
||||||
|
return ClassUnknown3
|
||||||
|
default:
|
||||||
|
return ClassUnknown4
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why this is correct:** ARP packets (no transport layer) have `dstPort=0, protocol=""` → bucket 0 → `ClassUnknown1`. This is fine; the dissonant tone still distinguishes them from known traffic. Determinism means the same port always lands in the same bucket across runs, which is desirable: if a user always sees a custom service on port 8443, it always gets the same tone.
|
||||||
|
|
||||||
|
**Where to apply:** Replace the 4 bare `ClassUnknown` return points in `classifier.go` with `hashBucket(result.DstPort, result.Protocol)`.
|
||||||
|
|
||||||
|
### Pattern 3: synth/config.go Update
|
||||||
|
|
||||||
|
**What:** Remove the `ClassUnknown` entry, add 4 new entries in the 850-1100 Hz dissonant range. Update `NumLayers` from 11 to 14 and `GainPerLayer` from `1.0/11` to `1.0/14`.
|
||||||
|
|
||||||
|
**Recommended Hz values (Claude's Discretion — within D-05 850-1100 Hz constraint):**
|
||||||
|
|
||||||
|
The original `ClassUnknown` was at 437 Hz (deliberately flat from SMTP's 440 Hz). The 4 new buckets should be spread across 850-1100 Hz with slightly detuned intervals to create subtle beating when multiple unknown buckets are active simultaneously. Suggested values:
|
||||||
|
|
||||||
|
| Class | Hz | Rationale |
|
||||||
|
|-------|-------|-----------|
|
||||||
|
| ClassUnknown1 | 862 | ~11 cents flat from 900 Hz |
|
||||||
|
| ClassUnknown2 | 920 | ~3 Hz above a "clean" interval from Unknown1 |
|
||||||
|
| ClassUnknown3 | 981 | ~7 cents flat from 990 Hz |
|
||||||
|
| ClassUnknown4 | 1047 | ~9 cents sharp from 1040 Hz |
|
||||||
|
|
||||||
|
Harmonics follow Phase 2's D-06 dissonant profile: `{1, 1.0}, {2, 0.8}, {3, 0.4}` (rich upper harmonics = "foreign" character).
|
||||||
|
|
||||||
|
**Recommended pan positions (Claude's Discretion):**
|
||||||
|
- ClassUnknown1: `0.6` (right-of-center)
|
||||||
|
- ClassUnknown2: `-0.6` (left-of-center)
|
||||||
|
- ClassUnknown3: `0.9` (wide right)
|
||||||
|
- ClassUnknown4: `-0.9` (wide left)
|
||||||
|
|
||||||
|
Spreading unknowns wide in the stereo field sonically separates them from the known protocols, which occupy center and moderate positions.
|
||||||
|
|
||||||
|
**GainPerLayer recalculation:** `1.0 / 14 ≈ 0.0714`. The existing `TestMixerNoClip` test exercises all-layers-at-max — it asserts `[-1, 1]` bounds. With 14 layers at `GainPerLayer=1/14` and `PanGains()` applying constant-power panning, worst-case is `14 * (1/14) * maxPanGain = 1.0 * maxPanGain`. The `PanGains` function returns values where `gainL^2 + gainR^2 = 1`, so `gainL, gainR <= 1.0`. Clipping cannot occur. The test will pass with the updated constant.
|
||||||
|
|
||||||
|
### Pattern 4: main.go Wiring and Feedback
|
||||||
|
|
||||||
|
**What:** Replace the TODO stub at line 99 with:
|
||||||
|
1. Move `PrintSummary` call BEFORE `RunSynthesis` (D-08: summary first)
|
||||||
|
2. Print encoding status line before calling `RunSynthesis` (D-07)
|
||||||
|
3. Measure encoding time with `time.Now()` (Claude's Discretion)
|
||||||
|
4. Call `encode.RunSynthesis(collectedSnapshots, outputPath)`
|
||||||
|
5. Compute audio duration from snapshot count: `float64(len(collectedSnapshots)) * float64(aggregate.DefaultWindowMs) / 1000.0`
|
||||||
|
6. Stat the output file for size
|
||||||
|
7. Print the "Saved" line (D-09)
|
||||||
|
|
||||||
|
**Import addition:** `"github.com/netsynth/netsynth/encode"` — the only new import in main.go.
|
||||||
|
|
||||||
|
**Example wiring sequence:**
|
||||||
|
```go
|
||||||
|
// Print exit summary FIRST (D-08)
|
||||||
|
dropped := atomic.LoadInt64(droppedPtr)
|
||||||
|
if dropped > 0 {
|
||||||
|
fmt.Fprintf(os.Stderr, "\nWarning: %d packets dropped (channel buffer full)\n", dropped)
|
||||||
|
}
|
||||||
|
aggregate.PrintSummary(os.Stderr, totals)
|
||||||
|
|
||||||
|
// Encode to MP3 with feedback (D-07/D-09/CAPT-03)
|
||||||
|
fmt.Fprintf(os.Stderr, "Encoding %d windows to %s...\n", len(collectedSnapshots), outputPath)
|
||||||
|
encodeStart := time.Now()
|
||||||
|
if err := encode.RunSynthesis(collectedSnapshots, outputPath); err != nil {
|
||||||
|
return fmt.Errorf("synthesis failed: %w", err)
|
||||||
|
}
|
||||||
|
encodeElapsed := time.Since(encodeStart)
|
||||||
|
audioDuration := float64(len(collectedSnapshots)) * float64(aggregate.DefaultWindowMs) / 1000.0
|
||||||
|
info, err := os.Stat(outputPath)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("stat output: %w", err)
|
||||||
|
}
|
||||||
|
fmt.Fprintf(os.Stderr, "Saved %s (%.1fs, %d KB, encoded in %.1fs)\n",
|
||||||
|
outputPath,
|
||||||
|
audioDuration,
|
||||||
|
info.Size()/1024,
|
||||||
|
encodeElapsed.Seconds(),
|
||||||
|
)
|
||||||
|
return nil
|
||||||
|
```
|
||||||
|
|
||||||
|
Note: The existing `return nil` at line 109 is replaced by this block.
|
||||||
|
|
||||||
|
### Anti-Patterns to Avoid
|
||||||
|
|
||||||
|
- **Keeping `ClassUnknown` as a fallback:** D-04 mandates clean removal. Any remaining `ClassUnknown` reference after Phase 3 is a bug — the compiler will catch it in synth/config.go if the map entry is not updated, but it won't catch it if a default return in classifier.go still uses it.
|
||||||
|
- **Forgetting to update `bank_test.go`:** `TestNewBankHas11Layers` hardcodes `11` — it must be updated to `14` or the test suite will fail after the `AllClasses()` change.
|
||||||
|
- **Calling `PrintSummary` after `RunSynthesis`:** D-08 requires summary first. The user may wait several seconds for encoding; seeing the protocol counts immediately is the UX goal.
|
||||||
|
- **Creating the output file before the zero-packet check:** `RunSynthesis` already guards this (Phase 2 OUT-03). The main.go wiring doesn't need a redundant check — just handle the returned error.
|
||||||
|
- **Hardcoding audio duration from time elapsed:** Audio duration is deterministic from snapshot count × window duration, not from wall clock. Use `len(collectedSnapshots) * DefaultWindowMs / 1000` seconds.
|
||||||
|
|
||||||
|
## Don't Hand-Roll
|
||||||
|
|
||||||
|
| Problem | Don't Build | Use Instead |
|
||||||
|
|---------|-------------|-------------|
|
||||||
|
| MP3 encoding | Custom MP3 writer | `encode.RunSynthesis()` — already exists and tested |
|
||||||
|
| Synthesis | Custom oscillator | `synth.NewBank()`/`bank.RenderWindow()` — already exists |
|
||||||
|
| Signal handling | Custom signal loop | `signal.NotifyContext` — already wired in main.go |
|
||||||
|
| Protocol classification | New classifier | Extend existing `Classify()` with `hashBucket()` tail |
|
||||||
|
| Elapsed time | Custom timer | `time.Now()` / `time.Since()` — stdlib |
|
||||||
|
|
||||||
|
**Key insight:** Phase 3 is almost all integration. Custom solutions for any of these would contradict already-shipped, tested implementations.
|
||||||
|
|
||||||
|
## Common Pitfalls
|
||||||
|
|
||||||
|
### Pitfall 1: Missing `ClassUnknown` removal causes silent map miss in bank.go
|
||||||
|
|
||||||
|
**What goes wrong:** If `ClassUnknown` is left in `synth/config.go` but removed from `AllClasses()`, `NewBank()` will never create a layer for it and the config entry is dead. Conversely, if `AllClasses()` still returns `ClassUnknown` but `ClassFreqConfigs` has no entry for it, `NewBank()` will call `NewLayer(cfg, ...)` with a zero `FreqConfig` — a silent oscillator at 0 Hz that wastes memory and produces no sound. The compiler does not catch this.
|
||||||
|
|
||||||
|
**How to avoid:** Update `types.go`, `config.go`, and `classifier.go` in the same plan wave so they're always consistent. Write a test that calls `AllClasses()` and asserts every returned class has a corresponding entry in `synth.ClassFreqConfigs`.
|
||||||
|
|
||||||
|
**Warning signs:** `TestNewBankHas11Layers` still passes but producing 11 ≠ 14 audio layers.
|
||||||
|
|
||||||
|
### Pitfall 2: `bank_test.go` hardcoded layer count
|
||||||
|
|
||||||
|
**What goes wrong:** `TestNewBankHas11Layers` asserts `len(b.layers) != 11`. After updating `AllClasses()` to return 14 entries, this test fails. The test suite will be broken before any synthesis runs.
|
||||||
|
|
||||||
|
**How to avoid:** Update `TestNewBankHas11Layers` to assert `14` when updating `synth/config.go` and `classify/types.go`.
|
||||||
|
|
||||||
|
### Pitfall 3: `classifier_test.go` TestClassifyUnknown expects `ClassUnknown`
|
||||||
|
|
||||||
|
**What goes wrong:** `TestClassifyUnknown` at line 222 asserts `got.Class != classify.ClassUnknown`. After D-04 removes `ClassUnknown`, this will fail to compile.
|
||||||
|
|
||||||
|
**How to avoid:** Update `TestClassifyUnknown` to assert that the returned class is one of `ClassUnknown1`-`ClassUnknown4` (i.e., the string has `"unknown-"` prefix), not the removed constant.
|
||||||
|
|
||||||
|
### Pitfall 4: Audio duration shows 0 seconds for short captures
|
||||||
|
|
||||||
|
**What goes wrong:** `len(collectedSnapshots) * DefaultWindowMs / 1000` uses integer arithmetic. For captures shorter than 1 second (< 2 windows at 500ms), integer division truncates to 0.
|
||||||
|
|
||||||
|
**How to avoid:** Cast to `float64` before dividing: `float64(len(collectedSnapshots)) * float64(aggregate.DefaultWindowMs) / 1000.0`.
|
||||||
|
|
||||||
|
### Pitfall 5: `os.Stat` on output path fails when `RunSynthesis` returns an error
|
||||||
|
|
||||||
|
**What goes wrong:** If `RunSynthesis` returns an error (e.g., zero packets), calling `os.Stat(outputPath)` immediately after will fail or stat an empty path because the file was never created. The error handling sequence matters.
|
||||||
|
|
||||||
|
**How to avoid:** Only call `os.Stat` in the happy path — after confirming `RunSynthesis` returned `nil`.
|
||||||
|
|
||||||
|
## Code Examples
|
||||||
|
|
||||||
|
### Hash-bucketing in classifier.go
|
||||||
|
|
||||||
|
```go
|
||||||
|
// At the end of Classify(), replace all ClassUnknown return sites:
|
||||||
|
|
||||||
|
// No recognized transport layer → deterministic bucket assignment (D-01/D-03)
|
||||||
|
result.Class = hashBucket(result.DstPort, result.Protocol)
|
||||||
|
return result
|
||||||
|
|
||||||
|
// hashBucket maps an unrecognized packet to ClassUnknown1-4.
|
||||||
|
func hashBucket(dstPort uint16, protocol string) TrafficClass {
|
||||||
|
var protoNum uint16
|
||||||
|
switch protocol {
|
||||||
|
case "tcp":
|
||||||
|
protoNum = 6
|
||||||
|
case "udp":
|
||||||
|
protoNum = 17
|
||||||
|
case "icmp":
|
||||||
|
protoNum = 1
|
||||||
|
}
|
||||||
|
h := uint32(dstPort)*31 + uint32(protoNum)*7
|
||||||
|
switch h % 4 {
|
||||||
|
case 0:
|
||||||
|
return ClassUnknown1
|
||||||
|
case 1:
|
||||||
|
return ClassUnknown2
|
||||||
|
case 2:
|
||||||
|
return ClassUnknown3
|
||||||
|
default:
|
||||||
|
return ClassUnknown4
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### synth/config.go updated constants
|
||||||
|
|
||||||
|
```go
|
||||||
|
const (
|
||||||
|
SampleRate = 44100
|
||||||
|
WindowMs = 500
|
||||||
|
SamplesPerWindow = SampleRate * WindowMs / 1000 // 22050
|
||||||
|
NumLayers = 14
|
||||||
|
GainPerLayer = 1.0 / float64(NumLayers) // ~0.0714
|
||||||
|
WhisperFloor = 0.03
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### main.go integration (key flow)
|
||||||
|
|
||||||
|
```go
|
||||||
|
// After snapshot accumulator loop completes:
|
||||||
|
dropped := atomic.LoadInt64(droppedPtr)
|
||||||
|
if dropped > 0 {
|
||||||
|
fmt.Fprintf(os.Stderr, "\nWarning: %d packets dropped (channel buffer full)\n", dropped)
|
||||||
|
}
|
||||||
|
// D-08: summary before encoding
|
||||||
|
aggregate.PrintSummary(os.Stderr, totals)
|
||||||
|
|
||||||
|
// D-07: encoding status
|
||||||
|
fmt.Fprintf(os.Stderr, "Encoding %d windows to %s...\n", len(collectedSnapshots), outputPath)
|
||||||
|
encodeStart := time.Now()
|
||||||
|
if err := encode.RunSynthesis(collectedSnapshots, outputPath); err != nil {
|
||||||
|
return fmt.Errorf("synthesis failed: %w", err)
|
||||||
|
}
|
||||||
|
encodeElapsed := time.Since(encodeStart)
|
||||||
|
|
||||||
|
// D-09: saved confirmation with stats
|
||||||
|
audioDuration := float64(len(collectedSnapshots)) * float64(aggregate.DefaultWindowMs) / 1000.0
|
||||||
|
info, statErr := os.Stat(outputPath)
|
||||||
|
if statErr != nil {
|
||||||
|
return fmt.Errorf("stat output file: %w", statErr)
|
||||||
|
}
|
||||||
|
fmt.Fprintf(os.Stderr, "Saved %s (%.1fs, %d KB, encoded in %.1fs)\n",
|
||||||
|
outputPath, audioDuration, info.Size()/1024, encodeElapsed.Seconds())
|
||||||
|
return nil
|
||||||
|
```
|
||||||
|
|
||||||
|
## Environment Availability
|
||||||
|
|
||||||
|
| Dependency | Required By | Available | Version | Fallback |
|
||||||
|
|------------|------------|-----------|---------|----------|
|
||||||
|
| Go toolchain | Build | Verified (go.mod exists, tests pass) | go 1.24.1 | — |
|
||||||
|
| C compiler (gcc) | `sjzar/go-lame` CGo build | Assumed (Phase 2 already built) | — | — |
|
||||||
|
| ffprobe | `encode/mp3_test.go` validation | Unknown | — | Tests skip via `t.Skip` if absent |
|
||||||
|
|
||||||
|
All tests pass as of Phase 3 start (`go test ./...` returns `ok` for all 6 packages). No new external dependencies required.
|
||||||
|
|
||||||
|
## Validation Architecture
|
||||||
|
|
||||||
|
### Test Framework
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
|----------|-------|
|
||||||
|
| Framework | Go standard `testing` package |
|
||||||
|
| Config file | none — `go test ./...` convention |
|
||||||
|
| Quick run command | `go test ./classify/... ./synth/... ./cmd/... -count=1` |
|
||||||
|
| Full suite command | `go test ./... -count=1` |
|
||||||
|
|
||||||
|
### Phase Requirements → Test Map
|
||||||
|
|
||||||
|
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|
||||||
|
|--------|----------|-----------|-------------------|-------------|
|
||||||
|
| CLAS-02 | `hashBucket()` distributes packets to ClassUnknown1-4 (not ClassUnknown) | unit | `go test ./classify/... -run TestClassifyUnknown -count=1` | Update existing ✅ |
|
||||||
|
| CLAS-02 | All 4 buckets are reachable (hash covers all 4 outputs) | unit | `go test ./classify/... -run TestHashBucketDistribution -count=1` | New ❌ Wave 0 |
|
||||||
|
| CLAS-02 | `AllClasses()` returns exactly 14 classes, none is `ClassUnknown` | unit | `go test ./classify/... -run TestAllClassesCount -count=1` | New ❌ Wave 0 |
|
||||||
|
| CLAS-02 | `NewBank()` creates exactly 14 layers | unit | `go test ./synth/... -run TestNewBankHas11Layers -count=1` | Update existing ✅ |
|
||||||
|
| CLAS-02 | `ClassFreqConfigs` has entry for every class in `AllClasses()` | unit | `go test ./synth/... -run TestClassFreqConfigsComplete -count=1` | New ❌ Wave 0 |
|
||||||
|
| CAPT-03 | `RunSynthesis` is called in main; non-empty capture produces MP3 | integration | `go test ./encode/... -run TestMP3Valid -count=1` | Exists ✅ (passes already) |
|
||||||
|
| CAPT-03 | Encoding messages printed (Encoding/Saved) — manual verify | manual | Build and run with `sudo ./netsynth -i lo -o /tmp/test.mp3` | N/A |
|
||||||
|
|
||||||
|
### Sampling Rate
|
||||||
|
|
||||||
|
- **Per task commit:** `go test ./classify/... ./synth/... -count=1`
|
||||||
|
- **Per wave merge:** `go test ./... -count=1`
|
||||||
|
- **Phase gate:** Full suite green before `/gsd:verify-work`
|
||||||
|
|
||||||
|
### Wave 0 Gaps
|
||||||
|
|
||||||
|
- [ ] `classify/types_test.go` (or add to `classifier_test.go`) — `TestAllClassesCount` and `TestAllClassesNoUnknown`
|
||||||
|
- [ ] `classify/classifier_test.go` — `TestHashBucketDistribution` covering all 4 bucket outputs
|
||||||
|
- [ ] `synth/config_test.go` — `TestClassFreqConfigsComplete` asserting every class in `AllClasses()` has a `FreqConfig` entry
|
||||||
|
- [ ] Update `synth/bank_test.go:TestNewBankHas11Layers` → assert `14` (not `11`)
|
||||||
|
- [ ] Update `classify/classifier_test.go:TestClassifyUnknown` → assert `unknown-` prefix, not removed `ClassUnknown` constant
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
1. **`TestMixerNoClip` with 14 layers — will it still pass?**
|
||||||
|
- What we know: `GainPerLayer = 1/14 ≈ 0.0714`. With PanGains constant-power law, stereo channels cannot exceed `14 * (1/14) * 1.0 = 1.0`. Mathematically safe.
|
||||||
|
- What's unclear: Whether the fast-EMA convergence test `(tau=0.01, 10 render passes)` produces audible clipping at a transient peak during the first few windows.
|
||||||
|
- Recommendation: Run `TestMixerNoClip` immediately after `NumLayers` and `GainPerLayer` changes. If it fails, the test scenario (all 14 layers at 1000 pkts simultaneously) is an extreme worst-case that won't occur in real captures; the EMA will naturally limit overshoot.
|
||||||
|
|
||||||
|
2. **Unknown packets with no DstPort (ARP, raw Ethernet, etc.) always land in same bucket?**
|
||||||
|
- What we know: `dstPort=0, protocol=""` → `protoNum=0`, `h = 0*31 + 0*7 = 0`, `0 % 4 = 0` → always `ClassUnknown1`.
|
||||||
|
- What's unclear: Whether this is a concern. ARP produces low packet rates and the behavior is deterministic — acceptable per D-01.
|
||||||
|
- Recommendation: Document in a comment in `hashBucket()`. No code change needed.
|
||||||
|
|
||||||
|
## Sources
|
||||||
|
|
||||||
|
### Primary (HIGH confidence)
|
||||||
|
|
||||||
|
- Direct code reading: `cmd/netsynth/main.go`, `classify/types.go`, `classify/classifier.go`, `synth/config.go`, `synth/bank.go`, `encode/mp3.go`, `aggregate/summary.go` — all integration points verified by inspection.
|
||||||
|
- `go test ./...` run result: all 6 packages pass — baseline confirmed.
|
||||||
|
- `go.mod` — all required dependencies already present at pinned versions.
|
||||||
|
|
||||||
|
### Secondary (MEDIUM confidence)
|
||||||
|
|
||||||
|
- Phase 2 STATE.md decisions (`[Phase 02]: GainPerLayer applied in bank.go during mixing`) — confirm loop structure, no surprises.
|
||||||
|
- Phase 3 CONTEXT.md — all implementation decisions locked (D-01 through D-10).
|
||||||
|
|
||||||
|
### Tertiary (LOW confidence)
|
||||||
|
|
||||||
|
- Tone Hz recommendations (862, 920, 981, 1047) — chosen within D-05 constraints by applying chromatic detuning logic; no user validation yet. Subjective listening validation remains the final arbiter.
|
||||||
|
|
||||||
|
## Metadata
|
||||||
|
|
||||||
|
**Confidence breakdown:**
|
||||||
|
- Integration wiring: HIGH — one-line change, existing APIs match
|
||||||
|
- Hash-bucketing logic: HIGH — deterministic, no external deps, existing test patterns apply
|
||||||
|
- Tone Hz values: MEDIUM — within D-05 constraint, detuning math is sound, but subjective validation needed
|
||||||
|
- Test gap identification: HIGH — all test files inspected, hardcoded constants located
|
||||||
|
|
||||||
|
**Research date:** 2026-03-26
|
||||||
|
**Valid until:** Phase 3 complete (low churn domain; these are all in-repo findings)
|
||||||
Reference in New Issue
Block a user