docs: complete project research
This commit is contained in:
+113
-118
@@ -1,187 +1,182 @@
|
||||
# Project Research Summary
|
||||
|
||||
**Project:** NetSynth v1.1 — Custom Sound Mappings
|
||||
**Domain:** Network traffic sonification CLI tool (Go)
|
||||
**Researched:** 2026-03-26
|
||||
**Project:** NetSynth v1.2 — Extended Protocol Coverage with Grouped Sound Families
|
||||
**Domain:** Network traffic sonification CLI (Go) — packet capture to ambient MP3
|
||||
**Researched:** 2026-03-27
|
||||
**Confidence:** HIGH
|
||||
|
||||
## Executive Summary
|
||||
|
||||
NetSynth v1.1 extends the working v1.0 CLI by adding user-customizable sound mappings through a TOML config file. The v1.0 codebase is a clean 6-package Go project (3,254 lines) with a well-separated pipeline: capture → classify → aggregate → synthesize → encode. The v1.1 milestone threads a new `config` package through this pipeline, enabling users to override frequencies and waveforms per traffic class, add new classification rules, and reference those custom classes in the synth layer. The recommended implementation path is incremental — introduce waveform types at the oscillator level first, then decouple bank construction from the global config, then add the TOML loader and wire it all together. Each step is independently testable before the next begins.
|
||||
NetSynth v1.2 extends an already-shipped Go CLI tool that captures live network traffic and synthesizes it into an ambient MP3 soundscape. The existing v1.1 codebase (~4,675 lines, 7 packages) provides a validated pipeline: go-pcap capture → port-based classification → EMA-smoothed additive synthesis → LAME MP3 encoding. The v1.2 milestone adds ~21 new protocol classes across 7 sound families (Mail, Remote Access, File Transfer, Infrastructure expansion, Database, Directory/Auth, VoIP), expanding from 14 total classes to ~35. No new external dependencies are required — every new protocol is detectable via the existing port-based Rule classifier, and group identity is expressed purely through frequency proximity and waveform consistency in `ClassFreqConfigs`.
|
||||
|
||||
The stack requires one new dependency: `github.com/BurntSushi/toml` v1.6.0 (zero indirect deps, `MetaData.Undecoded()` provides strict-mode typo detection). All other additions — waveform math, config file discovery, and class name validation — are stdlib-only. The single notable technical choice is waveform synthesis strategy: naive direct-math square/sawtooth/triangle waveforms produce audible aliasing at the frequencies NetSynth uses (65–1047 Hz). The existing additive synthesis infrastructure (`[]HarmonicDef`) is the correct approach, generating bandlimited harmonic series for each waveform type rather than direct time-domain computation.
|
||||
The recommended approach is strictly additive: extend three locations in the codebase (`classify/types.go` for constants, `classify/rules.go` for port rules, `synth/config.go` for frequency configs) in a fixed order to keep tests green throughout. The highest-value protocols for v1.2 are the Tier 1 set (IMAP, POP3, FTP, SMB, RDP, mDNS, SSDP, SNMP, MySQL, PostgreSQL, Redis) because they appear on nearly every network type. The group-as-frequency-proximity design means family identity emerges from the audio itself — no new data structures are needed for the group concept beyond a `Group string` metadata field on `FreqConfig`.
|
||||
|
||||
The highest-risk integration points are bank construction (which must be extended to handle user-defined classes not in the static `AllClasses()` list) and TOML merge semantics (TOML decoders zero-out absent fields, silently overwriting defaults unless pointer fields are used). Both are well-understood problems with clear prevention patterns that must be established before wiring config into the pipeline. The merge ordering for classification rules also requires deliberate design: user rules must precede specific built-in rules, which must precede catch-alls — three-layer ordering, not two.
|
||||
The key risks are all backward-compatibility and perceptual-design concerns rather than engineering complexity. The critical risks are: (1) frequency rebalancing silently invalidating existing user TOML configs, (2) the `autoAssignFreq` range in `config.go` colliding with new built-in frequencies if not updated, and (3) within-family detuning that is psychoacoustically too narrow to be perceptually distinct. These are all preventable with explicit design choices made before any code is written. The milestone complexity is LOW — all changes are data additions to existing packages, and the build order is clear from the architecture research.
|
||||
|
||||
## Key Findings
|
||||
|
||||
### Recommended Stack
|
||||
|
||||
The existing stack (gopacket v1.5.0, packetcap/go-pcap, sjzar/go-lame v0.0.9, cobra v1.10.2, hand-rolled sine oscillator + EMA) is unchanged. The only new external dependency is `BurntSushi/toml` v1.6.0, selected over `pelletier/go-toml v2` because `MetaData.Undecoded()` is more ergonomic for typo detection on a single startup config read, and over `spf13/viper` because Viper pulls in 20+ transitive deps for features (remote config, env var binding, hot reload) that are irrelevant here. See `.planning/research/STACK.md` for full alternatives analysis.
|
||||
The v1.2 stack is identical to v1.1 — no new dependencies are required. All existing packages (gopacket/gopacket v1.5.0, packetcap/go-pcap, sjzar/go-lame v0.0.9, spf13/cobra v1.10.2, BurntSushi/toml v1.6.0) are validated and unchanged. The gopacket `layers` package has native decoders for SIP (LayerTypeSIP, id 133) and TLS (LayerTypeTLS, id 140) at v1.5.0, but port-based `Rule` matching is the correct and simpler approach for all v1.2 protocols — using native layer decoders would add code paths without classification benefit, since port numbers are unambiguous for every protocol in scope.
|
||||
|
||||
**Core technologies:**
|
||||
- `github.com/gopacket/gopacket` v1.5.0 — packet decode — only maintained Go packet library, Go 1.24+
|
||||
- `github.com/packetcap/go-pcap` — pure-Go live capture backend — no CGo, mmap ring buffer, Linux/macOS
|
||||
- `github.com/sjzar/go-lame` v0.0.9 — MP3 encoding — embeds LAME C source, CGO_ENABLED=1, no system library required
|
||||
- `github.com/spf13/cobra` v1.10.2 — CLI structure and flag handling — industry standard
|
||||
- `github.com/BurntSushi/toml` v1.6.0 — TOML config parsing — zero transitive deps, strict-mode via `Undecoded()`
|
||||
- Hand-rolled additive oscillator (sine today, square/sawtooth/triangle in v1.1) — no audio library needed
|
||||
**Core technologies (unchanged from v1.1):**
|
||||
- `gopacket/gopacket v1.5.0`: Packet capture and protocol layer decoding — community fork, actively maintained, Go 1.24+
|
||||
- `packetcap/go-pcap`: Pure-Go live capture via mmap ring buffer — no CGo, Linux/macOS
|
||||
- `sjzar/go-lame v0.0.9`: MP3 encoding with embedded LAME C source — no system library dependency
|
||||
- `spf13/cobra v1.10.2`: CLI flags, signal handling, --help generation
|
||||
- `BurntSushi/toml v1.6.0`: TOML config loading with strict unknown-key validation
|
||||
- Hand-rolled additive synth + EMA: oscillators with bandlimited harmonics, exponential moving average amplitude smoothing per layer
|
||||
|
||||
### Expected Features
|
||||
|
||||
**Must have (table stakes) for v1.1:**
|
||||
- TOML config auto-discovery (`./netsynth.toml`, `~/.config/netsynth/config.toml`) — XDG Base Directory Specification standard
|
||||
- `--config` flag for explicit path, with hard error if file is absent
|
||||
- Partial override semantics — absent keys retain defaults; users must not replicate the full table to change one field
|
||||
- Custom frequency per traffic class — direct override of `synth.ClassFreqConfigs`
|
||||
- Custom waveform per traffic class — sine/square/sawtooth/triangle selection
|
||||
- User-defined classification rules with custom class names, prepended before built-in rules
|
||||
- Startup-time config validation with line-number errors (fail before capture begins, not after)
|
||||
- Unknown field detection — prevents silent typos (`frequncy` must be caught, not silently ignored)
|
||||
**Must have (table stakes for v1.2):**
|
||||
- Mail family: IMAP/IMAPS (TCP 143, 993), POP3/POP3S (TCP 110, 995), SMTP submission (TCP 587, 465) — present on every office and home network
|
||||
- Remote Access expansion: RDP (TCP 3389), VNC (TCP 5900), Telnet (TCP 23) — completes the SSH family
|
||||
- File Transfer family: FTP (TCP 20, 21), SMB (TCP 445), TFTP (UDP 69) — ubiquitous on NAS and Windows networks
|
||||
- Infrastructure expansion: mDNS (UDP 5353), SSDP (UDP 1900), SNMP (UDP 161/162), Syslog (UDP 514) — constant background on all LAN segments
|
||||
- Database family: MySQL (TCP 3306), PostgreSQL (TCP 5432), Redis (TCP 6379), MongoDB (TCP 27017) — all currently land in other-TCP
|
||||
- Frequency allocation into family bands (group concept expressed via Hz proximity, not a new data structure)
|
||||
- AllClasses() and ClassFreqConfigs updated atomically — existing test `TestNumLayersMatchesAllClasses` enforces this invariant
|
||||
- --print-config reflects all new classes, organized with group-header comments
|
||||
|
||||
**Should have (differentiators):**
|
||||
- `netsynth --print-config` subcommand dumping effective config as commented TOML — critical for discoverability
|
||||
- Named custom rules (display name appears in exit summary and `--verbose` output)
|
||||
- Clear error message listing valid waveform values on invalid input
|
||||
**Should have (differentiators for v1.2):**
|
||||
- Directory/Auth family: LDAP/LDAPS (TCP 389, 636), Kerberos (UDP/TCP 88) — every enterprise/Windows network
|
||||
- VoIP family: SIP/SIP-TLS (UDP/TCP 5060, 5061) — IP phone traffic on office networks
|
||||
- QUIC/HTTP3 class (UDP 443) — distinct from HTTPS on TCP 443; significant fraction of modern web traffic
|
||||
- Within-family waveform consistency: protocols in a family share the same waveform type for timbral family identity
|
||||
- Group-header section comments in --print-config output (cosmetic, high value for user discoverability)
|
||||
|
||||
**Defer (v2+):**
|
||||
- Harmonic override per class (expose `HarmonicDef` slice in TOML) — niche, adds TOML nesting complexity
|
||||
- Stereo pan position in config — explicitly deferred per project constraints
|
||||
- Config hot-reload during capture — mid-capture state change corrupts synthesis; not worth the complexity
|
||||
- Multiple config file includes/inheritance — single file merged with in-code defaults is sufficient
|
||||
- Dynamic port protocols: RTP (negotiated ephemeral ports), FTP data channel — require stateful flow tracking across packets
|
||||
- Deep packet inspection for application-layer classification — massive scope, no sonification benefit over port matching
|
||||
- Collapsed "Mail" or "Database" class — loses per-protocol identity (can't distinguish IMAP inbound from SMTP outbound)
|
||||
- MQTT, AMQP, Kafka, BGP, OSPF — IoT/routing protocols absent on general networks; niche
|
||||
- Port-range rule support in the Rule struct — needed for RTP; valid v1.3 enhancement
|
||||
- Runtime group concept as a data structure — groups emerge from frequency proximity; no struct needed in v1.2
|
||||
|
||||
### Architecture Approach
|
||||
|
||||
v1.1 adds a `config` package and threads it through the existing pipeline via dependency injection. The key architectural shift: `synth.NewBank` currently reads the package-level global `ClassFreqConfigs`; after v1.1 it accepts a `map[TrafficClass]FreqConfig` parameter, enabling user-defined classes and eliminating hidden global state. The `config` package owns TOML parsing, file discovery, default config wrapping, and merge logic. `classify.DefaultRules` splits into `SpecificRules` + `CatchAllRules` so user rules can be inserted between them. The oscillator gains a `Waveform` enum field with additive-synthesis dispatch. All changes are contained to well-bounded components; `classify/classifier.go` and `encode/mp3.go` change only at their call sites. See `.planning/research/ARCHITECTURE.md` for full data flow diagrams and step-by-step build order.
|
||||
The v1.2 architecture is strictly additive to the existing pipeline: `config.Load` → `classify.NewClassifier` → `encode.RunSynthesis` → `synth.NewBank` → MP3. The only new concept is `Group string` on `FreqConfig` in the synth package — this metadata field is used only by `PrintConfig` for section headers and has zero effect on synthesis math. `bank.go`, `encode/mp3.go`, and `main.go` require no changes. Group identity is expressed architecturally through frequency proximity alone: protocols in the same family are assigned `BaseHz` values within a shared band, and within-band detuning of at least a major second interval (ratio 1.122) ensures psychoacoustic distinctness while maintaining timbral family coherence.
|
||||
|
||||
**Major components:**
|
||||
1. `config/` (new) — TOML struct definitions, `Load()`, auto-discovery via `os.UserConfigDir()`, pointer-field merge, `DefaultConfig()`
|
||||
2. `synth/oscillator.go` (modified) — `Waveform` enum, `sampleAt()` dispatch, bandlimited harmonic series generation at config load time
|
||||
3. `synth/bank.go` (modified) — accepts freq config map param; iterates map keys, not hardcoded `AllClasses()`
|
||||
4. `classify/rules.go` (modified) — split into `SpecificRules` + `CatchAllRules`; `MergeRules(userRules)` export
|
||||
5. `cmd/netsynth/main.go` (modified) — `--config` flag, `config.Load()`, user rule merge, config forwarded to `RunSynthesis`
|
||||
**Major components and their v1.2 changes:**
|
||||
1. `classify/types.go` — new TrafficClass constants + AllClasses() extended in group-coherent order (additive)
|
||||
2. `classify/rules.go` — new Rule entries before catch-alls for all new protocols; multi-port-to-single-class mapping for secure/insecure variants (additive; ordering is critical)
|
||||
3. `synth/config.go` — Group field on FreqConfig; new ClassFreqConfigs entries; frequency rebalancing for family bands; NumLayers constant cleanup (highest-risk change due to backward compatibility)
|
||||
4. `config/config.go` — optional group-header comments in PrintConfig (isolated to string output, no behavioral change)
|
||||
5. `bank.go`, `encode/mp3.go`, `main.go` — no changes required
|
||||
|
||||
**Recommended build order (each step independently testable):**
|
||||
1. Test and constant cleanup — remove stale NumLayers/GainPerLayer, update TestFrequenciesInRange bounds
|
||||
2. Protocol list and frequency design (no code) — finalize all Hz values in family bands, check ERB constraints
|
||||
3. New TrafficClass constants + AllClasses() in classify/types.go
|
||||
4. New DefaultRules in classify/rules.go (depends on step 3)
|
||||
5. Add Group field to FreqConfig — independent of steps 3/4
|
||||
6. Add ClassFreqConfigs entries with finalized frequencies; update autoAssignFreq base
|
||||
7. PrintConfig group-header comments (optional polish)
|
||||
|
||||
### Critical Pitfalls
|
||||
|
||||
1. **TOML decoder zeros absent fields, silently overwriting defaults (Pitfall A1)** — Use pointer fields (`*float64`, `*string`) for all optional overrides in the decoded struct. Apply an explicit merge function that only writes non-nil values over the built-in defaults. Establish this pattern before any config is wired into the bank.
|
||||
1. **Frequency rebalancing silently invalidates v1.1 user TOML configs** — Users with explicit Hz overrides will retain stale v1.1 values after rebalancing; users without overrides hear unexplained soundscape changes. Prevention: assign all new protocols to frequency ranges above 1047 Hz (unoccupied by v1.1 built-ins), leaving the 65–1047 Hz existing layout frozen. If existing class frequencies must move, document every Hz change in release notes.
|
||||
|
||||
2. **User rules appended after catch-alls are unreachable (Pitfall A5/A9)** — `DefaultRules` ends with catch-all rules (`DstPort: 0`) that match any TCP/UDP packet. Appending user rules after them makes user rules unreachable. Split into `SpecificRules` + `CatchAllRules`; merge order must be `userRules + SpecificRules + CatchAllRules`.
|
||||
2. **autoAssignFreq range [1200, 2350] Hz collision with new built-ins** — If new built-in classes use frequencies in the 1200–2350 Hz auto-assign range, user-defined custom classes can hash to the same frequency silently. Prevention: push autoAssignFreq base above all built-in frequencies (e.g., 4500 Hz) after finalizing v1.2 ClassFreqConfigs. Add a compile-time test asserting no built-in falls in the auto-assign range.
|
||||
|
||||
3. **User-defined classes have no bank layer — panic or silence (Pitfall A6)** — `NewBank` currently iterates `classify.AllClasses()` (a hardcoded list of 14 built-in classes). User classes will not be in that list. `NewBank` must iterate the keys of the merged `FreqConfig` map instead. Validate at config load that every rule's class name resolves to a configured sound entry.
|
||||
3. **NumLayers/GainPerLayer exported constant is stale** — bank.go uses `1.0 / float64(len(cfgs))` dynamically; the exported `synth.GainPerLayer` constant is frozen at 14-layer value. Any new v1.2 code referencing `synth.GainPerLayer` will compute wrong gain. Prevention: remove or deprecate the constant before adding any new classes; bank.go's dynamic computation is the sole authoritative source.
|
||||
|
||||
4. **Naive square/sawtooth/triangle waveforms produce audible aliasing (Pitfall A3)** — Direct time-domain math generates infinite harmonics that alias above Nyquist. Use additive synthesis: generate a bandlimited `[]HarmonicDef` series (odd harmonics for square/triangle, all harmonics for sawtooth, truncated at Nyquist) at config load time. The existing `HarmonicDef` infrastructure already supports this approach.
|
||||
4. **TestFrequenciesInRange hardcodes [60, 1100] Hz** — CI fails immediately when adding classes above 1100 Hz. Prevention: update the test bounds to the new valid range (e.g., [60, 4000]) before adding any ClassFreqConfigs entries outside the current range. This is a false blocker if not addressed first.
|
||||
|
||||
5. **BurntSushi/toml silently ignores unknown keys by default (Pitfall A2)** — Use `toml.Decode()` (not `Unmarshal`) to obtain `MetaData`, then call `md.Undecoded()` and return an error listing any unrecognized keys. Implement strict decoding from the first config load function.
|
||||
5. **Within-family detuning too narrow for psychoacoustic distinctness** — Fixed small Hz steps (e.g., 10 Hz) fall inside the critical band (ERB) at higher frequencies — at 1000 Hz ERB is ~72 Hz; at 2000 Hz it is ~117 Hz. Tones within the critical band merge perceptually. Prevention: use logarithmic interval separations — at minimum a major second (ratio 1.122). Verify all within-family pairs satisfy `|f2 - f1| > ERB(min(f1,f2))` before committing Hz values.
|
||||
|
||||
6. **Three-location atomic update required for each new class** — Adding a protocol requires updating AllClasses(), ClassFreqConfigs, and the TrafficClass constant. Missing any one causes test failures that are diagnostic but confusing. Prevention: update all three in the same commit, or introduce a single `builtinClassDefs` slice that drives both AllClasses() and validates ClassFreqConfigs.
|
||||
|
||||
7. **Secure/insecure protocol variants create frequency overcrowding if treated as separate classes** — SMTP:25, SMTP-submit:587, SMTPS:465 as three classes produces three frequencies where users want one "mail" sound. Prevention: map all ports of a protocol family to a single TrafficClass using multiple Rule entries. Port-to-class is many-to-one within a family.
|
||||
|
||||
## Implications for Roadmap
|
||||
|
||||
The v1.1 work has clear dependency ordering that directly dictates phase structure. The build order in ARCHITECTURE.md (7 steps, each independently testable) maps naturally to implementation phases.
|
||||
Based on the research, v1.2 should follow a 4-phase sequence driven by dependency order and the need to resolve design decisions before code decisions.
|
||||
|
||||
### Phase 1: Waveform Types in the Oscillator
|
||||
### Phase 1: Test and Constant Cleanup
|
||||
|
||||
**Rationale:** Zero external dependencies; pure math testable in isolation with golden-sample unit tests. The critical design decision — additive synthesis vs. direct math — must be made and locked in here. Switching after integration is a full oscillator rewrite.
|
||||
**Delivers:** `Waveform` enum, `sampleAt()` dispatch, `BandlimitedHarmonics()` generator; all four waveform types produce correct, alias-free output at all NetSynth frequencies.
|
||||
**Addresses:** Custom waveform per class (table stakes)
|
||||
**Avoids:** Pitfall A3 (aliasing from naive waveforms)
|
||||
**Files changed:** `synth/oscillator.go` only
|
||||
**Rationale:** Three existing tests and one exported constant actively block v1.2 work if not addressed first. Cleaning these up prevents confusing CI failures throughout the milestone.
|
||||
**Delivers:** Stale NumLayers/GainPerLayer constant removed or deprecated; TestFrequenciesInRange updated to accept new range; TestNumLayersMatchesAllClasses renamed and its invariant documented clearly.
|
||||
**Addresses:** Pitfalls C3 (stale GainPerLayer), C4 (hardcoded range test blocking correct additions)
|
||||
**Avoids:** Wasted debugging time on pre-existing issues presenting as new failures
|
||||
|
||||
### Phase 2: Decouple Bank from Global Config
|
||||
### Phase 2: Protocol List and Frequency Design (No Code)
|
||||
|
||||
**Rationale:** Prerequisite for config injection. `NewBank` must accept an injected config map before the `config` package exists. Wiring `Waveform` through `FreqConfig` → `Layer` → `Oscillator` is included here; zero-value default (`WaveformSine = 0`) means existing tests pass unchanged.
|
||||
**Delivers:** `NewBank(tau, cfgs map[TrafficClass]FreqConfig)`, `FreqConfig.Waveform` field, `synth/layer.go` updated. System is functionally identical to v1.0 but injectable.
|
||||
**Avoids:** Global-read anti-pattern (multiple places reading `ClassFreqConfigs`, ambiguous merge point)
|
||||
**Files changed:** `synth/config.go`, `synth/layer.go`, `synth/bank.go`, `encode/mp3.go`
|
||||
**Rationale:** The frequency allocation must be designed before any ClassFreqConfigs entries are written. Getting this wrong after the fact requires touching every entry. The autoAssignFreq collision and psychoacoustic ERB constraints must be resolved at design time.
|
||||
**Delivers:** Final protocol class list (Tier 1 + Tier 2 from FEATURES.md, multi-port collapsed to single class per family), complete Hz allocation for all ~35 classes in family bands, autoAssignFreq base set above all built-ins, ERB check confirming all within-family pairs are perceptually distinct.
|
||||
**Addresses:** Pitfalls C1 (backward compat), C2 (auto-assign collision), C6 (critical band masking), C7 (secure/insecure variant crowding)
|
||||
**Avoids:** Frequency rebalancing causing retroactive rework; tone merging within families discovered only during listening tests
|
||||
|
||||
### Phase 3: Config Package — TOML Loading and Merge
|
||||
### Phase 3: Classification Layer
|
||||
|
||||
**Rationale:** Core new infrastructure. Builds on the injectable bank signature from Phase 2. All config correctness patterns (pointer fields, strict decode, validation, string normalization) must be established here in isolation before any features are wired to the bank. Retrofitting these patterns after pipeline integration is significantly more expensive.
|
||||
**Delivers:** `config/` package with `Load()`, `os.UserConfigDir()` discovery, pointer-field merge, `validate()` with string normalization and enum checking, `DefaultConfig()` wrapping existing values.
|
||||
**Addresses:** Config auto-discovery, `--config` flag, partial override semantics, unknown field detection, startup validation, clear error messages
|
||||
**Avoids:** Pitfalls A1 (zero-value overwrite), A2 (silent typos), A4 (waveform string case), A7 (XDG ignored), A8 (missing explicit path), A10 (empty class name)
|
||||
**Rationale:** New constants and rules must exist before synth entries can reference them. This phase is purely additive to the classify package with no audio impact — safe to land and test in isolation.
|
||||
**Delivers:** All ~21 new TrafficClass constants, AllClasses() updated in family-grouped order, DefaultRules extended with new port rules (multi-port-to-single-class pattern applied per family).
|
||||
**Addresses:** Protocol coverage table stakes (Mail, Remote Access, File Transfer, Infrastructure, Database, Directory/Auth, VoIP families from FEATURES.md)
|
||||
**Avoids:** Pitfall C8 (three-location atomicity — enforced by existing test), Pitfall C9 (range-based protocols like RTP deferred)
|
||||
**Uses:** Port-based Rule classification — same pattern as existing SSH/HTTP/DNS rules, no new code paths
|
||||
|
||||
### Phase 4: Classification Rule Merging
|
||||
### Phase 4: Synthesis and Config Layer
|
||||
|
||||
**Rationale:** `classify/rules.go` must be split before any config-loading logic references the rule slice. The three-layer merge order is a design decision that, if wrong, produces silent failures with no error messages — it must be validated with unit tests before pipeline integration.
|
||||
**Delivers:** `classify.SpecificRules`, `classify.CatchAllRules`, `classify.MergeRules(userRules []Rule) []Rule`; user rules prepend correctly with both catch-all and specific-rule ordering.
|
||||
**Avoids:** Pitfall A5 (unreachable rules after catch-alls), Pitfall A9 (user rule shadowed by built-in specific rule for same port)
|
||||
|
||||
### Phase 5: Wire Config Through Pipeline — Frequency and Waveform Overrides
|
||||
|
||||
**Rationale:** Connects the config package to the synth layer for built-in classes only. Validates the full pipeline end-to-end before adding the complexity of user-defined classes. `encode.RunSynthesis` signature change is a breaking API change — all call sites must be updated in a single commit.
|
||||
**Delivers:** `--config` cobra flag, `config.Load()` in `main.go`, merged freq config map passed to `RunSynthesis` and `NewBank`; end-to-end test: TOML sets HTTPS to 200 Hz sawtooth, bank produces 200 Hz sawtooth layer.
|
||||
**Implements:** Config → synth integration
|
||||
|
||||
### Phase 6: User-Defined Classes End-to-End
|
||||
|
||||
**Rationale:** The most complex integration; requires all prior phases. User-defined classes create new `TrafficClass` strings that must exist in both the merged rule set and the bank's layer map. The AllClasses() decoupling from Phase 2 makes this tractable.
|
||||
**Delivers:** `[[rules]]` TOML section, dynamic `TrafficClass` values from config, bank layers constructed from merged FreqConfig map keys, class name cross-validation at config load.
|
||||
**Avoids:** Pitfall A6 (user-defined class has no bank layer — nil panic or silence)
|
||||
|
||||
### Phase 7: Print-Config and UX Polish
|
||||
|
||||
**Rationale:** `--print-config` is independent of capture and must wait until all config structure is stable (Phase 6). Additive, zero regression risk.
|
||||
**Delivers:** `netsynth --print-config` subcommand with commented TOML output of effective config; optional `name` field on user rules displayed in exit summary.
|
||||
**Rationale:** ClassFreqConfigs entries depend on both the protocol list from Phase 2 design and the TrafficClass constants from Phase 3. The Group field on FreqConfig and PrintConfig group headers are the final polish on this phase.
|
||||
**Delivers:** All new FreqConfig entries with finalized frequencies from Phase 2; Group field added to FreqConfig; autoAssignFreq base updated and compile-time range test added; optional PrintConfig group-header section comments.
|
||||
**Addresses:** Table stakes (--print-config reflects new classes); differentiators (within-family tonal design, group headers)
|
||||
**Avoids:** Pitfall C2 (confirmed by new compile-time test), Pitfall C5 (no new TOML top-level keys since groups are not user-configurable)
|
||||
**Manual validation required:** A listening test with a real or synthetic pcap file is needed after this phase — automated tests cannot substitute for ear confirmation that family identity is perceptually clear.
|
||||
|
||||
### Phase Ordering Rationale
|
||||
|
||||
- Phases 1–2 are internal refactors with no user-visible change — the right starting point for establishing patterns safely
|
||||
- Phase 3 owns all config safety in one isolated package before anything is wired — retrofitting pointer-field merge after bank integration means touching multiple packages simultaneously
|
||||
- Phase 4 (rule splitting) must precede Phase 6 (user rules) or catch-all ordering bugs surface silently at integration with no clear failure signal
|
||||
- Phase 5 validates the full pipeline with familiar built-in classes before Phase 6 introduces the harder user-defined class problem
|
||||
- Phase 7 is pure additive polish with zero risk of breaking earlier phases
|
||||
- Phase 1 before everything because pre-existing test blockers cause false CI failures throughout the milestone
|
||||
- Phase 2 (design) before code because Hz allocation is the hardest-to-change decision with the broadest blast radius; fixing retroactively touches every ClassFreqConfigs entry
|
||||
- Phase 3 before Phase 4 because TrafficClass constants must exist before ClassFreqConfigs can reference them
|
||||
- Phase 4 is last because it depends on both the design (Phase 2) and the constants (Phase 3)
|
||||
|
||||
### Research Flags
|
||||
|
||||
Phases with well-documented patterns — skip additional research:
|
||||
- **Phase 1:** DSP textbook math; harmonic series are fully specified
|
||||
- **Phase 2:** Standard dependency injection refactor; no unknowns
|
||||
- **Phase 3:** BurntSushi/toml API is well-documented; pointer-field merge is a known TOML pattern
|
||||
- **Phase 4:** Simple slice manipulation; no external dependencies
|
||||
- **Phase 7:** Cobra subcommand and TOML marshal are standard patterns
|
||||
Phases with well-documented patterns (no additional research needed):
|
||||
- **Phase 1:** Straightforward constant and test cleanup; all relevant code is in the existing codebase
|
||||
- **Phase 3:** Port rules follow the exact same pattern as existing SSH/HTTP/DNS rules; no new patterns or unknowns
|
||||
- **Phase 4:** FreqConfig additions follow the exact same pattern as existing entries; the Group field is a non-functional metadata addition
|
||||
|
||||
Phases that may benefit from a targeted research pass or design review:
|
||||
- **Phase 5:** `encode.RunSynthesis` signature change is a breaking API change — verify all test call sites and plan a single-commit update
|
||||
- **Phase 6:** User-defined class name collision with built-in `TrafficClass` string values (e.g., user names a class `"HTTPS"`) requires a design decision: treat as override of built-in sound vs. reject as ambiguous. Not resolved in research; decide before coding Phase 6.
|
||||
Phases that require design validation:
|
||||
- **Phase 2:** The frequency allocation should be validated with a listening test on a real pcap file before committing to final Hz values. The ERB computations are straightforward math; the perceptual result requires ear confirmation. This is inherent to audio design work.
|
||||
|
||||
## Confidence Assessment
|
||||
|
||||
| Area | Confidence | Notes |
|
||||
|------|------------|-------|
|
||||
| Stack | HIGH | Existing stack validated in v1.0; BurntSushi/toml v1.6.0 confirmed via pkg.go.dev and GitHub releases; `os.UserConfigDir()` XDG behavior confirmed against official Go stdlib docs |
|
||||
| Features | HIGH | Config file conventions verified against XDG spec, git, golangci-lint, and mise patterns; TOML schema grounded in existing v1.0 codebase types |
|
||||
| Architecture | HIGH | Based on direct code inspection of the 3,254-line v1.0 codebase; all integration points identified with specific file/line references and build order prescribed |
|
||||
| Pitfalls | HIGH | TOML default-overwrite behavior verified against upstream issue trackers (BurntSushi/toml #47, go-toml #252); aliasing prevention verified against DSP literature (CCRMA paper, McGill bandlimited synthesis notes) |
|
||||
| Stack | HIGH | Validated against shipped v1.1 codebase; no new dependencies; gopacket layer types confirmed via direct GitHub source inspection |
|
||||
| Features | MEDIUM-HIGH | Protocol selection based on IANA port registry, nDPI taxonomy, Wireshark dissectors; real-world traffic frequency is inference, not measurement |
|
||||
| Architecture | HIGH | Derived from direct inspection of shipped v1.1 code (~4,675 lines); all integration points identified with specific file references and build order |
|
||||
| Pitfalls | HIGH | All critical pitfalls grounded in specific code locations (config.go merge(), bank.go gainPerLayer, synth/config_test.go assertions); audio masking values from Glasberg & Moore 1990 ERB model |
|
||||
|
||||
**Overall confidence:** HIGH
|
||||
**Overall confidence: HIGH**
|
||||
|
||||
### Gaps to Address
|
||||
|
||||
- **User class name collision with built-in class strings:** If a user writes `class = "HTTPS"` in a `[[rules]]` block, the intent could be "override built-in sound" or "create a parallel custom class." The merge logic needs an explicit decision before Phase 6: treat matching names as overrides (simplest) or require a separate TOML section. This is a UX design question — resolve before coding Phase 6.
|
||||
- **BurntSushi/toml vs. go-toml v2:** Both work for this use case. BurntSushi is recommended for ergonomics, but if the team prefers go-toml v2's `DisallowUnknownFields()` pattern, it is equally valid with minor API differences. Either choice is fine; just make one and be consistent.
|
||||
- **Frequency allocation requires listening validation:** No automated test replaces ear-testing. Plan a listening session with a diverse pcap file after Phase 4 before declaring the milestone complete.
|
||||
- **Protocol frequency on real networks:** Tier 1/2 ranking is based on typical network types, not measurement on the target user's actual network. mDNS and SSDP are prominent on home/office LANs but absent on cloud workloads. The catch-all classes handle unrecognized traffic regardless, so this is a coverage quality concern, not a correctness concern.
|
||||
- **Rule schema for range-based protocols:** RTP and NetBIOS (multi-port, non-contiguous) are explicitly deferred. If desired in v1.3+, the Rule struct needs `DstPortMin/Max` fields — a known future gap, not a v1.2 concern.
|
||||
|
||||
## Sources
|
||||
|
||||
### Primary (HIGH confidence)
|
||||
|
||||
- Direct code inspection: `synth/config.go`, `synth/oscillator.go`, `synth/bank.go`, `synth/layer.go`, `classify/classifier.go`, `classify/rules.go`, `classify/types.go`, `encode/mp3.go`, `cmd/netsynth/main.go`
|
||||
- `pkg.go.dev/github.com/BurntSushi/toml` — v1.6.0 API, `Undecoded()` strict mode, pointer field behavior
|
||||
- `github.com/BurntSushi/toml/issues/47` — default-overwrite behavior when using `Unmarshal` confirmed
|
||||
- `pkg.go.dev/os#UserConfigDir` — XDG_CONFIG_HOME behavior on Linux confirmed via official Go stdlib docs
|
||||
- `ccrma.stanford.edu/~stilti/papers/blit.pdf` — bandlimited synthesis theory (alias-free waveforms)
|
||||
- `music.mcgill.ca/~gary/307/week5/bandlimited.html` — truncated harmonic series approach confirmed
|
||||
- `github.com/gopacket/gopacket/blob/master/layers/layertypes.go` — LayerTypeSIP (id 133), LayerTypeTLS (id 140) confirmed at v1.5.0
|
||||
- `github.com/gopacket/gopacket/blob/master/layers/ports.go` — UDP/TCP port pre-registration confirmed; mDNS/SNMP/QUIC absent
|
||||
- `github.com/gopacket/gopacket/tree/master/layers` — directory listing; no mdns.go, quic.go, snmp.go, ldap.go, smb.go, rdp.go
|
||||
- Direct codebase inspection: `synth/config.go`, `synth/bank.go`, `classify/types.go`, `classify/rules.go`, `classify/classifier.go`, `config/config.go`, `encode/mp3.go`, `cmd/netsynth/main.go`
|
||||
- Glasberg & Moore 1990 ERB model: `ERB(f) = 24.7 * (4.37 * f/1000 + 1)` — critical bandwidth values for all relevant frequencies
|
||||
|
||||
### Secondary (MEDIUM confidence)
|
||||
|
||||
- XDG Base Directory Specification — config discovery precedence order
|
||||
- `mise.jdx.dev/configuration.html` — working-dir + XDG config discovery pattern
|
||||
- `golangci-lint.run/docs/configuration/cli/` — partial override config in Go CLI tools
|
||||
- `dylanmeeus.github.io/posts/audio-from-scratch-pt8/` — Go waveform synthesis from scratch, confirms no library needed
|
||||
- `pkg.go.dev/github.com/pelletier/go-toml/v2` — v2.3.0 strict decoder comparison; `pelletier/go-toml/issues/252` partial v2 resolution of default-overwrite
|
||||
- IANA Service Name and Transport Protocol Port Number Registry — authoritative port assignments for all new protocols
|
||||
- nDPI Protocols List (ntop) — 450+ protocol taxonomy; 17-category grouping model as design precedent
|
||||
- nDPI 5.0 Enhanced Traffic Fingerprinting blog post — category-based grouping confirmed as production approach
|
||||
- SoNSTAR: Sonification of Network Traffic (Paul Vickers) — academic network sonification reference
|
||||
- Sonification of network traffic flow (PLoS One 2018) — research on perceptually useful protocol groupings
|
||||
|
||||
### Tertiary (LOW confidence)
|
||||
|
||||
- Competitor feature table (SoNSTAR, Network-Sonification, Peep) — niche domain, limited documentation; used for context only, not binding decisions
|
||||
- Protocol prevalence on "typical" networks (home/office/server/cloud) — inferred from nDPI taxonomy and Wireshark dissector popularity; not empirically measured on target networks
|
||||
|
||||
---
|
||||
*Research completed: 2026-03-26*
|
||||
*Research completed: 2026-03-27*
|
||||
*Ready for roadmap: yes*
|
||||
|
||||
Reference in New Issue
Block a user