docs: complete project research

This commit is contained in:
2026-03-26 16:54:31 +01:00
parent 8651c47b0f
commit d413d1243f
5 changed files with 1218 additions and 865 deletions
+111 -112
View File
@@ -1,188 +1,187 @@
# Project Research Summary
**Project:** NetSynth
**Domain:** Network traffic sonification CLI — Go, packet capture, audio synthesis, MP3 encoding
**Researched:** 2026-03-24
**Confidence:** MEDIUM-HIGH
**Project:** NetSynth v1.1 — Custom Sound Mappings
**Domain:** Network traffic sonification CLI tool (Go)
**Researched:** 2026-03-26
**Confidence:** HIGH
## Executive Summary
NetSynth is a Go CLI tool that captures live network traffic, classifies packets by protocol, and synthesizes an ambient drone MP3 where each protocol layer produces a distinct tonal frequency whose amplitude evolves with traffic volume. There is no direct precedent for this exact form factor: comparable tools (SoNSTAR, Peep, Network-Sonification) all produce real-time audio through OS audio APIs rather than file output, are not single binaries, and do not auto-cluster unknown traffic. The recommended approach builds on well-understood Go concurrency primitives (channel-connected pipeline stages, ticker-driven time windows) rather than audio or ML libraries — the synthesis math is ~100 lines of Go and no external audio framework adds value for batch file output.
NetSynth v1.1 extends the working v1.0 CLI by adding user-customizable sound mappings through a TOML config file. The v1.0 codebase is a clean 6-package Go project (3,254 lines) with a well-separated pipeline: capture → classify → aggregate → synthesize → encode. The v1.1 milestone threads a new `config` package through this pipeline, enabling users to override frequencies and waveforms per traffic class, add new classification rules, and reference those custom classes in the synth layer. The recommended implementation path is incremental — introduce waveform types at the oscillator level first, then decouple bank construction from the global config, then add the TOML loader and wire it all together. Each step is independently testable before the next begins.
The recommended stack is `gopacket/gopacket` v1.5.0 (community fork, not the abandoned Google repo) for packet decode, `packetcap/go-pcap` for pure-Go live capture on Linux/macOS, `sjzar/go-lame` v0.0.9 for embedded-CGo MP3 encoding, and `spf13/cobra` v1.10.2 for CLI structure. The audio synthesis layer should be hand-rolled — additive sine oscillators with exponential-moving-average amplitude smoothing. This stack requires CGo at build time but produces a self-contained binary with no runtime library dependencies beyond `CAP_NET_RAW` or root for packet capture.
The stack requires one new dependency: `github.com/BurntSushi/toml` v1.6.0 (zero indirect deps, `MetaData.Undecoded()` provides strict-mode typo detection). All other additions — waveform math, config file discovery, and class name validation — are stdlib-only. The single notable technical choice is waveform synthesis strategy: naive direct-math square/sawtooth/triangle waveforms produce audible aliasing at the frequencies NetSynth uses (651047 Hz). The existing additive synthesis infrastructure (`[]HarmonicDef`) is the correct approach, generating bandlimited harmonic series for each waveform type rather than direct time-domain computation.
The two hardest risks are at opposite ends of the pipeline. On the capture side: privilege requirements, CGo binary distribution contracts, and kernel buffer drops all bite in production but not in local development. On the audio side: PCM integer overflow, perceptual tone masking between protocol frequencies, and LAME initialization order all produce silent or subtle corruption that integration tests must specifically cover. Both risk clusters must be resolved in Phase 1 and Phase 2 respectively — they cannot be retrofitted.
The highest-risk integration points are bank construction (which must be extended to handle user-defined classes not in the static `AllClasses()` list) and TOML merge semantics (TOML decoders zero-out absent fields, silently overwriting defaults unless pointer fields are used). Both are well-understood problems with clear prevention patterns that must be established before wiring config into the pipeline. The merge ordering for classification rules also requires deliberate design: user rules must precede specific built-in rules, which must precede catch-alls — three-layer ordering, not two.
## Key Findings
### Recommended Stack
The stack is straightforward for Go developers with one critical trap: `github.com/google/gopacket` is unmaintained (270 open issues, no active merges since 2022) and must never be used — the import path is `github.com/gopacket/gopacket` (community fork, v1.5.0, Go 1.24+ required). For the capture backend, `packetcap/go-pcap` is pure Go and eliminates libpcap CGo entirely; the MP3 encoder `sjzar/go-lame` embeds LAME C source and requires CGo but no system library on the target machine. Audio synthesis should be written directly — no audio library is appropriate for batch file output. See `.planning/research/STACK.md` for full alternatives matrix.
The existing stack (gopacket v1.5.0, packetcap/go-pcap, sjzar/go-lame v0.0.9, cobra v1.10.2, hand-rolled sine oscillator + EMA) is unchanged. The only new external dependency is `BurntSushi/toml` v1.6.0, selected over `pelletier/go-toml v2` because `MetaData.Undecoded()` is more ergonomic for typo detection on a single startup config read, and over `spf13/viper` because Viper pulls in 20+ transitive deps for features (remote config, env var binding, hot reload) that are irrelevant here. See `.planning/research/STACK.md` for full alternatives analysis.
**Core technologies:**
- `github.com/gopacket/gopacket` v1.5.0: packet decode (ICMP, DNS, TCP, UDP, TLS layers) — only maintained Go packet library
- `github.com/packetcap/go-pcap`: live capture backend — pure Go, mmap ring buffer, Linux/macOS, no CGo
- `github.com/sjzar/go-lame` v0.0.9: MP3 encoding — embeds LAME C source, no runtime `.so` dependency, requires CGo at build time
- `github.com/spf13/cobra` v1.10.2: CLI flag parsing — industry standard, handles signal plumbing and help generation
- `github.com/go-audio/wav`: WAV intermediate format — decouples synthesis from encoding, provides debug artifact
- Hand-rolled additive synthesizer: 30-50 lines of Go sine oscillators, no library needed
- `github.com/gopacket/gopacket` v1.5.0 packet decode — only maintained Go packet library, Go 1.24+
- `github.com/packetcap/go-pcap` — pure-Go live capture backend — no CGo, mmap ring buffer, Linux/macOS
- `github.com/sjzar/go-lame` v0.0.9 MP3 encoding — embeds LAME C source, CGO_ENABLED=1, no system library required
- `github.com/spf13/cobra` v1.10.2 CLI structure and flag handling — industry standard
- `github.com/BurntSushi/toml` v1.6.0 — TOML config parsing — zero transitive deps, strict-mode via `Undecoded()`
- Hand-rolled additive oscillator (sine today, square/sawtooth/triangle in v1.1) — no audio library needed
### Expected Features
NetSynth has a well-defined feature set. All precedent tools provide real-time audio output, not file output — this is both a differentiator and a source of user confusion to address in UX copy. Auto-clustering of unknown traffic is unique to NetSynth among comparable tools. See `.planning/research/FEATURES.md` for full prioritization matrix and competitor analysis.
**Must have (table stakes) for v1.1:**
- TOML config auto-discovery (`./netsynth.toml`, `~/.config/netsynth/config.toml`) — XDG Base Directory Specification standard
- `--config` flag for explicit path, with hard error if file is absent
- Partial override semantics — absent keys retain defaults; users must not replicate the full table to change one field
- Custom frequency per traffic class — direct override of `synth.ClassFreqConfigs`
- Custom waveform per traffic class — sine/square/sawtooth/triangle selection
- User-defined classification rules with custom class names, prepended before built-in rules
- Startup-time config validation with line-number errors (fail before capture begins, not after)
- Unknown field detection — prevents silent typos (`frequncy` must be caught, not silently ignored)
**Must have (table stakes):**
- Interface selection (`-i eth0`) and `--list-interfaces` — packet capture CLI convention
- Live capture with graceful Ctrl+C stop producing a valid MP3 — core interaction model
- Per-protocol sound distinction: ICMP, DNS, TCP/443, TCP/other, UDP — minimum fingerprint set
- Ambient drone synthesis with time-windowed amplitude evolution — core differentiator
- MP3 encoding and file output with sensible default filename — deliverable artifact
- Capture statistics summary on exit (stderr) — expected by every capture tool user
- Privilege error detection with actionable message — prevents silent failure
- Minimum viable duration guard — zero-packet capture must not produce a corrupt file
**Should have (competitive):**
- Auto-clustering of unrecognized traffic into stable drone layers — honest representation, unique feature
- BPF capture filter (`--filter`) — power user scope control
- Offline pcap file input (`--read`) — historical analysis and demos
- Verbose protocol activity log (`--verbose`) — debugging and demo use cases
- Configurable time window duration (`--window`) — tuning for responsiveness vs. smoothness
**Should have (differentiators):**
- `netsynth --print-config` subcommand dumping effective config as commented TOML — critical for discoverability
- Named custom rules (display name appears in exit summary and `--verbose` output)
- Clear error message listing valid waveform values on invalid input
**Defer (v2+):**
- Custom sound mapping configuration file — needs user research first; well-chosen defaults cover v1
- Improved clustering (full k-means on flow features) — hash-bucketing is sufficient for v1
- Multi-interface capture — adds deduplication complexity
- Real-time audio playback — anti-feature; triples cross-platform complexity, conflicts with file-output simplicity
- Harmonic override per class (expose `HarmonicDef` slice in TOML) — niche, adds TOML nesting complexity
- Stereo pan position in config — explicitly deferred per project constraints
- Config hot-reload during capture — mid-capture state change corrupts synthesis; not worth the complexity
- Multiple config file includes/inheritance — single file merged with in-code defaults is sufficient
### Architecture Approach
The architecture is a channel-connected pipeline of five stages, each a goroutine communicating via buffered channels, with a `done` channel closed on Ctrl+C driving clean shutdown across all stages. The stages are: Capture (pcap handle → `chan gopacket.Packet`), Classification (rule-based protocol dispatch + unknown traffic bucketer → `chan ClassifiedPacket`), Aggregation (ticker-driven time-window accumulator → `chan WindowSnapshot`), Synthesis (per-class sine oscillators with EMA amplitude smoothing → PCM blocks), and Encoding (LAME encoder goroutine consuming PCM blocks, separate from synthesis to decouple CGo latency). This is the idiomatic Go pipeline pattern and directly follows the build order the architecture research prescribes. See `.planning/research/ARCHITECTURE.md` for full data flow diagrams and channel buffer size recommendations.
v1.1 adds a `config` package and threads it through the existing pipeline via dependency injection. The key architectural shift: `synth.NewBank` currently reads the package-level global `ClassFreqConfigs`; after v1.1 it accepts a `map[TrafficClass]FreqConfig` parameter, enabling user-defined classes and eliminating hidden global state. The `config` package owns TOML parsing, file discovery, default config wrapping, and merge logic. `classify.DefaultRules` splits into `SpecificRules` + `CatchAllRules` so user rules can be inserted between them. The oscillator gains a `Waveform` enum field with additive-synthesis dispatch. All changes are contained to well-bounded components; `classify/classifier.go` and `encode/mp3.go` change only at their call sites. See `.planning/research/ARCHITECTURE.md` for full data flow diagrams and step-by-step build order.
**Major components:**
1. `capture/` — pcap handle wrapper; privilege/interface boundary; all downstream code is pcap-free
2. `classify/` — protocol rule dispatch + feature-hash bucketer for unknown traffic; testable with synthetic packets
3. `aggregate/` — ticker-driven time-window accumulator; the only stateful time-domain component
4. `synth/` — sine oscillators + EMA amplitude smoothing + mixer; pure PCM math, no I/O
5. `encode/` — CGo LAME boundary; decoupled encoder goroutine; if encoder changes, only this package changes
6. `config/` — static frequency-to-protocol mapping table; prevents magic numbers in synth/
1. `config/` (new) — TOML struct definitions, `Load()`, auto-discovery via `os.UserConfigDir()`, pointer-field merge, `DefaultConfig()`
2. `synth/oscillator.go` (modified) — `Waveform` enum, `sampleAt()` dispatch, bandlimited harmonic series generation at config load time
3. `synth/bank.go` (modified) — accepts freq config map param; iterates map keys, not hardcoded `AllClasses()`
4. `classify/rules.go` (modified) — split into `SpecificRules` + `CatchAllRules`; `MergeRules(userRules)` export
5. `cmd/netsynth/main.go` (modified) — `--config` flag, `config.Load()`, user rule merge, config forwarded to `RunSynthesis`
### Critical Pitfalls
Nine pitfalls identified, ranging from critical (causes data corruption or broken binaries) to moderate (causes poor audio quality). The top five require architectural decisions in Phase 1 or Phase 2 — they cannot be patched later without significant rewrite. See `.planning/research/PITFALLS.md` for recovery strategies and the full "Looks Done But Isn't" checklist.
1. **TOML decoder zeros absent fields, silently overwriting defaults (Pitfall A1)** — Use pointer fields (`*float64`, `*string`) for all optional overrides in the decoded struct. Apply an explicit merge function that only writes non-nil values over the built-in defaults. Establish this pattern before any config is wired into the bank.
1. **Wrong gopacket fork (`google/` vs `gopacket/`)** — use `github.com/gopacket/gopacket` from day one; import path migration across the whole codebase is the recovery cost
2. **CGo breaking the single-binary promise** — decide the static vs. dynamic linking strategy before writing capture code; verify with `ldd` on a clean Alpine container in CI
3. **`CAP_NET_RAW` + nosuid filesystem = silent failure** — install to `/usr/local/bin` for capability mode; always provide a `sudo ./netsynth` fallback with clear error messaging
4. **PCM integer overflow producing wrap-around distortion** — synthesize internally in `float64 [-1.0, 1.0]`, clamp before casting to `int16`; never do audio math in integer types
5. **Perceptual tone masking (all protocols in the same frequency band)** — design the frequency table with register separation: low drones for bulk traffic, mid for control, high for interactive; use harmonic intervals not arithmetic spacing
2. **User rules appended after catch-alls are unreachable (Pitfall A5/A9)**`DefaultRules` ends with catch-all rules (`DstPort: 0`) that match any TCP/UDP packet. Appending user rules after them makes user rules unreachable. Split into `SpecificRules` + `CatchAllRules`; merge order must be `userRules + SpecificRules + CatchAllRules`.
3. **User-defined classes have no bank layer — panic or silence (Pitfall A6)**`NewBank` currently iterates `classify.AllClasses()` (a hardcoded list of 14 built-in classes). User classes will not be in that list. `NewBank` must iterate the keys of the merged `FreqConfig` map instead. Validate at config load that every rule's class name resolves to a configured sound entry.
4. **Naive square/sawtooth/triangle waveforms produce audible aliasing (Pitfall A3)** — Direct time-domain math generates infinite harmonics that alias above Nyquist. Use additive synthesis: generate a bandlimited `[]HarmonicDef` series (odd harmonics for square/triangle, all harmonics for sawtooth, truncated at Nyquist) at config load time. The existing `HarmonicDef` infrastructure already supports this approach.
5. **BurntSushi/toml silently ignores unknown keys by default (Pitfall A2)** — Use `toml.Decode()` (not `Unmarshal`) to obtain `MetaData`, then call `md.Undecoded()` and return an error listing any unrecognized keys. Implement strict decoding from the first config load function.
## Implications for Roadmap
Based on research, the architecture's build-order prescription maps directly to a three-phase roadmap. Each phase is independently testable before the next is wired in.
The v1.1 work has clear dependency ordering that directly dictates phase structure. The build order in ARCHITECTURE.md (7 steps, each independently testable) maps naturally to implementation phases.
### Phase 1: Capture and Classification Pipeline
### Phase 1: Waveform Types in the Oscillator
**Rationale:** The packet capture layer carries the highest technical risk (privilege, CGo, binary distribution, buffer overflow). Validating it first — before any audio code exists — means the hardest pitfalls are resolved while the codebase is small. The architecture research explicitly names this as the correct first step.
**Rationale:** Zero external dependencies; pure math testable in isolation with golden-sample unit tests. The critical design decision — additive synthesis vs. direct math — must be made and locked in here. Switching after integration is a full oscillator rewrite.
**Delivers:** `Waveform` enum, `sampleAt()` dispatch, `BandlimitedHarmonics()` generator; all four waveform types produce correct, alias-free output at all NetSynth frequencies.
**Addresses:** Custom waveform per class (table stakes)
**Avoids:** Pitfall A3 (aliasing from naive waveforms)
**Files changed:** `synth/oscillator.go` only
**Delivers:** A working CLI that opens a network interface, classifies packets by protocol, and prints a live traffic summary to stderr. No audio output yet — just proof the pipeline works. The privilege error message and `--list-interfaces` flag ship here.
### Phase 2: Decouple Bank from Global Config
**Addresses:** Interface selection, live capture, protocol classification, privilege detection, `--list-interfaces`, capture statistics
**Rationale:** Prerequisite for config injection. `NewBank` must accept an injected config map before the `config` package exists. Wiring `Waveform` through `FreqConfig``Layer``Oscillator` is included here; zero-value default (`WaveformSine = 0`) means existing tests pass unchanged.
**Delivers:** `NewBank(tau, cfgs map[TrafficClass]FreqConfig)`, `FreqConfig.Waveform` field, `synth/layer.go` updated. System is functionally identical to v1.0 but injectable.
**Avoids:** Global-read anti-pattern (multiple places reading `ClassFreqConfigs`, ambiguous merge point)
**Files changed:** `synth/config.go`, `synth/layer.go`, `synth/bank.go`, `encode/mp3.go`
**Avoids:** Wrong gopacket fork (Pitfall 1), CGo distribution contract (Pitfall 2), CAP_NET_RAW binary location (Pitfall 3), packet buffer overflow (Pitfall 4), ZeroCopy use-after-free (Pitfall 5)
### Phase 3: Config Package — TOML Loading and Merge
### Phase 2: Audio Synthesis Engine
**Rationale:** Core new infrastructure. Builds on the injectable bank signature from Phase 2. All config correctness patterns (pointer fields, strict decode, validation, string normalization) must be established here in isolation before any features are wired to the bank. Retrofitting these patterns after pipeline integration is significantly more expensive.
**Delivers:** `config/` package with `Load()`, `os.UserConfigDir()` discovery, pointer-field merge, `validate()` with string normalization and enum checking, `DefaultConfig()` wrapping existing values.
**Addresses:** Config auto-discovery, `--config` flag, partial override semantics, unknown field detection, startup validation, clear error messages
**Avoids:** Pitfalls A1 (zero-value overwrite), A2 (silent typos), A4 (waveform string case), A7 (XDG ignored), A8 (missing explicit path), A10 (empty class name)
**Rationale:** The synthesis engine is pure PCM math with no pcap dependency. It can be built and tested in isolation with synthetic `WindowSnapshot` inputs before any real traffic flows through it. This is the architecture research's explicit recommendation. Separating synthesis from capture also means audio bugs are diagnosed without needing a live network.
### Phase 4: Classification Rule Merging
**Delivers:** A synthesizer that accepts `WindowSnapshot` inputs and produces a valid MP3 file. End-to-end smoke test: silence input → `ffprobe`-validated MP3 output. The frequency mapping table, EMA amplitude smoothing, and the LAME encoder goroutine all ship here.
**Rationale:** `classify/rules.go` must be split before any config-loading logic references the rule slice. The three-layer merge order is a design decision that, if wrong, produces silent failures with no error messages — it must be validated with unit tests before pipeline integration.
**Delivers:** `classify.SpecificRules`, `classify.CatchAllRules`, `classify.MergeRules(userRules []Rule) []Rule`; user rules prepend correctly with both catch-all and specific-rule ordering.
**Avoids:** Pitfall A5 (unreachable rules after catch-alls), Pitfall A9 (user rule shadowed by built-in specific rule for same port)
**Uses:** `sjzar/go-lame`, `go-audio/wav`, hand-rolled oscillator, `config/mapping.go` frequency table
### Phase 5: Wire Config Through Pipeline — Frequency and Waveform Overrides
**Implements:** `synth/` (oscillator, layer, mixer), `encode/` (LAME goroutine), `aggregate/` (time-window accumulator), `config/` (frequency mapping)
**Rationale:** Connects the config package to the synth layer for built-in classes only. Validates the full pipeline end-to-end before adding the complexity of user-defined classes. `encode.RunSynthesis` signature change is a breaking API change — all call sites must be updated in a single commit.
**Delivers:** `--config` cobra flag, `config.Load()` in `main.go`, merged freq config map passed to `RunSynthesis` and `NewBank`; end-to-end test: TOML sets HTTPS to 200 Hz sawtooth, bank produces 200 Hz sawtooth layer.
**Implements:** Config → synth integration
**Avoids:** LAME initialization errors (Pitfall 6), PCM overflow wrap-around (Pitfall 7), perceptual tone masking (Pitfall 8), time window jitter (Pitfall 9)
### Phase 6: User-Defined Classes End-to-End
### Phase 3: Pipeline Integration and CLI Polish
**Rationale:** The most complex integration; requires all prior phases. User-defined classes create new `TrafficClass` strings that must exist in both the merged rule set and the bank's layer map. The AllClasses() decoupling from Phase 2 makes this tractable.
**Delivers:** `[[rules]]` TOML section, dynamic `TrafficClass` values from config, bank layers constructed from merged FreqConfig map keys, class name cross-validation at config load.
**Avoids:** Pitfall A6 (user-defined class has no bank layer — nil panic or silence)
**Rationale:** Wire the Phase 1 capture/classify pipeline to the Phase 2 synthesis engine via the aggregation layer. Add Ctrl+C shutdown producing a valid MP3 (requires coordinated drain across all goroutines). Add auto-clustering of unknown traffic. Add UX features (progress output, file collision warning, graceful empty-capture error).
### Phase 7: Print-Config and UX Polish
**Delivers:** The complete v1 MVP: live capture → protocol classification + auto-clustering → time-windowed synthesis → MP3 file output. Graceful Ctrl+C with valid MP3. Full UX surface (startup interface announcement, per-window progress line, exit statistics).
**Addresses:** Auto-clustering, graceful Ctrl+C stop, capture statistics, progress feedback, output file collision warning, zero-packet guard, `main.go` pipeline wiring
**Uses:** All Phase 1 and Phase 2 components; `muesli/kmeans` or hash-bucketing for unknown traffic clustering
### Phase 4: Power User Features (v1.x)
**Rationale:** These features add value for specific user segments but have no blocking dependencies on each other — add in any order based on user feedback after the core is validated.
**Delivers:** BPF capture filter (`--filter`), offline pcap file input (`--read`), verbose protocol log (`--verbose`), configurable time window (`--window`), configurable output duration for pcap input (`--duration`)
**Addresses:** All P2 features from the prioritization matrix in FEATURES.md
**Rationale:** `--print-config` is independent of capture and must wait until all config structure is stable (Phase 6). Additive, zero regression risk.
**Delivers:** `netsynth --print-config` subcommand with commented TOML output of effective config; optional `name` field on user rules displayed in exit summary.
### Phase Ordering Rationale
- **Capture before synthesis:** The privilege and CGo pitfalls are foundational — an audio-first approach would hide them until integration and make them expensive to fix.
- **Synthesis in isolation:** Pure PCM math is independently testable. Building it against synthetic inputs before real traffic makes audio bugs fast to diagnose.
- **Integration as its own phase:** The shutdown coordination (Ctrl+C → drain → flush encoder → close file) across five goroutines is non-trivial; it deserves focused attention rather than being an afterthought of feature development.
- **Power features deferred:** BPF filter and offline pcap do not validate the core concept; they add complexity to the capture layer that should wait until the pipeline is stable.
- Phases 12 are internal refactors with no user-visible change — the right starting point for establishing patterns safely
- Phase 3 owns all config safety in one isolated package before anything is wired — retrofitting pointer-field merge after bank integration means touching multiple packages simultaneously
- Phase 4 (rule splitting) must precede Phase 6 (user rules) or catch-all ordering bugs surface silently at integration with no clear failure signal
- Phase 5 validates the full pipeline with familiar built-in classes before Phase 6 introduces the harder user-defined class problem
- Phase 7 is pure additive polish with zero risk of breaking earlier phases
### Research Flags
Phases likely needing deeper research during planning:
Phases with well-documented patterns — skip additional research:
- **Phase 1:** DSP textbook math; harmonic series are fully specified
- **Phase 2:** Standard dependency injection refactor; no unknowns
- **Phase 3:** BurntSushi/toml API is well-documented; pointer-field merge is a known TOML pattern
- **Phase 4:** Simple slice manipulation; no external dependencies
- **Phase 7:** Cobra subcommand and TOML marshal are standard patterns
- **Phase 2 (Audio Synthesis):** The perceptual frequency mapping table requires listening tests, not just code correctness. Research the auditory masking literature before finalizing `config/mapping.go`. Consider consulting the SoNSTAR PLOS One paper on time window choices.
- **Phase 3 (Auto-clustering):** The decision between simple hash-bucketing and k-means clustering (muesli/kmeans) depends on what "meaningfully distinct drone layers" means in practice. This needs a working synthesis engine to evaluate — defer the decision to Phase 3 planning.
- **Phase 4 (Offline pcap):** Time-compression of multi-hour pcap files to a fixed audio duration needs a clear algorithm decision (proportional window scaling vs. fixed window with truncation). Research this when Phase 4 is planned.
Phases with standard patterns (skip research-phase):
- **Phase 1 (Capture pipeline):** Go channel pipelines and gopacket usage are thoroughly documented. The pitfalls are known and avoidable with the guidance in PITFALLS.md.
- **Phase 3 (Pipeline integration):** Go done-channel shutdown patterns are canonical (Go Blog: Pipelines). No novel research needed.
Phases that may benefit from a targeted research pass or design review:
- **Phase 5:** `encode.RunSynthesis` signature change is a breaking API change — verify all test call sites and plan a single-commit update
- **Phase 6:** User-defined class name collision with built-in `TrafficClass` string values (e.g., user names a class `"HTTPS"`) requires a design decision: treat as override of built-in sound vs. reject as ambiguous. Not resolved in research; decide before coding Phase 6.
## Confidence Assessment
| Area | Confidence | Notes |
|------|------------|-------|
| Stack | HIGH | Core libraries confirmed via pkg.go.dev; version numbers verified; `go-audio/generator` archived status confirmed February 2026 |
| Features | MEDIUM | Niche domain with few direct CLI comparators; feature set derived from tcpdump conventions and sonification research, not user surveys |
| Architecture | MEDIUM-HIGH | Go pipeline patterns are HIGH confidence (official Go Blog); audio synthesis architecture inferred from SoNSTAR paper (MEDIUM, abstract-level access only) |
| Pitfalls | HIGH | Packet capture pitfalls verified against official gopacket issues and libpcap docs; audio pitfalls cross-referenced against DSP literature and encoder post-mortems |
| Stack | HIGH | Existing stack validated in v1.0; BurntSushi/toml v1.6.0 confirmed via pkg.go.dev and GitHub releases; `os.UserConfigDir()` XDG behavior confirmed against official Go stdlib docs |
| Features | HIGH | Config file conventions verified against XDG spec, git, golangci-lint, and mise patterns; TOML schema grounded in existing v1.0 codebase types |
| Architecture | HIGH | Based on direct code inspection of the 3,254-line v1.0 codebase; all integration points identified with specific file/line references and build order prescribed |
| Pitfalls | HIGH | TOML default-overwrite behavior verified against upstream issue trackers (BurntSushi/toml #47, go-toml #252); aliasing prevention verified against DSP literature (CCRMA paper, McGill bandlimited synthesis notes) |
**Overall confidence:** MEDIUM-HIGH
**Overall confidence:** HIGH
### Gaps to Address
- **Frequency mapping validation:** The correct frequency assignments for the protocol drone layers require subjective listening tests with real traffic. The research prescribes the *approach* (register separation, harmonic intervals) but not specific Hz values. Validate during Phase 2 with a listening session before Phase 3 integration.
- **Auto-clustering granularity:** How many unknown-traffic clusters are perceptually useful? The research suggests hash-bucketing is acceptable for v1, but does not validate how many distinct cluster tones are distinguishable simultaneously. Validate during Phase 3 with real mixed traffic.
- **muesli/kmeans maintenance status:** Last release July 2022 (LOW confidence on ongoing maintenance). If Go 1.24 compatibility issues emerge, the alternative is `mpraski/clusters` (online clustering) or a hand-rolled hash bucketer. Plan for substitution.
- **macOS privilege model:** CAP_NET_RAW pitfall was verified for Linux. macOS uses a different privilege model (BPF device permissions). If macOS is a target, verify the privilege flow and error messages during Phase 1.
- **User class name collision with built-in class strings:** If a user writes `class = "HTTPS"` in a `[[rules]]` block, the intent could be "override built-in sound" or "create a parallel custom class." The merge logic needs an explicit decision before Phase 6: treat matching names as overrides (simplest) or require a separate TOML section. This is a UX design question — resolve before coding Phase 6.
- **BurntSushi/toml vs. go-toml v2:** Both work for this use case. BurntSushi is recommended for ergonomics, but if the team prefers go-toml v2's `DisallowUnknownFields()` pattern, it is equally valid with minor API differences. Either choice is fine; just make one and be consistent.
## Sources
### Primary (HIGH confidence)
- `github.com/gopacket/gopacket` releases — v1.5.0 November 2025, Go 1.24+ confirmed
- `pkg.go.dev/github.com/packetcap/go-pcap` — pure Go, Linux/macOS confirmed
- `pkg.go.dev/github.com/sjzar/go-lame` — v0.0.9 April 2025, embedded C source confirmed
- `pkg.go.dev/github.com/spf13/cobra` — v1.10.2 December 2025
- Go Blog: Pipelines and cancellation — canonical Go pipeline pattern
- `google/gopacket` issue #1016 — unmaintained status confirmed
- `google/gopacket` issue #329 — 98% packet drop under high traffic, afpacket solution
- linuxvox.com: CAP_NET_RAW + nosuid filesystem behavior
- Direct code inspection: `synth/config.go`, `synth/oscillator.go`, `synth/bank.go`, `synth/layer.go`, `classify/classifier.go`, `classify/rules.go`, `classify/types.go`, `encode/mp3.go`, `cmd/netsynth/main.go`
- `pkg.go.dev/github.com/BurntSushi/toml` — v1.6.0 API, `Undecoded()` strict mode, pointer field behavior
- `github.com/BurntSushi/toml/issues/47` — default-overwrite behavior when using `Unmarshal` confirmed
- `pkg.go.dev/os#UserConfigDir` — XDG_CONFIG_HOME behavior on Linux confirmed via official Go stdlib docs
- `ccrma.stanford.edu/~stilti/papers/blit.pdf` — bandlimited synthesis theory (alias-free waveforms)
- `music.mcgill.ca/~gary/307/week5/bandlimited.html` — truncated harmonic series approach confirmed
### Secondary (MEDIUM confidence)
- SoNSTAR PLOS One paper (arXiv 1712.07029) — time window design and sonification architecture
- braheezy.github.io: Go MP3 encoding options — shine-mp3 not production-grade
- Dylan Meeus: Audio From Scratch With Go — PCM synthesis patterns
- `github.com/go-audio/generator` — archived February 2026, do not use (confirmed read-only)
- XDG Base Directory Specification — config discovery precedence order
- `mise.jdx.dev/configuration.html` — working-dir + XDG config discovery pattern
- `golangci-lint.run/docs/configuration/cli/` — partial override config in Go CLI tools
- `dylanmeeus.github.io/posts/audio-from-scratch-pt8/` — Go waveform synthesis from scratch, confirms no library needed
- `pkg.go.dev/github.com/pelletier/go-toml/v2` — v2.3.0 strict decoder comparison; `pelletier/go-toml/issues/252` partial v2 resolution of default-overwrite
### Tertiary (LOW confidence)
- `muesli/kmeans` v0.3.1 (July 2022) — last release date only; ongoing Go 1.24 compatibility unverified
- Drone auralization model, Acta Acustica 2024 — amplitude/frequency modulation patterns (MEDIUM, used for perceptual guidance)
- Competitor feature table (SoNSTAR, Network-Sonification, Peep) — niche domain, limited documentation; used for context only, not binding decisions
---
*Research completed: 2026-03-24*
*Research completed: 2026-03-26*
*Ready for roadmap: yes*