docs: complete project research
This commit is contained in:
+391
-213
@@ -1,333 +1,511 @@
|
||||
# Pitfalls Research
|
||||
|
||||
**Domain:** Network-traffic-to-audio synthesis CLI tool (Go)
|
||||
**Researched:** 2026-03-24
|
||||
**Confidence:** HIGH (packet capture / CGo pitfalls verified against official issues and docs; audio synthesis pitfalls cross-referenced against encoder project post-mortems and DSP literature)
|
||||
**Researched:** 2026-03-26 (v1.1 update — TOML config, waveform types, user-defined rules)
|
||||
**Confidence:** HIGH (TOML decoder behaviors verified against pkg.go.dev official docs and issue trackers; audio synthesis aliasing verified against DSP literature; config merging verified against BurntSushi/toml issue #47 and go-toml issue #252)
|
||||
|
||||
---
|
||||
|
||||
## Critical Pitfalls
|
||||
## v1.1 Milestone Pitfalls (New)
|
||||
|
||||
### Pitfall 1: Using `google/gopacket` Instead of the Active Community Fork
|
||||
These pitfalls are specific to adding TOML config, waveform types, and user-defined classification rules to the existing NetSynth codebase.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall A1: TOML Unmarshal Silently Overwrites Pre-filled Defaults with Zero Values
|
||||
|
||||
**What goes wrong:**
|
||||
The original `github.com/google/gopacket` repository is unmaintained. Bugs go unpatched, open PRs accumulate, and compatibility with newer Go versions degrades. Projects that import it are pinned to a stale library.
|
||||
You initialize a `Config` struct with built-in defaults, then call `toml.Unmarshal` to layer in user overrides. Any field the user *omits* from their TOML file is set to its Go zero value (`0`, `""`, `false`, `nil`) by the decoder — overwriting your defaults. A user who writes only `[sounds.DNS]` in their config file to change the DNS tone ends up wiping every other class back to zero Hz.
|
||||
|
||||
**Why it happens:**
|
||||
`google/gopacket` has enormous search mindshare and most tutorials still reference it. Developers reach for the first result without checking maintenance status.
|
||||
Both `BurntSushi/toml` and `pelletier/go-toml` v1 do not distinguish between "key was absent" and "key was explicitly set to zero". The decoder reflects over the struct and writes zero for every absent key. This was explicitly reported as a bug in BurntSushi/toml issue #47 and go-toml issue #252. go-toml v2 partially addresses it but still zeros primitive-type fields that are absent.
|
||||
|
||||
**How to avoid:**
|
||||
Import `github.com/gopacket/gopacket` (the community fork, v1.5.0 released November 2025, requires Go 1.24+). Major projects including Cilium have already migrated. Treat `google/gopacket` as deprecated.
|
||||
**Consequences:**
|
||||
- All non-overridden traffic classes play silence (0 Hz oscillator)
|
||||
- Classification rules get zeroed if user only partially fills `[[rules]]`
|
||||
- EMA tau, whisper floor, gain, and other synth parameters reset to 0
|
||||
|
||||
**Warning signs:**
|
||||
- `go.mod` referencing `github.com/google/gopacket`
|
||||
- Build errors on Go 1.21+ not fixed upstream
|
||||
**Prevention:**
|
||||
Use pointer fields (`*float64`, `*string`) in the decoded struct to distinguish "not provided" (nil pointer) from "explicitly set to zero" (non-nil pointer to 0). Apply a merge step: iterate over the decoded struct, and for each pointer field that is nil, keep the built-in default. For slice fields (like `[]RuleConfig`), nil slice means "user did not provide rules" — preserve defaults; non-nil empty slice (`[]RuleConfig{}`) means "user explicitly cleared rules" — respect that.
|
||||
|
||||
```go
|
||||
// In config struct, use pointers for optional overrides:
|
||||
type SoundConfig struct {
|
||||
FreqHz *float64 `toml:"freq_hz"`
|
||||
Waveform *string `toml:"waveform"`
|
||||
}
|
||||
|
||||
// Merge: for each class, override only non-nil fields
|
||||
func mergeSound(base synth.FreqConfig, override SoundConfig) synth.FreqConfig {
|
||||
if override.FreqHz != nil {
|
||||
base.BaseHz = *override.FreqHz
|
||||
}
|
||||
if override.Waveform != nil {
|
||||
base.WaveformType = *override.Waveform
|
||||
}
|
||||
return base
|
||||
}
|
||||
```
|
||||
|
||||
**Detection:**
|
||||
- User reports that classes they did not configure now produce no sound
|
||||
- Unit test: load a config that overrides only one class; verify all other classes retain built-in Hz values
|
||||
|
||||
**Phase to address:**
|
||||
Phase 1 (packet capture scaffolding) — set the correct import path from day one; migrating later is a find-and-replace across the whole codebase.
|
||||
Config loading phase (first phase of v1.1). Get the pointer-and-merge pattern established before wiring config into the bank. Retrofitting after the bank construction is wired is a significant churn.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 2: CGo Destroys the "Single Binary" Promise
|
||||
### Pitfall A2: BurntSushi/toml Silently Ignores Typos in Field Names
|
||||
|
||||
**What goes wrong:**
|
||||
`gopacket/pcap` requires `libpcap` via CGo. By default Go produces a dynamically linked binary. On a target machine without `libpcap.so` installed, the binary silently or loudly fails with `error while loading shared libraries: libpcap.so.0.8`. The "just copy the binary" distribution story breaks completely.
|
||||
A user writes `freq_hz = 440` but the struct tag is `toml:"freq_hz"` — this works. However if the user writes `freqhz = 440` or `FreqHz = 440` or a misspelled `frek_hz = 440`, the library silently ignores the key. The user's override is never applied. No error is returned. The user thinks their config is active; it is not.
|
||||
|
||||
**Why it happens:**
|
||||
CGo is enabled by default and Go gives no compile-time warning that the resulting binary has a runtime C dependency. The binary runs perfectly on the build machine (which has libpcap-dev installed) and fails on clean machines.
|
||||
`BurntSushi/toml` by default silently discards keys that do not map to any struct field. This is the documented default behavior ("will ignore options in the TOML file that you don't use"). It is the opposite of "strict mode."
|
||||
|
||||
**How to avoid:**
|
||||
Choose one of these strategies before writing a line of capture code:
|
||||
1. **Fully static build**: `CGO_ENABLED=1 go build -ldflags "-linkmode 'external' -extldflags '-static'"` with `libpcap.a` present. Requires `musl-gcc` or equivalent on Alpine/musl.
|
||||
2. **pcapgo (pure Go)**: `gopacket/pcapgo` provides an `EthernetHandle` that avoids CGo entirely — lower performance but zero C dependency. Sufficient for ambient audio capture at non-Gbps rates.
|
||||
3. **Document the dependency explicitly**: If CGo/dynamic linking is accepted, `README` must state "requires `libpcap` (`apt install libpcap-dev` / `brew install libpcap`)".
|
||||
**Consequences:**
|
||||
- Silent misconfiguration: user's customization is invisible
|
||||
- Debugging is very hard — no error to trace back to the TOML file
|
||||
|
||||
**Warning signs:**
|
||||
- `CGO_ENABLED` not explicitly set in your build script
|
||||
- `ldd ./netsynth` shows `libpcap.so` as a dependency
|
||||
- No CI test on a minimal (Alpine, scratch Docker) container
|
||||
**Prevention:**
|
||||
Use `toml.Decode` (not `toml.Unmarshal`) to obtain `MetaData`, then call `md.Undecoded()` and return an error listing any keys that were not decoded. This is BurntSushi's documented strict-mode pattern.
|
||||
|
||||
```go
|
||||
md, err := toml.Decode(string(data), &cfg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if keys := md.Undecoded(); len(keys) > 0 {
|
||||
return fmt.Errorf("unknown config keys (check for typos): %v", keys)
|
||||
}
|
||||
```
|
||||
|
||||
**Detection:**
|
||||
- Config change that should audibly alter the sound has no effect
|
||||
- Undecoded keys present but no warning/error logged
|
||||
|
||||
**Phase to address:**
|
||||
Phase 1 — this is a foundational architecture decision. Changing from dynamic to static after the fact is painful and causes build pipeline rewrites.
|
||||
Config loading phase. Implement strict decoding from the first config load function. Do not add this as an afterthought — it is the primary mechanism protecting users from silent misconfiguration.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 3: `CAP_NET_RAW` + Binary Location = Silent Failure on Linux
|
||||
### Pitfall A3: Naive Square/Sawtooth/Triangle Generation Produces Audible Aliasing Distortion
|
||||
|
||||
**What goes wrong:**
|
||||
On Ubuntu and many Linux distributions, `setcap cap_net_raw+eip ./netsynth` appears to succeed but the binary fails at runtime if it lives in `/home/user/bin`, `/tmp`, or any filesystem mounted `nosuid`. The kernel silently ignores the capability. AppArmor compounds this by enforcing path-based restrictions.
|
||||
Implementing waveforms by direct time-domain math — `sign(sin(phase))` for square, `2*frac(phase)-1` for sawtooth, `1-2*abs(frac(phase)-0.5)` for triangle — produces a waveform with infinite harmonics. At 44100 Hz, harmonics above 22050 Hz fold back into the audible range as aliasing. At the frequencies used in NetSynth (65–1047 Hz), aliasing from a naive square wave produces a buzzing distortion that is especially audible at higher drone frequencies and sounds like corruption rather than timbre.
|
||||
|
||||
**Why it happens:**
|
||||
Developers test from their build directory (`~/projects/netsynth/`) — a path frequently on a `nosuid` filesystem. The tool appears broken with no clear error message beyond "permission denied" or "you must be root."
|
||||
The mathematical waveforms are not bandlimited — they have infinite harmonic content. Direct sampling them at 44100 Hz aliases all energy above Nyquist back into the audible band. Developers who test at low frequencies (60–120 Hz) may not notice because the aliased harmonics land at very high frequencies with low perceptual impact; the problem worsens significantly above 400 Hz where aliases fold into the 1–5 kHz perceptually prominent range.
|
||||
|
||||
**How to avoid:**
|
||||
- Install to `/usr/local/bin` or `/usr/bin` for capability-based operation
|
||||
- Document two run modes: `sudo ./netsynth` (always works) vs. `setcap` (requires standard path)
|
||||
- In the CLI, detect permission failure and emit a clear message: "Packet capture requires root or CAP_NET_RAW. Run as root or: sudo setcap cap_net_raw+eip $(which netsynth)"
|
||||
- Test capability-mode explicitly from a non-home path in CI
|
||||
**Consequences:**
|
||||
- Square/sawtooth at SSH (330 Hz) and higher frequencies sounds harsh and buzzy
|
||||
- The effect worsens at higher frequencies, making SMTP (440 Hz) and DHCP (600 Hz) drones sound distorted
|
||||
- Aliasing cannot be filtered out post-synthesis (it is interleaved with desired signal)
|
||||
|
||||
**Warning signs:**
|
||||
- Testing only via `sudo go run .`
|
||||
- No test of the installed-binary path in README instructions
|
||||
- macOS-only development (macOS uses a different privilege model; Linux `nosuid` behavior won't surface)
|
||||
**Prevention:**
|
||||
Use additive synthesis — the approach already in use for sine waves in `oscillator.go`. The existing `Oscillator.Advance(harmonics []HarmonicDef)` computes `sin(2π * phase * ratio)` for each partial. Square, sawtooth, and triangle waveforms are all expressible as harmonic series:
|
||||
|
||||
- **Square:** odd harmonics only, amplitude `1/k` for harmonic `k`: ratios 1, 3, 5, 7, ... with amplitudes 1.0, 0.33, 0.20, 0.14, ... Truncate at Nyquist.
|
||||
- **Sawtooth:** all harmonics, amplitude `1/k`: ratios 1, 2, 3, 4, ... with amplitudes 1.0, 0.5, 0.33, 0.25, ... Truncate at Nyquist.
|
||||
- **Triangle:** odd harmonics, amplitude `1/k²`, alternating sign: ratios 1, 3, 5, ... with amplitudes 1.0, 0.11, 0.04, ... Truncate at Nyquist.
|
||||
|
||||
The truncation (only sum harmonics where `freq * ratio < sampleRate / 2`) is the critical step that makes the synthesis bandlimited. The existing `[]HarmonicDef` structure in `synth/config.go` already supports this — waveform type selection just requires generating the right harmonic series for each `FreqConfig`.
|
||||
|
||||
Waveform presets should be pre-computed `[]HarmonicDef` slices, not runtime computation of naive waveform math:
|
||||
|
||||
```go
|
||||
// BandlimitedHarmonics returns a bandlimited harmonic series for the given waveform type.
|
||||
// It truncates harmonics at Nyquist (sampleRate/2) to prevent aliasing.
|
||||
func BandlimitedHarmonics(waveform string, baseHz float64, sampleRate int) []HarmonicDef {
|
||||
nyquist := float64(sampleRate) / 2.0
|
||||
var defs []HarmonicDef
|
||||
switch waveform {
|
||||
case "square":
|
||||
for k := 1; float64(k)*baseHz < nyquist; k += 2 { // odd only
|
||||
defs = append(defs, HarmonicDef{Ratio: k, Amplitude: 1.0 / float64(k)})
|
||||
}
|
||||
case "sawtooth":
|
||||
for k := 1; float64(k)*baseHz < nyquist; k++ {
|
||||
defs = append(defs, HarmonicDef{Ratio: k, Amplitude: 1.0 / float64(k)})
|
||||
}
|
||||
case "triangle":
|
||||
sign := 1.0
|
||||
for k := 1; float64(k)*baseHz < nyquist; k += 2 {
|
||||
defs = append(defs, HarmonicDef{Ratio: k, Amplitude: sign / float64(k*k)})
|
||||
sign = -sign
|
||||
}
|
||||
default: // "sine"
|
||||
defs = []HarmonicDef{{Ratio: 1, Amplitude: 1.0}}
|
||||
}
|
||||
return defs
|
||||
}
|
||||
```
|
||||
|
||||
**Detection:**
|
||||
- Audible buzzing or grainy texture on drone layers above 300 Hz with non-sine waveforms
|
||||
- Square/sawtooth waveforms sound harsher than expected at high frequencies
|
||||
|
||||
**Phase to address:**
|
||||
Phase 1 (capture scaffolding) and the CLI UX phase — the error message is user-facing and needs to be explicit.
|
||||
Waveform type implementation phase. The design decision (additive synthesis, not direct waveform math) must be made before coding waveform support. Switching from direct math to additive after the fact requires rewriting the oscillator API.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 4: Packet Buffer Overflow Under Moderate Traffic Load
|
||||
### Pitfall A4: Waveform String Validation Fails Silently, Falls Back to Silence
|
||||
|
||||
**What goes wrong:**
|
||||
At high packet rates (busy LAN, server NIC), gopacket's kernel ring buffer fills faster than the processing goroutine consumes it. The OS drops packets silently. The tool appears to work, but 50-98% of packets never reach the classifier. The audio output misrepresents actual traffic.
|
||||
A user writes `waveform = "Sawtooth"` (capital S) or `waveform = "saw"` (abbreviation). The config loading code does a simple equality check (`if waveform == "sawtooth"`), finds no match, and either panics, silently emits silence, or applies a default without telling the user. In all cases the user's intent is invisible.
|
||||
|
||||
**Why it happens:**
|
||||
The default pcap buffer is 1-2 MB. Each packet triggers a cgo call (with `pcap` backend), creating per-packet overhead that compounds at speed. Developers test on quiet home networks and never observe drops.
|
||||
String-based enumerations in config files have no compile-time type checking. Case sensitivity and abbreviations are user expectations that must be explicitly handled.
|
||||
|
||||
**How to avoid:**
|
||||
- Set a large capture buffer explicitly: `handle.SetBufferSize(32 * 1024 * 1024)` (32 MB)
|
||||
- Use a non-blocking channel between capture and classification goroutines with a buffer of at least 1000 packets; drop metrics count drops so they are visible
|
||||
- For high-throughput scenarios, prefer `afpacket` backend over `pcap` — `afpacket` eliminates per-packet CGo calls and dramatically improves throughput (benchmark: 1.27 MB/s → 21.17 MB/s)
|
||||
- NetSynth's ambient audio goal tolerates lossy capture — document this explicitly so users understand the tool provides a statistical fingerprint, not a perfect census
|
||||
**Consequences:**
|
||||
- Silent misconfiguration: wrong waveform with no feedback
|
||||
- Hard to debug: config appears valid, sound is just wrong
|
||||
|
||||
**Warning signs:**
|
||||
- Capture and classification in a single goroutine
|
||||
- No `SetBufferSize` call
|
||||
- Testing only on loopback (`lo`) which has near-zero real packet rates
|
||||
**Prevention:**
|
||||
Normalize waveform strings at parse time (`strings.ToLower`, `strings.TrimSpace`), validate against the accepted set, and return an explicit error with the accepted values if the string is unrecognized:
|
||||
|
||||
```go
|
||||
var validWaveforms = map[string]struct{}{
|
||||
"sine": {}, "square": {}, "sawtooth": {}, "triangle": {},
|
||||
}
|
||||
func validateWaveform(s string) (string, error) {
|
||||
normalized := strings.ToLower(strings.TrimSpace(s))
|
||||
if _, ok := validWaveforms[normalized]; !ok {
|
||||
return "", fmt.Errorf("unknown waveform %q: must be one of sine, square, sawtooth, triangle", s)
|
||||
}
|
||||
return normalized, nil
|
||||
}
|
||||
```
|
||||
|
||||
**Phase to address:**
|
||||
Phase 1/2 (capture pipeline) — the goroutine architecture must be designed for async processing from the start. Retrofitting is a significant rewrite.
|
||||
Config validation step (same phase as config loading). Implement all string field validation in a single `validate(cfg Config) error` function called immediately after decoding.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 5: ZeroCopy Packet Data Use-After-Free
|
||||
### Pitfall A5: User Rules Appended After Catch-All Rules Are Unreachable
|
||||
|
||||
**What goes wrong:**
|
||||
`ZeroCopyReadPacketData()` returns a slice pointing into a buffer owned by the pcap handle. The next call to `ZeroCopyReadPacketData()` invalidates the previous slice's backing memory. If any goroutine holds a reference to old packet bytes and reads them after the next call, it reads corrupted or incorrect data. This produces silent data corruption — wrong protocol classifications, no crash.
|
||||
The existing `DefaultRules` slice ends with two catch-alls:
|
||||
|
||||
```go
|
||||
{Protocol: "tcp", DstPort: 0, Class: ClassOtherTCP},
|
||||
{Protocol: "udp", DstPort: 0, Class: ClassOtherUDP},
|
||||
```
|
||||
|
||||
If user-defined rules are simply appended to this slice (`append(DefaultRules, userRules...)`), the catch-alls match first (DstPort=0 matches any port for that protocol), and the user's rules are unreachable. Every custom rule maps to `ClassOtherTCP` or `ClassOtherUDP` instead. The user gets no sound from their custom class.
|
||||
|
||||
**Why it happens:**
|
||||
The zero-copy API looks identical to the copying API. Developers reach for it for performance without reading the "each call invalidates previous data" contract.
|
||||
The first-match-wins semantics of `Classifier.Classify()` mean ordering is semantically critical. `DefaultRules` is a named var that exists precisely as an ordered slice — the comment `// Catch-alls (must be last)` documents this constraint. But "must be last in the defaults" does not automatically mean "must be last in the final merged slice." Developers who concatenate slices without thinking about this invariant break the system.
|
||||
|
||||
**How to avoid:**
|
||||
Use `ReadPacketData()` (copies data) unless you have profiling evidence that allocation is a bottleneck. If `ZeroCopyReadPacketData()` is used, never pass the slice to another goroutine without first copying it: `data := append([]byte(nil), raw...)`.
|
||||
**Consequences:**
|
||||
- All user-defined rules are silently swallowed by catch-alls
|
||||
- User's custom class never activates
|
||||
- No error — the pipeline works, just wrong
|
||||
|
||||
**Warning signs:**
|
||||
- `ZeroCopyReadPacketData` in a goroutine-per-packet pattern
|
||||
- Intermittent wrong protocol classifications that are not reproducible
|
||||
- Using `gopacket.Lazy` decode with concurrent goroutines (the gopacket docs explicitly warn against this combination)
|
||||
**Prevention:**
|
||||
Always insert user rules *before* catch-all rules. The merge strategy must be: `specificDefaultRules + userRules + catchAllRules`. Implement this with an explicit split in the default rule set:
|
||||
|
||||
```go
|
||||
// In classify/rules.go, split into two exported slices:
|
||||
var SpecificRules = []Rule{ /* ICMP through DHCP */ }
|
||||
var CatchAllRules = []Rule{
|
||||
{Protocol: "tcp", DstPort: 0, Class: ClassOtherTCP},
|
||||
{Protocol: "udp", DstPort: 0, Class: ClassOtherUDP},
|
||||
}
|
||||
|
||||
// Merge function used by config loading:
|
||||
func MergeRules(userRules []Rule) []Rule {
|
||||
result := make([]Rule, 0, len(SpecificRules)+len(userRules)+len(CatchAllRules))
|
||||
result = append(result, SpecificRules...)
|
||||
result = append(result, userRules...)
|
||||
result = append(result, CatchAllRules...)
|
||||
return result
|
||||
}
|
||||
```
|
||||
|
||||
Alternatively, annotate each default rule with a `CatchAll bool` field and sort before use. The split-slice approach is simpler and more explicit.
|
||||
|
||||
**Detection:**
|
||||
- User-defined rule that should match traffic does not produce its custom sound
|
||||
- `--verbose` output shows traffic being classified as `OtherTCP`/`OtherUDP` instead of the custom class
|
||||
- Test: write a rule for port 8080, send HTTP traffic to port 8080, verify it hits the custom class and not `ClassOtherTCP`
|
||||
|
||||
**Phase to address:**
|
||||
Phase 1 (capture/decode) — establish the correct API choice at the read loop level.
|
||||
User-defined rules phase. The `classify/rules.go` split must be the first code change before any config loading logic references the rule slice.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 6: MP3 Output Is Corrupt or Unplayable Due to LAME Initialization Errors
|
||||
### Pitfall A6: User-Defined Classes Have No synth.FreqConfig Entry — Bank Panics or Plays Silence
|
||||
|
||||
**What goes wrong:**
|
||||
MP3 encoding via CGo LAME bindings requires calling `InitParams()` after setting all encoder parameters. Skipping or reordering this call produces a file with a valid `.mp3` extension that most players refuse to open or that plays as noise. The encoder returns no error from the encode calls themselves.
|
||||
`OscillatorBank.NewBank()` iterates over `classify.AllClasses()` and looks up each class in `ClassFreqConfigs`. A user-defined rule creates a new `TrafficClass` (e.g., `"my-api"`). This class is not in `AllClasses()`, so the bank has no layer for it. The aggregator increments a count for `"my-api"`, `RenderWindow` looks up `b.layers["my-api"]`, gets nil, and either panics (nil pointer dereference on `layer.AdvanceSample()`) or silently contributes nothing to the mix.
|
||||
|
||||
**Why it happens:**
|
||||
The LAME C API is stateful and order-dependent. Go wrappers vary in how much they enforce initialization order. Many tutorial examples show minimal code that happens to work for 44100 Hz stereo but silently breaks for other configurations.
|
||||
`classify.AllClasses()` is a hardcoded list of the 14 built-in classes. The synth bank is constructed once at startup from this static list. User-defined classes are a runtime extension that the bank knows nothing about.
|
||||
|
||||
**How to avoid:**
|
||||
- Always call `InitParams()` before writing any frames
|
||||
- Restrict to known-safe parameters: sample rate 44100 or 48000, stereo or mono (LAME does not support dual-channel mode)
|
||||
- Write a single integration test that encodes 1 second of silence and confirms the output file is valid (use `mp3val` or `ffprobe` in CI)
|
||||
- Consider `shine-mp3` (pure Go port) as an alternative that eliminates CGo entirely; output files are larger but the library has no C initialization state
|
||||
**Consequences:**
|
||||
- Nil pointer panic in `RenderWindow` if the layer map lookup is not nil-guarded
|
||||
- Or silent: user-defined class traffic is captured and aggregated but never rendered to audio
|
||||
- In either case the user's primary feature request (custom sounds for custom classes) silently fails
|
||||
|
||||
**Warning signs:**
|
||||
- No test that validates the output MP3 with an external tool
|
||||
- Sample rate set to anything other than 44100 or 48000
|
||||
- Encoder parameters set after `InitParams()` has been called
|
||||
**Prevention:**
|
||||
The bank must be constructed from the *full* set of active classes, including user-defined ones. The construction path should be:
|
||||
|
||||
1. Load config (parse TOML, validate)
|
||||
2. Compute effective rule set (built-in + user rules)
|
||||
3. Extract the complete set of `TrafficClass` values referenced by all rules
|
||||
4. Pass this full class set to `NewBank` (or equivalent) so a layer is created for every reachable class
|
||||
5. Wire user-defined class frequencies from config into the bank
|
||||
|
||||
`AllClasses()` in `classify/types.go` should either remain the static built-in list (used for display/iteration of built-ins) or be replaced by a dynamic function that takes the active rule set as input. Do not rely on the hardcoded list in the bank-construction path when user-defined classes are possible.
|
||||
|
||||
**Detection:**
|
||||
- Panic: `runtime error: invalid memory address or nil pointer dereference` in `synth/bank.go:RenderWindow`
|
||||
- Or: user-defined class produces no sound, no error
|
||||
- Test: create a config with one user rule using a custom class; verify the bank is built with a layer for that class and that layer produces sound
|
||||
|
||||
**Phase to address:**
|
||||
Audio synthesis / encoding phase — establish the encode pipeline with an end-to-end smoke test (silence → valid MP3) before wiring up synthesis.
|
||||
User-defined rules phase, specifically the bank initialization step. This is the deepest integration point — it touches the pipeline at capture → classify → aggregate → synthesize.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 7: PCM Sample Overflow Produces Wrap-Around Distortion
|
||||
### Pitfall A7: Config Auto-Discovery Follows Wrong Order or Ignores XDG Variables
|
||||
|
||||
**What goes wrong:**
|
||||
Synthesizing audio as `int16` samples and summing multiple sine layers without clamping causes integer overflow. The value wraps around (e.g., 32767 + 100 = -32667 in int16), producing a sharp click or a buzzing distortion that corrupts the ambient soundscape. This is not clipping — it is a distinctly worse artifact.
|
||||
The spec calls for auto-discovery from `./netsynth.toml` then `~/.config/netsynth/config.toml`. A naive implementation uses `os.UserHomeDir()` to build the fallback path. On systems where `$XDG_CONFIG_HOME` is set to a non-default location (common on NixOS, custom dotfile managers, CI environments), the tool ignores the user's configured config directory and looks in `~/.config` anyway. The user has a config at `$XDG_CONFIG_HOME/netsynth/config.toml` that is never found.
|
||||
|
||||
Additionally, `os.UserHomeDir()` returns an error if `$HOME` is unset (e.g., inside some Docker containers or cron jobs). If this error is not handled, the path construction silently produces `"/.config/netsynth/config.toml"` (an absolute path starting with `/.config`) rather than failing with a useful message.
|
||||
|
||||
**Why it happens:**
|
||||
Developers model audio math in their head as real-valued floats, implement it in int16 for "efficiency," and forget that Go integer overflow is undefined-behavior-free but still wraps. With 6-8 drone layers simultaneously active, summing them easily exceeds ±32767.
|
||||
Go's `os.UserConfigDir()` already implements the XDG lookup (`$XDG_CONFIG_HOME` → `~/.config` on Linux, `~/Library/Application Support` on macOS). Most developers reach for `os.UserHomeDir()` + hardcoded `".config"` string because it is the first function they find in the stdlib.
|
||||
|
||||
**How to avoid:**
|
||||
Synthesize internally in `float64` in the range `[-1.0, 1.0]`. Apply a normalisation/soft-limiter pass before converting to `int16` for encoding. Clamp before cast: `sample := int16(math.Max(-1.0, math.Min(1.0, floatSample)) * 32767)`. Never do mixed-type audio math that passes through int16 as an intermediate.
|
||||
**Consequences:**
|
||||
- User's config is silently ignored when `$XDG_CONFIG_HOME` is non-default
|
||||
- Confusing behavior difference between development machines and CI
|
||||
|
||||
**Warning signs:**
|
||||
- Audio synthesis structs storing amplitude as `int16` or `int32`
|
||||
- Adding layer outputs with `+=` without a final normalisation step
|
||||
- Distorted output that correlates with traffic spikes (more active layers = more overflow)
|
||||
**Prevention:**
|
||||
Use `os.UserConfigDir()` (stdlib, Go 1.13+) for the platform-appropriate config directory. This correctly respects `$XDG_CONFIG_HOME` on Linux and `APPDATA` on Windows (if ever relevant). The discovery order should be:
|
||||
|
||||
```go
|
||||
func configSearchPaths() []string {
|
||||
var paths []string
|
||||
// 1. Current directory (highest precedence)
|
||||
paths = append(paths, "netsynth.toml")
|
||||
// 2. XDG/platform config dir
|
||||
if cfgDir, err := os.UserConfigDir(); err == nil {
|
||||
paths = append(paths, filepath.Join(cfgDir, "netsynth", "config.toml"))
|
||||
}
|
||||
return paths
|
||||
}
|
||||
```
|
||||
|
||||
If `--config` flag is set, use that path exclusively and return a clear error if the file is absent (do not fall through to auto-discovery when explicit path is provided).
|
||||
|
||||
**Detection:**
|
||||
- Config not loaded on systems where `$XDG_CONFIG_HOME=/custom/path`
|
||||
- Silent "no config found" behavior when a config clearly exists at the XDG path
|
||||
|
||||
**Phase to address:**
|
||||
Audio synthesis phase — establish the internal sample representation as `float64` from the start.
|
||||
Config loading phase. Implement the path discovery with `os.UserConfigDir()` from the start. Fix before the feature ships.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 8: Tone-per-Protocol Mapping Produces Perceptual Chaos
|
||||
### Pitfall A8: Explicit --config Flag Does Not Error on Missing File
|
||||
|
||||
**What goes wrong:**
|
||||
Assigning arbitrary frequencies to protocols (e.g., DNS=440 Hz, HTTPS=880 Hz, ICMP=1320 Hz, SSH=1760 Hz, 6 auto-clusters=random) creates a soundscape where all tones are in the same frequency range, fighting each other. At moderate traffic the result is an undifferentiated buzz rather than distinct recognizable layers.
|
||||
When `--config path/to/file.toml` is specified, the user expects an error if the file does not exist. If the config loader falls through to auto-discovery when the explicit path is missing, or silently uses defaults, the user has no way to detect a typo in their `--config` argument. They run a session, get "unexpected" default sounds, and have no indication their config was never loaded.
|
||||
|
||||
**Why it happens:**
|
||||
Developers choose frequencies programmatically (e.g., multiples of a base frequency) without considering auditory scene analysis — the human perceptual process by which listeners separate simultaneous sounds into distinct streams. Sounds too close in frequency mask each other.
|
||||
Auto-discovery logic is convenient to write as "try these paths, use first found." Developers reuse this logic even for the `--config` code path.
|
||||
|
||||
**How to avoid:**
|
||||
Space protocol tones across register bands: low drones (80-200 Hz) for high-volume background traffic (HTTPS bulk), mid tones (300-600 Hz) for control traffic (DNS, NTP), high tones (800-1600 Hz) for interactive protocols (SSH, ICMP). Use harmonic or musical intervals (octaves, fifths) rather than arithmetic spacing. Keep auto-cluster frequencies in the 200-500 Hz mid-range so they don't obscure the "signature" tones. Limit simultaneous active layers to avoid masking.
|
||||
**Prevention:**
|
||||
Separate the two code paths:
|
||||
- `--config` specified → `os.Open(flagValue)`, return error immediately if `errors.Is(err, os.ErrNotExist)`
|
||||
- No flag → `configSearchPaths()` loop, silently skip missing files, proceed with defaults if none found
|
||||
|
||||
**Warning signs:**
|
||||
- Frequency assignments as an arithmetic sequence: `baseFreq + n*200`
|
||||
- No perceptual test — only waveform-level correctness checks
|
||||
- Auto-cluster frequencies chosen randomly from the full audible range
|
||||
**Detection:**
|
||||
- `--config missing.toml` runs without error, uses defaults
|
||||
- User misses that their config file path has a typo
|
||||
|
||||
**Phase to address:**
|
||||
Audio mapping / synthesis phase — the frequency mapping table should be designed up front with the perceptual goals in mind, not patched after "it sounds like noise" feedback.
|
||||
Config loading phase. A one-line `if flagValue != "" { /* require it */ }` branch is sufficient.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 9: Time Window Too Short — Unstable, Jittery Audio
|
||||
### Pitfall A9: User Rules That Target the Same Port as Built-in Rules Are Silently Shadowed
|
||||
|
||||
**What goes wrong:**
|
||||
Aggregating traffic into windows shorter than ~500ms causes rapid amplitude oscillation in the synthesized drones. A single ICMP ping becomes a brief tone burst; a DNS query causes a momentary volume spike. The output sounds jittery and event-driven rather than ambient.
|
||||
A user writes a rule for `{Protocol: "tcp", DstPort: 443, Class: "my-api"}` intending to reclassify their internal HTTPS traffic. If built-in `ClassHTTPS` still appears before the user rule in the merged slice, the built-in rule wins every time. The user's intent ("I want my port-443 traffic to sound different") is silently defeated.
|
||||
|
||||
**Why it happens:**
|
||||
Developers choose a "natural" update interval (100ms or 200ms matches CPU scheduling intuition) without considering audio envelope times. Human perception of tonal stability requires note durations of at least 200-500ms; drones need even longer.
|
||||
First-match-wins with `SpecificRules + userRules + CatchAllRules` means built-in specific rules still precede user rules. A user trying to *override* a built-in mapping must replace it, not add after it.
|
||||
|
||||
**How to avoid:**
|
||||
- Use a minimum window of 500ms for amplitude updates; 1-2s for tonal shifts
|
||||
- Apply amplitude smoothing (exponential moving average with a decay of ~2-5s) so a single-packet burst doesn't cause an immediate amplitude jump
|
||||
- Separate the "data collection" window (can be shorter) from the "audio parameter update" window (should be longer)
|
||||
**Consequences:**
|
||||
- User's specific rule is unreachable if a built-in rule covers the same port/protocol
|
||||
- No error, no warning
|
||||
- Functionally the same as Pitfall A5 but for specific (non-catch-all) built-in rules
|
||||
|
||||
**Warning signs:**
|
||||
- `time.Tick(100 * time.Millisecond)` driving audio parameter updates
|
||||
- No smoothing/interpolation between amplitude values
|
||||
- Testing with ping floods (bursty) rather than continuous traffic
|
||||
**Prevention:**
|
||||
Two viable strategies:
|
||||
1. **User rules first:** `userRules + specificDefaultRules + catchAllRules`. User rules always take precedence. Built-ins serve as fallback. This is the simplest design and most aligned with user expectations ("I configure what I care about; defaults handle everything else").
|
||||
2. **Conflict detection:** After merging, scan for duplicate `(protocol, dstPort)` pairs and emit a warning: `"User rule for tcp:443 shadows built-in HTTPS rule. Did you mean to replace it?"`.
|
||||
|
||||
Option 1 is recommended for simplicity. Document it clearly: "User-defined rules are evaluated before built-in rules."
|
||||
|
||||
**Detection:**
|
||||
- User-defined rule for a built-in port (80, 443, 22, etc.) never activates
|
||||
- Verbose output shows built-in class instead of user class for the expected traffic
|
||||
|
||||
**Phase to address:**
|
||||
Traffic aggregation / audio mapping phase — establish the window and smoothing strategy before wiring traffic data to audio parameters.
|
||||
User-defined rules phase, merge strategy design. Address at the same time as Pitfall A5.
|
||||
|
||||
---
|
||||
|
||||
## Technical Debt Patterns
|
||||
### Pitfall A10: New TrafficClass Strings From Config Are Not Validated — Empty String or Whitespace Is a Valid Key
|
||||
|
||||
| Shortcut | Immediate Benefit | Long-term Cost | When Acceptable |
|
||||
|----------|-------------------|----------------|-----------------|
|
||||
| `google/gopacket` instead of `gopacket/gopacket` | Familiar, more tutorials | Unmaintained; Go compat breaks | Never |
|
||||
| `sudo ./netsynth` only, no `setcap` docs | Simpler setup instructions | Users won't run as root in practice; tool appears broken | MVP only — document the limitation |
|
||||
| Dynamic libpcap linking (no static build) | Faster to compile | Binary doesn't work on target machines without libpcap installed | Only acceptable if distributing via package manager that declares the dep |
|
||||
| `ReadPacketData` (copying) instead of `ZeroCopy` | Safe, simple | ~20% memory overhead at high packet rates | Always acceptable; optimize only if profiling proves allocation bottleneck |
|
||||
| Sine-wave-only synthesis (no ADSR, no envelope) | Much simpler code | Tonal changes are abrupt, not perceptually smooth | Acceptable for v1 ambient/drone if EMA smoothing is applied to amplitude |
|
||||
| Hard-coded frequency table (no config) | No CLI complexity | Can't tune without recompiling | Acceptable for v1 per PROJECT.md out-of-scope decision |
|
||||
**What goes wrong:**
|
||||
A user writes:
|
||||
|
||||
```toml
|
||||
[[rules]]
|
||||
protocol = "tcp"
|
||||
dst_port = 9200
|
||||
class = ""
|
||||
```
|
||||
|
||||
The string `""` decodes without error. It is a valid Go map key. It gets inserted into the `WindowSnapshot.Counts` map and the aggregator increments `Counts[""]`. The bank has no layer for `""`. The behavior is undefined — silent or panic depending on nil-guard presence.
|
||||
|
||||
Similarly, `class = " elasticsearch "` (padded spaces) decodes to a string with leading/trailing whitespace that does not match any configured sound entry (because the config sound entry key is `"elasticsearch"` without spaces).
|
||||
|
||||
**Prevention:**
|
||||
Validate all `Class` string values from user rules in the `validate()` step:
|
||||
```go
|
||||
if strings.TrimSpace(rule.Class) == "" {
|
||||
return fmt.Errorf("rule %d: class name must not be empty", i)
|
||||
}
|
||||
rule.Class = strings.TrimSpace(rule.Class)
|
||||
```
|
||||
Also validate that class names do not collide with reserved built-in class names (`"ICMP"`, `"DNS"`, etc.) unless the user is explicitly overriding a built-in sound (which is a distinct feature — it should be opt-in, not accidental).
|
||||
|
||||
**Phase to address:**
|
||||
Config validation step.
|
||||
|
||||
---
|
||||
|
||||
## Integration Gotchas
|
||||
## v1.0 Pitfalls (Retained for Reference)
|
||||
|
||||
The following pitfalls from the initial MVP research remain valid. They are retained in condensed form for reference.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall B1: Using `google/gopacket` Instead of the Active Community Fork
|
||||
|
||||
**What goes wrong:** Import of the unmaintained original — 270 open issues, Go compat degrades.
|
||||
**Prevention:** Import `github.com/gopacket/gopacket` (v1.5.0, requires Go 1.24+).
|
||||
**Phase:** Phase 1 — set correct import path from day one.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall B2: CGo Destroys the "Single Binary" Promise
|
||||
|
||||
**What goes wrong:** `gopacket/pcap` (CGo + libpcap) produces a dynamically-linked binary that fails on machines without `libpcap.so`.
|
||||
**Prevention:** Use `packetcap/go-pcap` (pure Go capture, already the chosen stack). Verify with `ldd ./netsynth`.
|
||||
**Phase:** Phase 1 — foundational architecture decision.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall B3: `CAP_NET_RAW` + Binary Location = Silent Failure on Linux
|
||||
|
||||
**What goes wrong:** `setcap` is silently ignored on `nosuid` filesystems. Binary appears broken from home directories.
|
||||
**Prevention:** Install to `/usr/local/bin`; document two run modes; emit clear privilege error.
|
||||
**Phase:** Phase 1 + CLI UX.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall B4: Packet Buffer Overflow Under Moderate Traffic Load
|
||||
|
||||
**What goes wrong:** Default capture buffer fills faster than the classifier consumes it; silent packet drops misrepresent traffic.
|
||||
**Prevention:** Large capture buffer (32 MB); buffered channel between capture and classify goroutines.
|
||||
**Phase:** Phase 1/2 (capture pipeline architecture).
|
||||
|
||||
---
|
||||
|
||||
### Pitfall B5: ZeroCopy Packet Data Use-After-Free
|
||||
|
||||
**What goes wrong:** `ZeroCopyReadPacketData()` invalidates previous slice on each call; silent data corruption in concurrent code.
|
||||
**Prevention:** Use `ReadPacketData()` (copying API) unless profiling proves allocation bottleneck.
|
||||
**Phase:** Phase 1 (capture/decode).
|
||||
|
||||
---
|
||||
|
||||
### Pitfall B6: MP3 Output Is Corrupt Due to LAME Initialization Order
|
||||
|
||||
**What goes wrong:** Skipping `InitParams()` or setting parameters out of order produces unplayable MP3.
|
||||
**Prevention:** Always call `InitParams()` before writing frames; smoke test with `ffprobe`.
|
||||
**Phase:** Audio synthesis / encoding phase.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall B7: PCM Sample Overflow Produces Wrap-Around Distortion
|
||||
|
||||
**What goes wrong:** Summing `int16` layers overflows and wraps (32767 + 100 = -32667), producing buzzing distortion.
|
||||
**Prevention:** Synthesize in `float64 [-1.0, 1.0]`; clamp before int16 cast. Already implemented in `synth/mixer.go`.
|
||||
**Phase:** Audio synthesis (already addressed in v1.0).
|
||||
|
||||
---
|
||||
|
||||
### Pitfall B8: Tone-per-Protocol Mapping Produces Perceptual Chaos
|
||||
|
||||
**What goes wrong:** Frequencies too close together mask each other; output is undifferentiated buzz.
|
||||
**Prevention:** Space protocols across register bands; use harmonic/musical intervals. Already addressed in v1.0.
|
||||
**Phase:** Audio mapping (already addressed in v1.0).
|
||||
|
||||
---
|
||||
|
||||
## v1.1 Phase-Specific Warnings
|
||||
|
||||
| Phase Topic | Likely Pitfall | Mitigation |
|
||||
|-------------|---------------|------------|
|
||||
| TOML struct design | A1: zero-value overwrites defaults | Use pointer fields for all optional overrides |
|
||||
| Config strict decode | A2: typos silently ignored | Use `md.Undecoded()` as strict mode check |
|
||||
| Waveform implementation | A3: naive waveform aliases | Use additive synthesis (bandlimited harmonic series) — compatible with existing `[]HarmonicDef` API |
|
||||
| Waveform string input | A4: case/abbreviation mismatches | Normalize + validate with clear error listing accepted values |
|
||||
| Rule merge ordering (catch-alls) | A5: user rules after catch-alls are unreachable | Split `DefaultRules` into `SpecificRules` + `CatchAllRules`; user rules go in between |
|
||||
| Bank construction | A6: custom class has no synth layer | Derive full class set from merged rule slice; pass to bank constructor |
|
||||
| Config discovery | A7: XDG ignored, `~/.config` hardcoded | Use `os.UserConfigDir()` not `os.UserHomeDir() + "/.config"` |
|
||||
| --config flag path | A8: missing explicit path silently ignored | Two distinct code paths: flag path (require) vs auto-discovery (skip-missing) |
|
||||
| Rule merge ordering (specific built-ins) | A9: user rule shadowed by built-in for same port | User rules first in merged slice (`userRules + specificDefaults + catchAlls`) |
|
||||
| Class name validation | A10: empty/whitespace class name is valid Go string | Validate and trim all class strings in `validate()` |
|
||||
|
||||
---
|
||||
|
||||
## Integration Gotchas (v1.1 Additions)
|
||||
|
||||
| Integration | Common Mistake | Correct Approach |
|
||||
|-------------|----------------|------------------|
|
||||
| `gopacket/pcap` handle | Not calling `handle.Close()` on signal — leaks capture resources | Use `defer handle.Close()` and ensure the goroutine exits before process termination |
|
||||
| LAME CGo encoder | Not flushing the encoder before closing — truncated final MP3 frame | Call `encoder.Flush()` / `lame.EncodeFlush()` after the sample loop ends |
|
||||
| OS signal handling (`SIGINT`) | Goroutine receives SIGINT but the capture loop is blocked on `ReadPacketData` | Use `handle.SetReadDeadline(time.Now())` or close the handle to unblock |
|
||||
| MP3 encoder sample format | Passing `float64` samples directly to LAME (expects `int16` or `float32` depending on binding) | Explicitly convert and clamp to the binding's expected type; check each binding's API |
|
||||
| `pcapgo.EthernetHandle` | Only captures Ethernet frames — fails on WiFi (802.11), loopback, or tunnel interfaces | For non-Ethernet interfaces, use the `pcap` backend or check link type at startup |
|
||||
|
||||
---
|
||||
|
||||
## Performance Traps
|
||||
|
||||
| Trap | Symptoms | Prevention | When It Breaks |
|
||||
|------|----------|------------|----------------|
|
||||
| Single goroutine: capture + classify + synthesize | CPU-bound synthesis blocks packet reads; drops spike under any real traffic | Three-stage pipeline: capture goroutine → classify channel → synthesis goroutine | Breaks on any network with > ~1000 pps |
|
||||
| One goroutine per packet | Goroutine creation overhead exceeds packet processing time; OOM on busy networks | Channel-based batching: one reader, N classifiers from a worker pool | Breaks above ~10k pps |
|
||||
| Recomputing sine wave sample-by-sample in inner loop using `math.Sin` | CPU pegged at 100% during synthesis; output can't keep pace | Precompute wavetable per frequency; iterate with phase accumulator | Breaks with > 4-5 simultaneous drone layers at 44100 Hz |
|
||||
| Blocking channel between capture and synthesis with no buffer | Any synthesis stall causes packet drops | Buffered channel of 1000+ packets; separate goroutines | Breaks immediately on any CPU scheduling hiccup |
|
||||
|
||||
---
|
||||
|
||||
## Security Mistakes
|
||||
|
||||
| Mistake | Risk | Prevention |
|
||||
|---------|------|------------|
|
||||
| Requesting full `root` and keeping it throughout capture | Privilege escalation if a parsing bug in gopacket can be exploited via crafted packets | Drop privileges after opening the capture handle: `syscall.Setuid(originalUID)` |
|
||||
| Promiscuous mode on by default without user opt-in | Captures all LAN traffic, not just traffic to/from the host — legal and privacy risk on shared networks | Default to non-promiscuous; add `--promiscuous` flag with a warning message |
|
||||
| No limit on capture duration or file size | Unbounded run produces an arbitrarily large MP3 or consumes all memory in the aggregator maps | Add `--max-duration` flag (default: warn at 10min, hard limit at 1hr); prune old flow state periodically |
|
||||
| Logging decoded packet payloads in debug mode | Inadvertently logs credentials or private data | Never log packet payload bytes; log only headers and metadata |
|
||||
|
||||
---
|
||||
|
||||
## UX Pitfalls
|
||||
|
||||
| Pitfall | User Impact | Better Approach |
|
||||
|---------|-------------|-----------------|
|
||||
| Silent failure when interface doesn't exist | User specifies `-i eth1` on a machine with only `ens3`; tool exits with cryptic libpcap error | List available interfaces at startup with `pcap.FindAllDevs()` and suggest correct name |
|
||||
| No progress feedback during capture | User has no idea if the tool is working; assumes it hung | Print periodic status line: "Capturing... 1,234 packets classified (HTTPS:45% DNS:30% ICMP:8% other:17%)" |
|
||||
| Output MP3 path collision without warning | Re-running overwrites previous output | Warn if output file exists; suggest timestamped default filename |
|
||||
| Ctrl+C produces empty or invalid MP3 | User interrupts too quickly before any traffic is captured | Detect zero-packet case and emit an error instead of an empty file |
|
||||
| No indication of which interface is being captured | Confusing when multiple interfaces exist | Print "Capturing on: eth0 (192.168.1.5)" at startup |
|
||||
|
||||
---
|
||||
|
||||
## "Looks Done But Isn't" Checklist
|
||||
|
||||
- [ ] **Packet capture:** Binary runs as non-root user with `setcap` — verify from a non-`/home` path, not just from the build directory
|
||||
- [ ] **MP3 output:** File validates with `ffprobe` or `mp3val` — not just "has .mp3 extension and non-zero size"
|
||||
- [ ] **Static binary:** `ldd ./netsynth` shows "not a dynamic executable" (or explicitly "requires libpcap" if dynamic is accepted)
|
||||
- [ ] **Signal handling:** Ctrl+C during capture produces a valid (playable) MP3, not a truncated file
|
||||
- [ ] **High-traffic:** Drop counter is zero (or documented/acceptable) when tested against a network with > 1000 pps
|
||||
- [ ] **Audio layers:** Output with 6+ simultaneous traffic types does not distort — no wrap-around clipping audible
|
||||
- [ ] **Empty capture:** Graceful error message when zero packets were captured, not a silent empty file
|
||||
- [ ] **Interface not found:** Helpful error with available interface list, not a libpcap raw error string
|
||||
|
||||
---
|
||||
|
||||
## Recovery Strategies
|
||||
|
||||
| Pitfall | Recovery Cost | Recovery Steps |
|
||||
|---------|---------------|----------------|
|
||||
| Wrong gopacket fork | LOW | `go mod edit -replace github.com/google/gopacket=github.com/gopacket/gopacket@v1.5.0`; update import paths |
|
||||
| Dynamic binary on clean machine | MEDIUM | Add static build Makefile target; update CI; update README |
|
||||
| PCM overflow / distortion | LOW | Refactor synthesis to float64 internal representation; add clamp before int16 cast |
|
||||
| Corrupt MP3 (missing flush) | LOW | Add `Flush()` call in the shutdown path |
|
||||
| Perceptual chaos (tone mapping) | MEDIUM | Redesign frequency table (no code change to synthesis engine); requires subjective listening tests |
|
||||
| Time window jitter | LOW | Add EMA smoothing and increase window; no architectural change needed |
|
||||
| `ZeroCopy` data corruption | MEDIUM | Replace `ZeroCopyReadPacketData` with `ReadPacketData`; audit all goroutine handoffs |
|
||||
|
||||
---
|
||||
|
||||
## Pitfall-to-Phase Mapping
|
||||
|
||||
| Pitfall | Prevention Phase | Verification |
|
||||
|---------|------------------|--------------|
|
||||
| Wrong gopacket fork | Phase 1: Packet Capture | `go.mod` references `gopacket/gopacket`; `go list -m github.com/gopacket/gopacket` |
|
||||
| CGo / single binary contract | Phase 1: Packet Capture | `ldd` output on CI; test on clean Alpine container |
|
||||
| CAP_NET_RAW binary location | Phase 1 + CLI UX phase | Test `setcap` from `/usr/local/bin`; verify helpful error message from non-root |
|
||||
| Packet buffer overflow | Phase 1/2: Capture Pipeline | `SetBufferSize` call present; goroutine architecture is async (channel-separated) |
|
||||
| ZeroCopy use-after-free | Phase 1: Capture/Decode | Code review: no `ZeroCopy` passed to goroutines without copy; or use `ReadPacketData` |
|
||||
| LAME init errors / corrupt MP3 | Audio synthesis phase | CI smoke test: 1s silence → `ffprobe` validates output file |
|
||||
| PCM overflow wrap-around | Audio synthesis phase | Unit test: 8 simultaneous max-amplitude layers produce no distortion |
|
||||
| Perceptual tone chaos | Audio mapping phase | Subjective listen test with mixed traffic capture; frequency table reviewed against auditory masking |
|
||||
| Time window jitter | Traffic aggregation / mapping phase | Capture test with bursty traffic; verify EMA smoothing produces stable amplitude |
|
||||
| Config → Bank wire-up | Pass `classify.AllClasses()` to bank; custom classes missing | Derive layer set from `Classifier.ActiveClasses()` — all classes reachable via the effective rule set |
|
||||
| Waveform → FreqConfig | Add `WaveformType string` to `FreqConfig`; forget to generate harmonics at bank init | Generate `[]HarmonicDef` from waveform+freq at bank/layer construction time, not at sample render time |
|
||||
| User rules → Classifier | Replace `DefaultRules` var directly; breaks tests relying on it | Keep `DefaultRules` immutable; construct `mergedRules` for runtime use |
|
||||
| Config file absent | Return error if no config found | Return nil (no config = all defaults). Only error on explicit `--config` path that is missing |
|
||||
| Sound overrides for built-in class | User sets freq for "HTTPS" — must hit `ClassHTTPS` layer | Match config sound keys case-insensitively against `TrafficClass` string values; map `"HTTPS"` → `classify.ClassHTTPS` |
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
- [gopacket/gopacket (community fork, v1.5.0)](https://github.com/gopacket/gopacket) — active fork status
|
||||
- [google/gopacket issue #329: 98% packet loss under high traffic](https://github.com/google/gopacket/issues/329) — buffer overflow and afpacket solution
|
||||
- [google/gopacket issue #1016: current project status](https://github.com/google/gopacket/issues/1016) — unmaintained status of original repo
|
||||
- [google/gopacket issue #1167: static linking libpcap.a](https://github.com/google/gopacket/issues/1167) — static build complications
|
||||
- [ZeroCopyReadPacketData docs (gopacket/pcap)](https://pkg.go.dev/github.com/google/gopacket/pcap) — memory ownership contract
|
||||
- [linuxvox.com: CAP_NET_RAW outside /usr/bin](https://linuxvox.com/blog/raw-capture-capabilities-cap-net-raw-cap-net-admin-not-working-outside-usr-bin-and-friends-for-packet-capture-program-using-libpcap/) — nosuid and AppArmor restrictions
|
||||
- [braheezy.github.io: What I Learned About MP3 Encoding](https://braheezy.github.io/posts/what-i-learned-about-mp3-encoding/) — Go MP3 encoding pitfalls
|
||||
- [github.com/braheezy/shine-mp3](https://github.com/braheezy/shine-mp3) — pure Go MP3 encoder (no CGo)
|
||||
- [Eli Bendersky: Building Static Binaries with Go on Linux](https://eli.thegreenplace.net/2024/building-static-binaries-with-go-on-linux/) — CGo static linking strategy
|
||||
- [SoNSTAR: Sonification of Networks for Situational Awareness](https://github.com/nuson/SoNSTAR) — reference architecture for network sonification
|
||||
- [PLOS One: Sonification of Network Traffic Flow](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0195948) — time window and design lessons
|
||||
- [KVR Audio: PCM float-to-int clipping and wrap-around](https://www.kvraudio.com/forum/viewtopic.php?t=414666) — PCM overflow consequences
|
||||
- [bjornroche.com: ABCs of PCM Digital Audio](http://blog.bjornroche.com/2013/05/the-abcs-of-pcm-uncompressed-digital.html) — sample format fundamentals
|
||||
- [BurntSushi/toml pkg.go.dev](https://pkg.go.dev/github.com/BurntSushi/toml) — `Undecoded()` strict mode, pointer field behavior, `MetaData` API
|
||||
- [BurntSushi/toml issue #47: Unmarshal with default values](https://github.com/BurntSushi/toml/issues/47) — confirms default-overwrite behavior
|
||||
- [pelletier/go-toml issue #252: Unmarshal overrides origin values if key is omitted](https://github.com/pelletier/go-toml/issues/252) — confirms same behavior in v1; v2 partially resolves
|
||||
- [pelletier/go-toml v2 pkg.go.dev](https://pkg.go.dev/github.com/pelletier/go-toml/v2) — strict decoder mode documentation
|
||||
- [golang/go issue #29960: os: add UserConfigDir](https://github.com/golang/go/issues/29960) — rationale for `os.UserConfigDir()` (XDG-aware)
|
||||
- [WolfSound: Basic Waveforms in Synthesis](https://thewolfsound.com/sine-saw-square-triangle-pulse-basic-waveforms-in-synthesis/) — aliasing and harmonic series for square/saw/triangle
|
||||
- [CCRMA: Alias-Free Digital Synthesis of Classic Analog Waveforms](https://ccrma.stanford.edu/~stilti/papers/blit.pdf) — bandlimited synthesis theory
|
||||
- [McGill Bandlimited Synthesis of Classic Waveforms](https://www.music.mcgill.ca/~gary/307/week5/bandlimited.html) — truncated harmonic series approach
|
||||
- [Teensy Forum: triangle & sawtooth oscillators aliasing](https://forum.pjrc.com/threads/61269-triangle-amp-sawtooth-oscillators-how-to-deal-with-aliasing) — practical aliasing impact at different frequencies
|
||||
- [adrg/xdg package](https://github.com/adrg/xdg) — XDG Base Directory Specification Go implementation (reference; stdlib `os.UserConfigDir()` is sufficient for NetSynth's needs)
|
||||
|
||||
---
|
||||
*Pitfalls research for: network-traffic-to-audio synthesis CLI (Go) — NetSynth*
|
||||
*Researched: 2026-03-24*
|
||||
*Pitfalls research for: NetSynth v1.1 — TOML config, waveform types, user-defined rules*
|
||||
*Updated: 2026-03-26*
|
||||
|
||||
Reference in New Issue
Block a user