Files
yoloyolo/.planning/research/PITFALLS.md
T

554 lines
36 KiB
Markdown
Raw Normal View History

2026-03-27 08:25:02 +01:00
# Domain Pitfalls
2026-03-24 22:36:28 +01:00
2026-03-27 08:25:02 +01:00
**Domain:** NetSynth — Network traffic sonification CLI tool (Go)
**Researched:** 2026-03-26 (v1.1 original); 2026-03-27 (v1.2 update — extended protocol coverage, grouped sound families)
**Confidence:** HIGH (all pitfalls grounded in direct codebase inspection; audio masking values from Glasberg & Moore 1990 ERB model; TOML behaviors from BurntSushi docs and issue history; Go performance from first-principles analysis of existing code)
2026-03-24 22:36:28 +01:00
---
2026-03-27 08:25:02 +01:00
## v1.2 Milestone Pitfalls (New)
2026-03-24 22:36:28 +01:00
2026-03-27 08:25:02 +01:00
These pitfalls are specific to adding expanded protocol classification with grouped sound families to the existing NetSynth v1.1 codebase. They are ordered by severity: critical pitfalls cause incorrect output or broken configs without obvious errors; moderate pitfalls degrade audio quality or developer experience; minor pitfalls are friction points with clear workarounds.
2026-03-26 16:54:31 +01:00
---
2026-03-27 08:25:02 +01:00
## Critical Pitfalls
### Pitfall C1: Frequency Rebalancing Silently Invalidates User v1.1 Configs
2026-03-24 22:36:28 +01:00
**What goes wrong:**
2026-03-27 08:25:02 +01:00
Users who created a `netsynth.toml` under v1.1 specified absolute Hz values for built-in classes — for example `[sounds.HTTPS] frequency = 175.0`. If v1.2 rebalances that class to a new default (say 340 Hz to make room for new protocols), the user's config now overrides the new default back to the old v1.1 value (175 Hz). The user gets a v1.1 sound for HTTPS even after upgrading, with no warning that their override value has become stale relative to the rebalanced layout.
The reverse is also possible: a user who did NOT set a frequency override had HTTPS at 175 Hz; after rebalancing it moves silently to a different Hz value. Their soundscape has changed without explanation.
2026-03-24 22:36:28 +01:00
**Why it happens:**
2026-03-27 08:25:02 +01:00
The `merge()` function in `config/config.go` applies any `[sounds.X] frequency = Y` from the user's TOML unconditionally. There is no concept of "this override is relative to a previous default" — it is applied as a fixed Hz value. Changing the default in `ClassFreqConfigs` does not trigger any validation that existing user overrides remain intentional.
2026-03-24 22:36:28 +01:00
2026-03-26 16:54:31 +01:00
**Consequences:**
2026-03-27 08:25:02 +01:00
- v1.1 user configs produce different-than-expected audio on v1.2 without any error or warning
- Users with explicit overrides are stuck at v1.1 frequency values — the rebalancing has zero effect for them
- Users without overrides hear an unexplained soundscape change after upgrade
2026-03-24 22:36:28 +01:00
2026-03-26 16:54:31 +01:00
**Prevention:**
2026-03-27 08:25:02 +01:00
Two complementary strategies:
2026-03-26 16:54:31 +01:00
2026-03-27 08:25:02 +01:00
1. **Minimize rebalancing scope.** Assign new protocol classes to frequency ranges not yet occupied by v1.1 built-ins. The current v1.1 built-in range is 65780 Hz (known protocols) and 8621047 Hz (unknown buckets), with a gap at 781861 Hz. New families can be allocated into ranges above 1100 Hz (e.g., 11004000 Hz), leaving all existing Hz assignments untouched. This eliminates the backward-compatibility problem entirely for users who have not overridden values in that range.
2026-03-26 16:54:31 +01:00
2026-03-27 08:25:02 +01:00
2. **Changelog + --print-config.** If rebalancing IS required, document every changed Hz value in the release notes and update `--print-config` so users can diff their effective config against what they saved. Add a comment to `--print-config` output when a user override matches a value that was the v1.1 default (warn that it may be stale).
2026-03-26 16:54:31 +01:00
**Detection:**
2026-03-27 08:25:02 +01:00
- User reports HTTPS sounds wrong after upgrade
- `--print-config` shows `(override)` for a class the user never intentionally customized — they set it once to the default value and now that exact value is stale
2026-03-24 22:36:28 +01:00
**Phase to address:**
2026-03-27 08:25:02 +01:00
Frequency allocation design phase (first phase of v1.2). The spectrum layout must be finalized before writing any `ClassFreqConfigs` entries. Treat the 651047 Hz range as frozen for backward compatibility.
2026-03-24 22:36:28 +01:00
---
2026-03-27 08:25:02 +01:00
### Pitfall C2: `autoAssignFreq` Range Collision With New Built-in Frequencies
2026-03-24 22:36:28 +01:00
**What goes wrong:**
2026-03-27 08:25:02 +01:00
`autoAssignFreq()` in `config/config.go` assigns custom user classes to frequencies in `[1200, 2350]` Hz using 24 steps of 50 Hz each. If new v1.2 built-in protocol classes are assigned frequencies in this same range (e.g., placing SIP at 1500 Hz or IMAP at 1200 Hz), a user's custom class may hash to the same frequency as a new built-in. The user's custom class and the new built-in will produce the same tone — the soundscape loses discriminability, and the user has no way to know their custom class has collided.
2026-03-24 22:36:28 +01:00
**Why it happens:**
2026-03-27 08:25:02 +01:00
The `[1200, 2350]` range was deliberately chosen in v1.1 as "unused 1200-2350 Hz range" (per the STACK.md annotation). If v1.2 extends built-ins into that range without also updating `autoAssignFreq`, the guarantee is broken.
2026-03-24 22:36:28 +01:00
2026-03-26 16:54:31 +01:00
**Consequences:**
2026-03-27 08:25:02 +01:00
- Silent frequency collision: two classes (one built-in, one user-defined) play the same tone
- User's custom classification is perceptually indistinguishable from the colliding built-in
- `--print-config` will show different Hz values in the TOML text, but the audio output is identical
2026-03-24 22:36:28 +01:00
2026-03-26 16:54:31 +01:00
**Prevention:**
2026-03-27 08:25:02 +01:00
Update `autoAssignFreq` in `config/config.go` whenever new built-in frequency ranges are allocated. Specifically:
- After finalizing all v1.2 `ClassFreqConfigs` frequencies, compute the highest built-in Hz value
- Set `autoAssignFreq` base above that value, e.g., `baseHz = 4500.0` with a range that is guaranteed to be clear of all built-ins
- Add a compile-time assertion (test) that verifies no `ClassFreqConfigs` entry falls inside the auto-assign range
2026-03-26 16:54:31 +01:00
```go
2026-03-27 08:25:02 +01:00
// In synth/config_test.go:
func TestAutoAssignRangeIsEmpty(t *testing.T) {
const autoBase = 4500.0
const autoTop = 6000.0
for class, cfg := range ClassFreqConfigs {
if cfg.BaseHz >= autoBase && cfg.BaseHz <= autoTop {
t.Errorf("built-in class %q frequency %.1f Hz falls in auto-assign range [%.0f, %.0f]",
class, cfg.BaseHz, autoBase, autoTop)
2026-03-26 16:54:31 +01:00
}
2026-03-27 08:25:02 +01:00
}
}
```
**Detection:**
- Two classes with different names produce identical tones
- `--print-config` shows correct but coincidentally matching Hz values for a user class and a built-in
**Phase to address:**
Frequency allocation design phase. After all new built-in Hz values are set, update `autoAssignFreq` constants and add the compile-time range check before merging.
---
### Pitfall C3: Hardcoded `NumLayers = 14` Constant Becomes a Lie — But `GainPerLayer` Stays Wrong
**What goes wrong:**
`synth/config.go` defines:
```go
NumLayers = 14
GainPerLayer = 1.0 / float64(NumLayers) // 0.0714
```
The current `bank.go` correctly uses `1.0 / float64(len(cfgs))` at bank construction time, so the bank itself handles more layers correctly. However, `GainPerLayer` is still exported as a package-level constant. If any code outside `bank.go` references `synth.GainPerLayer` for gain calculations (including tests, the encode pipeline, or future code added in v1.2), it will use the stale `0.0714` value even when 25+ layers are active. The bank sounds louder than expected at low class counts, or softer at high class counts, depending on which reference is used.
Additionally, `TestNumLayersMatchesAllClasses` in `synth/config_test.go` checks:
```go
if len(synth.ClassFreqConfigs) != len(classify.AllClasses())
```
This test enforces that `ClassFreqConfigs` and `AllClasses()` stay in sync. Adding new classes to one without the other causes this test to fail — which is the right behavior, but the fix is non-obvious: you must update both `ClassFreqConfigs` (with a new `FreqConfig` entry) AND `AllClasses()` (by appending to the return value). Miss either and the test blocks the build.
**Why it happens:**
`NumLayers` was introduced in v1.0 when the layer count was static. It was not removed when `NewBank` was refactored to use `len(cfgs)` dynamically. The constant is now a documentation artifact that can mislead future code.
**Consequences:**
- Any code added in v1.2 that references `synth.GainPerLayer` uses an incorrect value
- Possible audio clipping (if gain is too high) or inaudibly quiet output (if computed with wrong layer count)
- `TestNumLayersMatchesAllClasses` fails if `AllClasses()` and `ClassFreqConfigs` are updated independently
**Prevention:**
At the start of v1.2 protocol rule addition:
1. Remove `NumLayers` and `GainPerLayer` constants from `synth/config.go` (or mark them deprecated with a clear comment)
2. Rename `TestNumLayersMatchesAllClasses` to `TestClassFreqConfigsMatchesAllClasses` and update its comment to explain the invariant
3. When adding each new protocol class: update `AllClasses()` and `ClassFreqConfigs` atomically in the same commit — the test will catch any missed entry
**Detection:**
- `TestNumLayersMatchesAllClasses` fails after adding new classes to only one of the two locations
- Audio output is unexpectedly loud or quiet compared to previous version
**Phase to address:**
First code phase of v1.2, before adding any new protocol classes. Removing the stale constant and renaming the test is a two-minute cleanup that prevents confusion throughout the milestone.
---
### Pitfall C4: `TestFrequenciesInRange` Hardcodes `[60, 1100]` — Will Fail for New High-Frequency Classes
**What goes wrong:**
`synth/config_test.go` contains:
```go
func TestFrequenciesInRange(t *testing.T) {
for class, cfg := range synth.ClassFreqConfigs {
if cfg.BaseHz < 60 || cfg.BaseHz > 1100 {
t.Errorf("class %q BaseHz=%.1f is out of range [60, 1100]", class, cfg.BaseHz)
2026-03-26 16:54:31 +01:00
}
2026-03-27 08:25:02 +01:00
}
}
```
If any new v1.2 protocol class is assigned a frequency above 1100 Hz (which is necessary if new families extend into the 11004000 Hz range), this test fails immediately. The test was written for v1.0's 14-class spectrum. It will now become a false blocker, making every correct new assignment fail CI.
**Why it happens:**
Range-assertion tests like this encode a snapshot of the system state at the time they were written. They do not automatically update when the valid range evolves.
**Consequences:**
- CI red on every correct new class addition until the test is updated
- Developer wastes time diagnosing a failing test that is wrong, not the code
- Risk: developer deletes the test entirely rather than updating it, losing the coverage
**Prevention:**
Update the test when the frequency allocation design is finalized. The new range should accommodate whatever spectrum is decided, e.g.:
```go
func TestFrequenciesInRange(t *testing.T) {
for class, cfg := range synth.ClassFreqConfigs {
if cfg.BaseHz < 60 || cfg.BaseHz > 4000 {
t.Errorf("class %q BaseHz=%.1f is out of range [60, 4000]", class, cfg.BaseHz)
2026-03-26 16:54:31 +01:00
}
}
}
```
2026-03-27 08:25:02 +01:00
Alternatively, replace the range test with a `TestFrequenciesUnique` variant that only checks for collisions (already present), which remains valid regardless of range expansion. The range test can instead verify family-level groupings: all Mail family classes are in [X, Y] Hz range, all VoIP classes in [A, B] Hz range.
2026-03-26 16:54:31 +01:00
**Detection:**
2026-03-27 08:25:02 +01:00
- CI fails on `TestFrequenciesInRange` after adding first new class above 1100 Hz
- The test name suggests a range violation but the code is correct
2026-03-24 22:36:28 +01:00
**Phase to address:**
2026-03-27 08:25:02 +01:00
Immediately when frequency allocation is decided — before adding any `ClassFreqConfigs` entries outside [60, 1100].
2026-03-24 22:36:28 +01:00
---
2026-03-27 08:25:02 +01:00
### Pitfall C5: `[families]` TOML Block Rejected by Strict Unknown-Key Validation
2026-03-24 22:36:28 +01:00
**What goes wrong:**
2026-03-27 08:25:02 +01:00
If v1.2 adds a `[families]` section (or any new top-level TOML key) to the config schema to support per-family sound configuration, a user who adds this to their TOML file will get an error when running an unpatched v1.1 binary:
```
config: unknown key "families" — check spelling
```
The `parseFile()` function uses `md.Undecoded()` as strict mode, which rejects any key not in `rawConfig`. The `rawConfig` struct only knows about `sounds` and `rules`. Adding `families` to the user's TOML will break the tool for anyone running the v1.1 binary — even if they do not care about family features yet.
2026-03-24 22:36:28 +01:00
**Why it happens:**
2026-03-27 08:25:02 +01:00
This is the correct and intended behavior of `md.Undecoded()` (introduced specifically to prevent silent misconfiguration). But it means the config schema is strictly versioned: any new field must be added to `rawConfig` before any user can write it to their TOML.
2026-03-24 22:36:28 +01:00
2026-03-26 16:54:31 +01:00
**Consequences:**
2026-03-27 08:25:02 +01:00
- User adds `[families]` to their config, upgrades their TOML, then tries to run v1.1 binary (e.g., from a build that hasn't shipped yet) — immediate error
- More critically: if v1.2 adds `[families]` to `rawConfig` but the user's v1.1 config does not have `[families]` at all — this direction is fine, since absent keys are not reported by `Undecoded()`
2026-03-24 22:36:28 +01:00
2026-03-26 16:54:31 +01:00
**Prevention:**
2026-03-27 08:25:02 +01:00
The direction of concern is: user writes v1.2 TOML, runs v1.1 binary. Since this project does not provide v1.1 binary distribution to end users (it's a CLI built from source), this pitfall is primarily about development workflow and test fixtures:
1. Update `rawConfig` struct to include `families` (or whatever the new group config key is named) before any tests or documentation reference the new config format
2. Any test fixture `.toml` files should use the schema matching the current binary's `rawConfig` struct
3. For documentation examples: do not publish TOML samples containing `[families]` until the code that handles it is shipped
The v1.1 → v1.2 migration path for the config struct should be:
- `rawConfig` adds `Groups map[string]GroupOverride` (or similar) — unknown key validation now accepts it
- If absent in user's TOML (the common case): `raw.Groups` is nil — safe, no behavior change
- If present: processed in the new merge step
**Detection:**
- Test fixture containing new config key causes `config: unknown key` error in a test that still uses old `rawConfig`
- `--print-config` output contains new group annotations but user running v1.1 binary sees error
**Phase to address:**
Config schema extension phase. Update `rawConfig` and `PrintConfig` before any code that generates or consumes the new TOML format.
---
## Moderate Pitfalls
### Pitfall C6: Within-Family Detuning Causes Critical Band Masking at High Frequencies
**What goes wrong:**
The v1.2 spec calls for "within-group sound design: shared base frequency, different waveforms or slight detuning." If two classes in the same family are assigned frequencies closer than one critical bandwidth, the human auditory system treats them as a single tone rather than two distinct sounds. The "slightly detuned" design goal backfires: instead of sounding like two related-but-distinct protocols, the two tones merge perceptually into one broader tone with beating artifacts.
Critical bandwidth (ERB) formula: `ERB(f) = 24.7 * (4.37 * f/1000 + 1)` Hz.
Concrete values for the NetSynth frequency range:
- At 100 Hz: ~35 Hz critical bandwidth (tones must be >35 Hz apart)
- At 500 Hz: ~48 Hz critical bandwidth (tones must be >48 Hz apart)
- At 1000 Hz: ~72 Hz critical bandwidth (tones must be >72 Hz apart)
- At 2000 Hz: ~117 Hz critical bandwidth (tones must be >117 Hz apart)
The v1.1 `autoAssignFreq` used 50 Hz steps in the 12002350 Hz range. At 1200 Hz, critical bandwidth is ~88 Hz. A 50 Hz step at that frequency is inside the critical band — the two tones will mask each other.
**Why it happens:**
Frequency step sizes that feel visually reasonable (e.g., 50 Hz) do not scale with the logarithmic nature of human pitch perception. The critical band narrows in Hz as frequency decreases but the absolute Hz separation needed for perceptual distinctness increases with frequency.
**Consequences:**
- Same-family protocols sound identical rather than "related but distinct"
- Within-family distinguishability — a core design goal — is not achieved even though the Hz values differ
- The bug is hard to detect: `--print-config` shows different Hz values, but the audio is perceptually undifferentiated
**Prevention:**
Use a logarithmic (musical interval) separation for within-family detuning rather than fixed-Hz steps. A minor third (ratio 1.2) or major second (ratio 1.125) provides psychoacoustically safe separation across the full frequency range used by NetSynth:
- Family base at 800 Hz, member 2 at 800 * 1.125 = 900 Hz (100 Hz gap, safe)
- Family base at 2000 Hz, member 2 at 2000 * 1.125 = 2250 Hz (250 Hz gap, safe vs 117 Hz critical band)
Rule of thumb for NetSynth protocol family design: within a family, space members at least 1.25x the critical bandwidth of the lower tone. Using a minor second (semitone, ratio 1.059) as the minimum separation gives ~75 Hz at 1300 Hz — marginal. Use at least a major second (ratio 1.122) for reliable perceptual separation.
**Detection:**
- Two protocols in the same family sound identical in listening test
- The beating artifact (amplitude modulation at the difference frequency) is audible when two closely-spaced tones are both active
**Phase to address:**
Frequency allocation and family sound design phase. Compute critical bandwidths for all proposed family member frequencies before finalizing the allocation. A short spreadsheet checking `ERB(f) < |f2 - f1|` for each pair catches this before any code is written.
---
### Pitfall C7: Adding ~20 Rules to the Rule Slice Does Not Degrade Classification Performance, But Dual-Port Rules Do
**What goes wrong:**
The existing `DefaultRules` slice has 12 entries. Adding 2030 more for protocols like IMAP (143), IMAPS (993), POP3 (110), LDAP (389), RDP (3389), FTP (21), SIP (5060), SNMP (161), etc., increases the linear scan from ~12 comparisons to ~40 comparisons per packet.
At 44100 Hz / 22050 SamplesPerWindow = 2 windows/sec, and typical home/office network rates of 10005000 packets/sec, the classifier is called ~2500 times/sec. Each call does a linear scan over 40 rules. At ~4 ns per comparison (cache-warm slice iteration), 40-rule scan ≈ 160 ns per packet. For 5000 packets/sec, that is 0.8 ms/sec total classifier CPU — negligible.
The actual risk is not linear scan overhead but **dual-port rule confusion**: many protocols have both a plain and a TLS/secure variant on different ports (HTTP:80 and HTTPS:443, SMTP:25 and SMTPS:587, IMAP:143 and IMAPS:993, POP3:110 and POP3S:995, LDAP:389 and LDAPS:636). If these are naively placed into separate classes with different frequencies, a mail server using SMTPS at 587 will be classified differently from one using SMTP at 25 — even though they are the same protocol family. The frequency space gets overcrowded with variants that sound like separate protocols but represent the same thing.
**Why it happens:**
The classification decision "same class vs separate classes" for secure and insecure protocol variants is not obvious. The default instinct is to add more rules = more specificity = better, but perceptually the user wants "I can hear that I have mail traffic" not "I can tell the exact TLS variant."
**Consequences:**
- Frequency spectrum crowded with 2x the expected number of mail-related tones
- Secure and insecure variants of the same protocol family cancel each other's coherence
- The "family" identity becomes invisible — SMTP and SMTPS sound like different protocols
**Prevention:**
Group insecure and secure variants of the same protocol into the same `TrafficClass`:
- `ClassSMTPFamily` covers ports 25, 465, 587
- `ClassIMAPFamily` covers ports 143, 993
- `ClassPOP3Family` covers ports 110, 995
Add multiple `Rule` entries for the same class (one per port). The classifier already supports this — multiple rules with different ports mapping to the same class are correct. The frequency allocation should then be: one slot per protocol family, not one slot per port.
2026-03-26 16:54:31 +01:00
```go
2026-03-27 08:25:02 +01:00
// In classify/rules.go:
{Protocol: "tcp", DstPort: 25, Class: ClassMailSMTP},
{Protocol: "tcp", DstPort: 587, Class: ClassMailSMTP},
{Protocol: "tcp", DstPort: 465, Class: ClassMailSMTP},
```
**Detection:**
- Frequency assignment table has 30+ entries for ~15 conceptual protocol families
- Listening test: mail traffic sounds like 3 separate overlapping tones instead of one identifiable mail layer
**Phase to address:**
Protocol list design phase (before any rule code). Define the class list (family → single class name → list of ports) before implementing rules. The family grouping decision should drive the `TrafficClass` constant list.
---
### Pitfall C8: `AllClasses()` and `ClassFreqConfigs` Must Both Be Updated Atomically — Two Callsites, Not One
**What goes wrong:**
Adding a new protocol class to NetSynth requires touching three locations:
1. A new `ClassXxx TrafficClass = "xxx"` constant in `classify/types.go`
2. A new entry in `classify.AllClasses()` return slice in `classify/types.go`
3. A new entry in `synth.ClassFreqConfigs` map in `synth/config.go`
If any one of these is missing:
- Missing from `AllClasses()`: `--print-config` does not emit it; `config/config.go`'s `PrintConfig` classifies it as "user-defined" rather than "built-in"; `TestNumLayersMatchesAllClasses` fails
- Missing from `ClassFreqConfigs`: `TestAllClassesHaveConfig` fails; `copyDefaults()` does not include it; user can't override it in TOML
- Missing constant (using a string literal instead): compiles, but typos create a second unintended class
When adding 20+ new classes, the three-location update is repeated 20+ times. The likelihood of a missed entry in one location is high.
**Why it happens:**
Go does not have enum types that automatically enforce that a new member must be registered in every relevant collection. The `classify.TrafficClass` type is a `string` type alias — adding a constant does not force updates to `AllClasses()` or `ClassFreqConfigs`.
**Consequences:**
- Test failure that is diagnostic but potentially confusing ("I added the class, why does the test fail?")
- Less dangerous but still: `--print-config` shows wrong annotation (user-defined vs built-in) for new classes
**Prevention:**
Write a single source-of-truth Go data structure that drives all three, rather than maintaining them independently:
```go
// In classify/types.go: define the authoritative ordered list
var builtinClassDefs = []struct {
Class TrafficClass
Display string
}{
{ClassICMP, "ICMP"},
// ... all classes ...
{ClassMailSMTP, "mail-smtp"},
2026-03-26 16:54:31 +01:00
}
2026-03-27 08:25:02 +01:00
// AllClasses() derives from this:
func AllClasses() []TrafficClass {
classes := make([]TrafficClass, len(builtinClassDefs))
for i, def := range builtinClassDefs {
classes[i] = def.Class
2026-03-26 16:54:31 +01:00
}
2026-03-27 08:25:02 +01:00
return classes
2026-03-26 16:54:31 +01:00
}
```
2026-03-24 22:36:28 +01:00
2026-03-27 08:25:02 +01:00
Then `ClassFreqConfigs` can be validated against `AllClasses()` at test time rather than being separately maintained. Adding a new class means updating only `builtinClassDefs` — the rest is derived.
Alternatively: add a comment above both `AllClasses()` and `ClassFreqConfigs` stating "KEEP IN SYNC — adding a class requires updating both" and rely on the existing `TestNumLayersMatchesAllClasses` test to catch mismatches.
**Detection:**
- `TestNumLayersMatchesAllClasses` fails
- `TestAllClassesHaveConfig` fails
2026-03-24 22:36:28 +01:00
**Phase to address:**
2026-03-27 08:25:02 +01:00
First code phase of v1.2 protocol additions, before adding any new classes. Decide on the sync strategy and document it clearly so all 20+ additions follow the same pattern.
2026-03-24 22:36:28 +01:00
---
2026-03-27 08:25:02 +01:00
### Pitfall C9: Port-Range and Multi-Port Rules Require Protocol Rule Schema Extension
2026-03-24 22:36:28 +01:00
**What goes wrong:**
2026-03-27 08:25:02 +01:00
Some protocols use dynamic or high-number ports that cannot be expressed as a single `DstPort uint16` rule. Examples:
- RTP (VoIP media) uses ephemeral UDP ports in a range (typically 1638432767 or 4915265535)
- mDNS (multicast DNS/discovery) uses UDP port 5353 but also matches on IP multicast addresses
- NetBIOS uses ports 137, 138, 139 — three separate ports for the same protocol family
2026-03-26 16:54:31 +01:00
2026-03-27 08:25:02 +01:00
The current `Rule` struct only supports `{Protocol, DstPort, Class}`. Adding RTP and other range-based protocols cannot be represented without extending the rule schema.
2026-03-24 22:36:28 +01:00
**Why it happens:**
2026-03-27 08:25:02 +01:00
The v1.0 rule design was sufficient for well-known single-port protocols. Port ranges are a natural extension that was not anticipated. Extending the schema now risks breaking the existing TOML rule syntax (`[[rules]]` blocks) that v1.1 users have written.
2026-03-24 22:36:28 +01:00
2026-03-26 16:54:31 +01:00
**Consequences:**
2026-03-27 08:25:02 +01:00
- RTP, mDNS, and other range-based protocols cannot be classified with the current rule model
- Attempting to add them as single-port rules misses the vast majority of their traffic
- If the rule struct is extended (e.g., `DstPortMin uint16, DstPortMax uint16`), all existing rule-reading code must be updated, and the TOML format changes
2026-03-24 22:36:28 +01:00
2026-03-26 16:54:31 +01:00
**Prevention:**
2026-03-27 08:25:02 +01:00
Decide explicitly which protocols to include in v1.2 scope. If a protocol requires port-range matching, either:
1. Exclude it from v1.2 and note it as requiring a rule schema extension in a future milestone
2. Implement the range extension in the rule struct first — but verify it does not break existing TOML `[[rules]]` parsing (it should not, since adding optional fields to `RawRule` with pointer types is backward-compatible with existing configs that omit those fields)
2026-03-26 16:54:31 +01:00
2026-03-27 08:25:02 +01:00
For the v1.2 protocol list, favor protocols with well-known single static ports (IMAP:143, POP3:110, LDAP:389, RDP:3389, etc.) and defer RTP, dynamic SIP media, and NetBIOS-style multi-port protocols to a future "advanced rule types" milestone.
2026-03-26 16:54:31 +01:00
**Detection:**
2026-03-27 08:25:02 +01:00
- RTP traffic appears as `ClassOtherUDP` even after adding a rule
- Attempting to write a TOML rule for RTP using a single port produces incorrect results
2026-03-24 22:36:28 +01:00
**Phase to address:**
2026-03-27 08:25:02 +01:00
Protocol list design phase. Before implementation, filter the candidate protocol list to only those expressible with current `{Protocol, DstPort, Class}` semantics, or decide up front to extend the schema and account for the additional complexity.
2026-03-24 22:36:28 +01:00
---
2026-03-27 08:25:02 +01:00
## Minor Pitfalls
### Pitfall C10: `--print-config` Group Annotations Must Not Break Existing TOML Output Parsing
2026-03-24 22:36:28 +01:00
**What goes wrong:**
2026-03-27 08:25:02 +01:00
`PrintConfig()` in `config/config.go` emits commented TOML that users may use as a template. If v1.2 changes the output format — for example, adding group header comments like `# === Web Family ===` above related classes — and a user pipes `--print-config` output back to a config file, the comments are harmless. However, if `PrintConfig` emits actual TOML key-value pairs for a `[families]` section that the current `rawConfig` struct cannot parse, loading that output as a config file will fail with `unknown key "families"`.
2026-03-24 22:36:28 +01:00
2026-03-26 16:54:31 +01:00
**Prevention:**
2026-03-27 08:25:02 +01:00
All new group-related output in `--print-config` must either be:
1. Comments only (lines starting with `#`) — safe, TOML ignores them
2. Actual config keys that `rawConfig` can parse — requires adding them to `rawConfig` first
2026-03-26 16:54:31 +01:00
2026-03-27 08:25:02 +01:00
Never emit a `[families]` or `[groups]` TOML block in `--print-config` output before the corresponding struct field exists in `rawConfig`.
2026-03-26 16:54:31 +01:00
2026-03-27 08:25:02 +01:00
**Phase to address:** Config output phase.
2026-03-24 22:36:28 +01:00
---
2026-03-27 08:25:02 +01:00
### Pitfall C11: New Class Constants Named Inconsistently With Existing Pattern
2026-03-24 22:36:28 +01:00
**What goes wrong:**
2026-03-27 08:25:02 +01:00
The existing constants follow `ClassHTTPS`, `ClassSSH`, `ClassNTP` — protocol name in CamelCase. For grouped protocols, if constants are named `ClassMailSMTP`, `ClassMailIMAP`, `ClassMailPOP3`, the "Mail" prefix creates a new naming convention that does not match the flat naming of existing classes. The `TrafficClass` string values (e.g., `"mail-smtp"`, `"mail-imap"`) become the identifiers users reference in TOML — if these are kebab-case with family prefix (e.g., `[sounds.mail-smtp]`), that is a new pattern that does not match existing class names like `[sounds.HTTPS]` (uppercase) or `[sounds.other-TCP]` (mixed case with hyphen).
2026-03-24 22:36:28 +01:00
2026-03-26 16:54:31 +01:00
**Prevention:**
2026-03-27 08:25:02 +01:00
Decide the naming convention for grouped class string values before adding any constants:
- Option A: `"smtp"`, `"imap"`, `"pop3"` — flat names, consistent with `"SSH"`, `"DNS"` (but drops family grouping in the config key)
- Option B: `"mail-smtp"`, `"mail-imap"` — family-prefixed, makes grouping visible in TOML but is a new pattern
2026-03-26 16:54:31 +01:00
2026-03-27 08:25:02 +01:00
Existing classes use all-caps for protocols (`"HTTPS"`, `"ICMP"`) and lowercase-hyphenated for non-standard ones (`"other-TCP"`, `"unknown-1"`). New classes should follow the lowercase-hyphenated pattern for multi-word names. Document the convention at the top of `classify/types.go`.
2026-03-26 16:54:31 +01:00
2026-03-27 08:25:02 +01:00
**Phase to address:** Protocol list design phase, before writing constants.
2026-03-24 22:36:28 +01:00
---
2026-03-27 08:25:02 +01:00
### Pitfall C12: Too Many Active Layers Degrades Ambient Distinctness (Perceptual Density Threshold)
2026-03-24 22:36:28 +01:00
**What goes wrong:**
2026-03-27 08:25:02 +01:00
v1.1 has 14 layers. v1.2 will add 2030 more, reaching a total of ~3544 layers. When all layers are simultaneously active at whisper-floor amplitude, the combined output is 35 × WhisperFloor × gainPerLayer = 35 × 0.03 × (1/35) = 3% of max amplitude — still quiet. The whisper floor plus gain-per-layer math continues to work correctly.
2026-03-24 22:36:28 +01:00
2026-03-27 08:25:02 +01:00
The perceptual problem is different: with 35 simultaneous drone layers, the ambient sound loses definition. Below 58 simultaneous distinct tones, listeners can track individual threads. Above 1012, the output becomes a dense textural wash. This is not a technical bug but a UX risk: the "network fingerprint" value proposition weakens because the output sounds less like "I can identify HTTPS vs SSH" and more like "everything is one undifferentiated cloud."
2026-03-24 22:36:28 +01:00
2026-03-26 16:54:31 +01:00
**Prevention:**
2026-03-27 08:25:02 +01:00
Group protocols into families specifically to mitigate this: a family's members should share enough spectral character (same or nearby frequency, similar waveform) that they fuse into a single perceptible "family layer" rather than adding N separate threads. The "distinct family tone" becomes the perceptual unit, not each individual protocol.
2026-03-24 22:36:28 +01:00
2026-03-27 08:25:02 +01:00
Additionally, consider whether `WhisperFloor` should be reduced for high layer counts. At 35 layers, 35 × 0.03 × (1/35) = 0.03 (3% amplitude from whisper alone when all active) — which is fine. The math is self-correcting. The concern is purely perceptual richness, not clipping or silence.
2026-03-24 22:36:28 +01:00
2026-03-27 08:25:02 +01:00
**Phase to address:** Sound design review after all frequencies are assigned. Listening test with a mix of protocols active simultaneously is the definitive check.
2026-03-24 22:36:28 +01:00
---
2026-03-27 08:25:02 +01:00
## Phase-Specific Warnings (v1.2)
2026-03-26 16:54:31 +01:00
| Phase Topic | Likely Pitfall | Mitigation |
|-------------|---------------|------------|
2026-03-27 08:25:02 +01:00
| Frequency spectrum design | C1: existing Hz overrides become stale | Allocate new classes above 1100 Hz; leave v1.1 range frozen |
| Frequency spectrum design | C6: within-family tones too close | Enforce >1 critical bandwidth separation; use musical interval ratios |
| Auto-assign range update | C2: new built-ins collide with user custom class auto-assign range | Move auto-assign base above highest new built-in Hz; add range-check test |
| Test suite update | C4: TestFrequenciesInRange fails on new Hz values | Update range bound in test before adding any class above 1100 Hz |
| Protocol list curation | C7: secure + insecure variants fill 2x slots | Decide: one class per family (covering all ports) or one class per variant |
| Protocol list curation | C9: RTP and range-based protocols not expressible | Exclude from v1.2 or extend rule schema; decide before writing rules |
| Adding class constants | C3: NumLayers stale constant misleads | Remove or document-only; update test name; keep AllClasses+ClassFreqConfigs atomic |
| Adding class constants | C8: three-location update forgetting one | Establish single source of truth or add checklist in classify/types.go comment |
| Group config schema | C5: new TOML key rejected by old binary | Add to rawConfig struct before documenting or emitting the key anywhere |
| --print-config update | C10: output contains unparseable TOML | All group output must be comments only, or rawConfig must accept the new keys |
| Class constant naming | C11: naming convention drift | Decide convention once in design phase; document in types.go |
| Perceptual density | C12: 35+ layers is a wash | Design family groupings to fuse into ~10 perceptual units, not 35 threads |
2026-03-26 16:54:31 +01:00
---
2026-03-27 08:25:02 +01:00
## Backward Compatibility Summary (v1.1 → v1.2)
2026-03-24 22:36:28 +01:00
2026-03-27 08:25:02 +01:00
| Change Type | Impact on v1.1 User Configs | Mitigation |
|-------------|----------------------------|------------|
| New built-in classes added | None — absent TOML keys silently default; existing overrides unaffected | Safe |
| Existing built-in Hz values changed | User overrides silently re-apply old v1.1 Hz values, masking the change | Freeze v1.1 Hz range; do not reassign existing classes |
| New top-level TOML key added (e.g., `[families]`) | v1.1 binary rejects config with new key via `Undecoded()` | Acceptable since user controls which binary they run |
| Auto-assign range shifted | User custom classes get different Hz values than before | Announce in changelog; update `autoAssignFreq` constants and document |
| `[[rules]]` schema extended (port range fields) | Existing rules without new fields: no change (pointer types are nil = absent) | Backward-compatible if new fields are optional pointers |
| Class string values renamed | TOML `[sounds.old-name]` silently produces "unknown class" warning (not error) | Do not rename existing class strings |
---
## v1.1 Pitfalls (Retained)
The following pitfalls from v1.1 research remain valid and fully resolved in the codebase. They are retained in condensed form for reference. See the original v1.1 entries for full detail.
### Pitfall A1: TOML Unmarshal Silently Overwrites Defaults With Zero Values
Use pointer fields (`*float64`, `*string`) — implemented in `config/config.go` via `SoundOverride`.
### Pitfall A2: BurntSushi/toml Silently Ignores Typos
Use `md.Undecoded()` — implemented in `parseFile()`.
### Pitfall A3: Naive Square/Sawtooth/Triangle Produces Aliasing
Use bandlimited additive synthesis — implemented in `WaveformPresetHarmonics()`.
### Pitfall A4: Waveform String Validation Fails Silently
Normalize + validate — implemented in `parseWaveform()`.
### Pitfall A5: User Rules After Catch-All Rules Are Unreachable
User rules prepend before built-ins — implemented: user rules inserted first in merged slice.
### Pitfall A6: User-Defined Classes Have No synth.FreqConfig Entry
`addAutoFreqEntries()` handles this — implemented in `config/config.go`.
### Pitfall A7: Config Auto-Discovery Ignores XDG Variables
Use `os.UserConfigDir()` — implemented in `discoverPath()`.
### Pitfall A8: Explicit --config Flag Does Not Error on Missing File
Separate code paths for explicit vs auto-discovery — implemented in `resolvePath()`.
### Pitfall A9: User Rule Shadowed by Built-in for Same Port
User rules evaluate first — implemented via prepend ordering.
### Pitfall A10: Empty/Whitespace Class Name Is a Valid Go String
Validated in `validateRules()`.
---
## v1.0 Pitfalls (Retained, Condensed)
### Pitfall B1: `google/gopacket` (Unmaintained)
Use `gopacket/gopacket` v1.5.0.
### Pitfall B2: CGo Destroys Single Binary
Use `packetcap/go-pcap` (pure Go capture).
### Pitfall B3: CAP_NET_RAW + nosuid Filesystem
Install to `/usr/local/bin`; emit clear privilege error.
### Pitfall B4: Packet Buffer Overflow
Large capture buffer (32 MB); buffered channel between goroutines.
### Pitfall B5: ZeroCopy Packet Use-After-Free
Use `ReadPacketData()` (copying API).
### Pitfall B6: LAME Initialization Order
Call `InitParams()` before writing frames.
### Pitfall B7: PCM Sample Overflow
Synthesize in float64 [-1, 1]; clamp before int16 cast.
### Pitfall B8: Tone-per-Protocol Frequency Masking
Space protocols across register bands using musical intervals.
2026-03-24 22:36:28 +01:00
---
## Sources
2026-03-27 08:25:02 +01:00
- Direct inspection of `/home/dev/workspace/yoloyolo/synth/config.go`, `config.go`, `classify/types.go`, `synth/bank.go`, `synth/config_test.go` — codebase analysis is HIGH confidence
- [Glasberg & Moore (1990) ERB formula via Wikipedia Critical Band article](https://en.wikipedia.org/wiki/Critical_band) — critical bandwidth values at specific Hz, HIGH confidence
- [BurntSushi/toml pkg.go.dev — Undecoded() strict mode](https://pkg.go.dev/github.com/BurntSushi/toml) — unknown key validation behavior, HIGH confidence
- [BurntSushi/toml issue #47: default value behavior](https://github.com/BurntSushi/toml/issues/47) — TOML zero-value overwrite behavior, HIGH confidence
- [Evanjones.ca: The Unreasonable Effectiveness of Linear Search](https://www.evanjones.ca/linear-search.html) — linear search competitive with map for N < ~100, MEDIUM confidence
- [vitalvas.com: Slice vs Map Performance in Golang (2025)](https://blog.vitalvas.com/post/2025/10/03/slice-vs-map-performance-in-golang/) — map outperforms slice at N > 10, MEDIUM confidence; 40-rule estimate of 160 ns is first-principles, not benchmarked
- [QSC Blog: Auditory Masking and its Effect on Perception](https://blogs.qsc.com/live-sound/auditory-masking-and-its-effect-on-our-perception-of-sound/) — masking principles, MEDIUM confidence
- [Wikipedia: List of TCP and UDP port numbers](https://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers) — port numbers for IMAP, POP3, LDAP, RDP, SIP, SNMP, Syslog, HIGH confidence
- [RF Wireless World: Well-Known Port Numbers](https://www.rfwireless-world.com/terminology/well-known-port-numbers) — port reference, MEDIUM confidence
2026-03-24 22:36:28 +01:00
---
2026-03-27 08:25:02 +01:00
*Pitfalls research for: NetSynth v1.2 — Extended protocol coverage, grouped sound families*
*Updated: 2026-03-27*