docs: complete project research
This commit is contained in:
+366
-252
@@ -1,208 +1,332 @@
|
||||
# Architecture Patterns
|
||||
|
||||
**Domain:** Network traffic sonification CLI (Go) — v1.1 Custom Sound Mappings
|
||||
**Researched:** 2026-03-26
|
||||
**Confidence:** HIGH — based on direct code inspection of the existing v1.0 codebase
|
||||
**Domain:** Network traffic sonification CLI (Go) — v1.2 Extended Protocol Coverage with Grouped Families
|
||||
**Researched:** 2026-03-26 (v1.0/v1.1), updated 2026-03-27 (v1.2 grouped protocol families)
|
||||
**Confidence:** HIGH — based on direct code inspection of the shipped v1.1 codebase
|
||||
|
||||
---
|
||||
|
||||
## v1.1 Integration Overview
|
||||
## v1.2 Integration Overview
|
||||
|
||||
This document supersedes the pre-implementation v1.0 architecture research. It is grounded in the actual codebase (3,254 lines, 6 packages) and answers: what changes, what's new, and in what order.
|
||||
This document supersedes the pre-implementation v1.0/v1.1 architecture research. It is grounded in the actual shipped v1.1 codebase (~4,675 lines, 7 packages) and answers specifically: how does the *group concept* thread through classify, synth, config, and CLI? What changes, what stays, and in what order?
|
||||
|
||||
---
|
||||
|
||||
## Existing Package Map (v1.0 Baseline)
|
||||
## Current Package Map (v1.1 Baseline — the starting point)
|
||||
|
||||
```
|
||||
cmd/netsynth/main.go CLI, pipeline wiring, Cobra flags
|
||||
capture/ go-pcap live capture + pcap file reader + BPF
|
||||
classify/
|
||||
types.go TrafficClass, ClassifiedPacket, WindowSnapshot
|
||||
types.go TrafficClass (string type), ClassifiedPacket, WindowSnapshot, AllClasses()
|
||||
classifier.go NewClassifier(rules []Rule) — first-match-wins
|
||||
rules.go DefaultRules []Rule (12 hardcoded rules)
|
||||
rules.go DefaultRules []Rule (12 built-in rules)
|
||||
aggregate/
|
||||
window.go 500ms time-windowed snapshot accumulation
|
||||
synth/
|
||||
config.go ClassFreqConfigs — fixed map[TrafficClass]FreqConfig
|
||||
oscillator.go Phase-accumulator oscillator — sine only
|
||||
layer.go EMA amplitude smoothing per layer
|
||||
bank.go NewBank(tau) — one Layer per AllClasses()
|
||||
config.go FreqConfig{BaseHz, Harmonics, Pan, WaveformType}, ClassFreqConfigs map
|
||||
oscillator.go Phase-accumulator oscillator — sine+additive harmonics
|
||||
layer.go EMA amplitude smoothing per Layer
|
||||
bank.go NewBank(tau, cfgs map) — one Layer per config entry
|
||||
mixer.go PanGains, StereoFramesToInt16Bytes
|
||||
encode/
|
||||
mp3.go RunSynthesis(snapshots, path) — NewBank + EncodeMP3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What v1.1 Adds
|
||||
|
||||
Three independent but related features:
|
||||
|
||||
1. **TOML config file** — override frequencies and waveforms per built-in class
|
||||
2. **Additional waveforms** — square, sawtooth, triangle alongside existing sine
|
||||
3. **User-defined classification rules** — TOML-defined rules prepended before DefaultRules
|
||||
|
||||
---
|
||||
|
||||
## Integration Point Analysis
|
||||
|
||||
### Feature 1: TOML Config File
|
||||
|
||||
**Where config is consumed today:** `synth/config.go` holds a package-level `var ClassFreqConfigs`. `synth/bank.go:NewBank()` reads it directly with `ClassFreqConfigs[class]`. No config is passed through `encode.RunSynthesis` or `main.go`.
|
||||
|
||||
**Required change:** `NewBank` must accept a config parameter instead of reading the global. `encode.RunSynthesis` must accept and forward a config. `main.go` must load config from disk and pass it in.
|
||||
|
||||
**New package: `config/`**
|
||||
|
||||
This package does not exist yet in the codebase (the pre-implementation research anticipated it but it was deferred). It should own:
|
||||
- TOML struct definitions
|
||||
- File discovery logic (auto-detect `./netsynth.toml`, then `~/.config/netsynth/config.toml`)
|
||||
- Merging: loaded config overlays defaults, does not replace them entirely
|
||||
|
||||
```
|
||||
config/
|
||||
config.go Config struct, Load(path string) (*Config, error)
|
||||
defaults.go DefaultConfig() — wraps existing ClassFreqConfigs values
|
||||
config.go TOML parse, merge, validate, auto-freq assignment, PrintConfig
|
||||
encode/
|
||||
mp3.go RunSynthesis(snapshots, path, freqCfgs) — NewBank + EncodeMP3
|
||||
```
|
||||
|
||||
**TOML struct shape:**
|
||||
|
||||
```toml
|
||||
[[class]]
|
||||
name = "HTTPS"
|
||||
frequency_hz = 200.0
|
||||
waveform = "sawtooth"
|
||||
|
||||
[[class]]
|
||||
name = "myservice" # user-defined class (Feature 3)
|
||||
frequency_hz = 350.0
|
||||
waveform = "triangle"
|
||||
```
|
||||
|
||||
The `Config` struct passed into `NewBank` should merge with `ClassFreqConfigs`:
|
||||
|
||||
```go
|
||||
// config/config.go
|
||||
type ClassConfig struct {
|
||||
Name string `toml:"name"`
|
||||
FrequencyHz float64 `toml:"frequency_hz"`
|
||||
Waveform string `toml:"waveform"` // "sine" | "square" | "sawtooth" | "triangle"
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
Classes []ClassConfig `toml:"class"`
|
||||
Rules []RuleConfig `toml:"rule"` // Feature 3
|
||||
}
|
||||
```
|
||||
|
||||
**TOML library:** Use `github.com/BurntSushi/toml`. It is the de-facto standard for TOML in Go (used by Hugo, dep, buf, etc.). Already a transitive dependency in many Go module graphs. Provides struct-tag-based decode, good error messages.
|
||||
|
||||
---
|
||||
|
||||
### Feature 2: Additional Waveforms
|
||||
## What v1.2 Adds
|
||||
|
||||
**Where waveform logic lives today:** `synth/oscillator.go:Advance()` — pure sine via `math.Sin`. The `HarmonicDef.Ratio` and `HarmonicDef.Amplitude` fields are stored in `FreqConfig.Harmonics` but the waveform function is hardcoded.
|
||||
Two related but separable capabilities:
|
||||
|
||||
**Required change:** `Oscillator.Advance` must dispatch on a waveform type. Two clean approaches:
|
||||
1. **More built-in protocol rules** — extended DefaultRules covering Mail, Remote Access, Database, Discovery, File Transfer, VoIP, etc.
|
||||
2. **Group concept** — related protocols share a recognizable sound family (shared base frequency, detuned members, optional shared waveform character)
|
||||
|
||||
**Option A (recommended): Waveform enum on Oscillator**
|
||||
These must be designed together because the group concept directly affects frequency allocation, and frequency allocation directly affects DefaultRules ordering decisions.
|
||||
|
||||
Add a `waveform` field to `Oscillator`. `Advance` switches on it. `NewOscillator` gains a waveform parameter.
|
||||
---
|
||||
|
||||
## The Group Concept: What It Means Architecturally
|
||||
|
||||
A "group" means: protocols in the same family share a musical neighborhood. Listeners learn "low rumble = infrastructure traffic" and "mid-range shimmer = web traffic" without needing to identify individual tones.
|
||||
|
||||
The group concept requires:
|
||||
- A **group identifier** stored somewhere accessible by both classify (for `--print-config` ordering) and synth (for within-group detuning computation)
|
||||
- A **group base frequency** from which individual protocol frequencies are derived by small detuning offsets
|
||||
- **Within-group waveform consistency** — all members of a group use the same waveform type so the family sounds recognizable even as individual pitches differ
|
||||
|
||||
The key architectural question is: *where does the group concept live?*
|
||||
|
||||
---
|
||||
|
||||
## Decision: Group Lives in `synth/config.go`, Not in `classify/`
|
||||
|
||||
### Option A: Group field on `classify.Rule` or new `TrafficClassGroup` type in `classify`
|
||||
|
||||
Adding `Group string` to `classify.Rule` would put group information at the classification boundary, visible from the capture pipeline forward. But:
|
||||
- The classifier does not care about groups — it only cares about protocol matching
|
||||
- `AllClasses()` in `classify/types.go` already returns ordered class names without needing sound semantics
|
||||
- Pollutes the `classify` package with audio-domain concerns
|
||||
|
||||
### Option B: Group field in `synth.FreqConfig` (recommended)
|
||||
|
||||
`FreqConfig` already owns all sound parameters (frequency, harmonics, pan, waveform). Adding a `Group string` field keeps group logic inside the audio domain. The `classify` package stays clean — it only knows about `TrafficClass` names.
|
||||
|
||||
```go
|
||||
type Waveform int
|
||||
|
||||
const (
|
||||
WaveformSine Waveform = iota
|
||||
WaveformSquare
|
||||
WaveformSawtooth
|
||||
WaveformTriangle
|
||||
)
|
||||
|
||||
type Oscillator struct {
|
||||
phase float64
|
||||
freq float64
|
||||
sr float64
|
||||
waveform Waveform
|
||||
}
|
||||
|
||||
func (o *Oscillator) sampleAt(phase, ratio float64) float64 {
|
||||
p := phase * float64(ratio)
|
||||
p -= math.Floor(p) // wrap to [0, 1)
|
||||
switch o.waveform {
|
||||
case WaveformSquare:
|
||||
if p < 0.5 { return 1.0 }
|
||||
return -1.0
|
||||
case WaveformSawtooth:
|
||||
return 2.0*p - 1.0
|
||||
case WaveformTriangle:
|
||||
if p < 0.5 { return 4.0*p - 1.0 }
|
||||
return 3.0 - 4.0*p
|
||||
default: // WaveformSine
|
||||
return math.Sin(2 * math.Pi * p)
|
||||
}
|
||||
// synth/config.go
|
||||
type FreqConfig struct {
|
||||
BaseHz float64
|
||||
Harmonics []HarmonicDef
|
||||
Pan float64
|
||||
WaveformType WaveformType
|
||||
Group string // NEW: e.g., "mail", "web", "remote-access", "" for ungrouped
|
||||
}
|
||||
```
|
||||
|
||||
**Option B: Function field on Oscillator**
|
||||
**`classify.TrafficClass` does NOT get a Group field.** The group is a sound-design concept, not a classification concept. Classification says "this is SMTP"; synthesis says "SMTP belongs to the Mail group at 440 Hz + 0 cents detuning."
|
||||
|
||||
Store `waveFn func(phase float64) float64`. More flexible but harder to serialize/configure.
|
||||
### Why this is the right boundary
|
||||
|
||||
Option A is preferred because waveform type maps cleanly to the TOML `waveform` string field without reflection tricks.
|
||||
| Package | Knows About | Does Not Know About |
|
||||
|---------|-------------|---------------------|
|
||||
| `classify` | Protocol, port, traffic class name | Group, frequency, waveform |
|
||||
| `synth` | Frequency, harmonics, pan, waveform, group | Protocol, port |
|
||||
| `config` | Merges both sides; reads TOML; owns PrintConfig | Packet capture |
|
||||
|
||||
**`FreqConfig` change:** Add `Waveform` field:
|
||||
---
|
||||
|
||||
## Frequency Allocation Strategy for Groups
|
||||
|
||||
### Current allocation (v1.1)
|
||||
|
||||
The 14 existing classes span 65 Hz to 1047 Hz in a roughly logarithmic spread:
|
||||
|
||||
```
|
||||
ICMP: 65 Hz (sub-bass)
|
||||
DNS: 110 Hz
|
||||
HTTPS: 175 Hz
|
||||
HTTP: 220 Hz
|
||||
SSH: 330 Hz
|
||||
SMTP: 440 Hz
|
||||
NTP: 520 Hz
|
||||
DHCP: 600 Hz
|
||||
other-TCP: 700 Hz
|
||||
other-UDP: 780 Hz
|
||||
unknown-1: 862 Hz
|
||||
unknown-2: 920 Hz
|
||||
unknown-3: 981 Hz
|
||||
unknown-4: 1047 Hz
|
||||
auto-range: 1200-2350 Hz (FNV-32a hash for user-defined classes)
|
||||
```
|
||||
|
||||
### v1.2 group-based allocation
|
||||
|
||||
New protocols need frequency homes. With ~25-35 total built-in classes after expansion, maintaining individual hand-tuned frequencies becomes fragile and collision-prone. The group model solves this by assigning each group a frequency band, then placing members within that band at small detuning offsets.
|
||||
|
||||
**Recommended group bands (within the 65-1100 Hz range already owned by built-ins):**
|
||||
|
||||
| Group | Band Center | Role | Protocols |
|
||||
|-------|-------------|------|-----------|
|
||||
| Infrastructure | 65-130 Hz | Core network services (barely heard, always present) | ICMP, NTP, DHCP, mDNS/SSDP |
|
||||
| Web | 175-280 Hz | HTTP-family traffic | HTTP, HTTPS, HTTP/3, WebSocket |
|
||||
| Mail | 380-480 Hz | Email protocols | SMTP, IMAP, POP3, SMTP-submission |
|
||||
| Remote Access | 300-360 Hz | Interactive sessions | SSH, RDP, Telnet, VNC |
|
||||
| Database | 500-580 Hz | Backend data stores | MySQL, PostgreSQL, Redis, MongoDB |
|
||||
| File Transfer | 620-700 Hz | File movement protocols | FTP, SFTP, FTPS, SMB, NFS |
|
||||
| VoIP / Streaming | 720-820 Hz | Real-time media | SIP, RTP, RTSP |
|
||||
| DNS | 110 Hz | Singleton — already well-placed | DNS (UDP+TCP) |
|
||||
| Unknown | 850-1100 Hz | Auto-bucketed unrecognized traffic | unknown-1 through unknown-4 |
|
||||
|
||||
**Within-group detuning model:** Each group has a `GroupBaseHz`. Members are offset by small cent-based detuning or fixed Hz offsets:
|
||||
|
||||
```
|
||||
Mail group base: 420 Hz
|
||||
SMTP: 420 Hz (group base, no detune)
|
||||
IMAP: 430 Hz (+10 Hz)
|
||||
POP3: 440 Hz (+20 Hz)
|
||||
SMTP-submission: 450 Hz (+30 Hz)
|
||||
```
|
||||
|
||||
Offsets of 10-30 Hz are audible as distinct pitches but harmonically related enough to sound like a family. Do NOT use harmonic ratios (2x, 3x) for detuning — that would place group members an octave or fifth apart, destroying the "family sound" effect.
|
||||
|
||||
**This is a design constraint, not a library feature.** The detuning is expressed as explicit `BaseHz` values in `ClassFreqConfigs`. There is no runtime detuning computation needed — it is baked into the default config map.
|
||||
|
||||
### Frequency rebalancing required
|
||||
|
||||
Adding ~15-20 new protocols means the existing 65-1100 Hz range must be replanned. The existing classes should be reassigned to group-coherent frequencies even if this moves them from v1.1 values. This is a conscious breaking change to the default sound, acceptable in a minor version bump where the goal is improved audio design.
|
||||
|
||||
The `unknown-1..4` range at 850-1047 Hz is preserved as-is — those are intentionally dissonant and serve a distinct purpose.
|
||||
|
||||
---
|
||||
|
||||
## Component-by-Component Changes
|
||||
|
||||
### `classify/rules.go` — Extend DefaultRules
|
||||
|
||||
**What changes:** Add ~15-20 new `Rule` entries for the expanded protocol set.
|
||||
|
||||
**What does NOT change:** `Rule` struct, `NewClassifier`, the first-match-wins algorithm.
|
||||
|
||||
**Required care:** The catch-all rules (`DstPort: 0` for TCP and UDP) must remain last. New protocol-specific rules must be inserted before these catch-alls. `DefaultRules` is an ordered slice — insertion order matters.
|
||||
|
||||
```go
|
||||
// New rules inserted before catch-alls, e.g.:
|
||||
{Protocol: "tcp", DstPort: 143, Class: ClassIMAP},
|
||||
{Protocol: "tcp", DstPort: 110, Class: ClassPOP3},
|
||||
{Protocol: "tcp", DstPort: 587, Class: ClassSMTPSubmit},
|
||||
{Protocol: "tcp", DstPort: 3306, Class: ClassMySQL},
|
||||
{Protocol: "tcp", DstPort: 5432, Class: ClassPostgres},
|
||||
{Protocol: "tcp", DstPort: 6379, Class: ClassRedis},
|
||||
{Protocol: "tcp", DstPort: 3389, Class: ClassRDP},
|
||||
{Protocol: "tcp", DstPort: 5900, Class: ClassVNC},
|
||||
{Protocol: "udp", DstPort: 5060, Class: ClassSIP},
|
||||
{Protocol: "tcp", DstPort: 5060, Class: ClassSIP},
|
||||
// ... etc
|
||||
```
|
||||
|
||||
### `classify/types.go` — New Constants and Updated AllClasses()
|
||||
|
||||
**What changes:** Add new `TrafficClass` constants for each new protocol. Update `AllClasses()` to include them in display-sensible order (grouped by family, matching PrintConfig output order).
|
||||
|
||||
**What does NOT change:** `ClassifiedPacket`, `WindowSnapshot`, the `TrafficClass` string type itself.
|
||||
|
||||
```go
|
||||
// New constants (representative subset):
|
||||
const (
|
||||
ClassIMAP TrafficClass = "IMAP"
|
||||
ClassPOP3 TrafficClass = "POP3"
|
||||
ClassSMTPSubmit TrafficClass = "SMTP-submit"
|
||||
ClassMySQL TrafficClass = "MySQL"
|
||||
ClassPostgres TrafficClass = "PostgreSQL"
|
||||
ClassRedis TrafficClass = "Redis"
|
||||
ClassMongoDB TrafficClass = "MongoDB"
|
||||
ClassRDP TrafficClass = "RDP"
|
||||
ClassVNC TrafficClass = "VNC"
|
||||
ClassTelnet TrafficClass = "Telnet"
|
||||
ClassFTP TrafficClass = "FTP"
|
||||
ClassFTPS TrafficClass = "FTPS"
|
||||
ClassSMB TrafficClass = "SMB"
|
||||
ClassNFS TrafficClass = "NFS"
|
||||
ClassSIP TrafficClass = "SIP"
|
||||
ClassHTTP3 TrafficClass = "HTTP3" // UDP 443 (QUIC)
|
||||
ClassmDNS TrafficClass = "mDNS" // UDP 5353
|
||||
ClassSSDPDisc TrafficClass = "SSDP" // UDP 1900
|
||||
// ... additional as determined by feature research
|
||||
)
|
||||
```
|
||||
|
||||
**`AllClasses()` ordering:** Grouped by family in the same order as the group bands above. This order is consumed by `config.PrintConfig` for output ordering — so the output will naturally group protocols by family with no additional logic needed in `config`.
|
||||
|
||||
### `synth/config.go` — Add Group Field and Rebalanced ClassFreqConfigs
|
||||
|
||||
**What changes:**
|
||||
|
||||
1. Add `Group string` to `FreqConfig`:
|
||||
|
||||
```go
|
||||
type FreqConfig struct {
|
||||
BaseHz float64
|
||||
Harmonics []HarmonicDef
|
||||
Pan float64
|
||||
Waveform Waveform // NEW: defaults to WaveformSine
|
||||
BaseHz float64
|
||||
Harmonics []HarmonicDef
|
||||
Pan float64
|
||||
WaveformType WaveformType
|
||||
Group string // NEW: sound family identifier; "" = ungrouped
|
||||
}
|
||||
```
|
||||
|
||||
`NewLayer` passes `cfg.Waveform` to `NewOscillator`. `NewOscillator` signature changes to accept the waveform.
|
||||
2. Add `ClassFreqConfigs` entries for all new protocols with group-coherent frequencies and within-group detuning.
|
||||
|
||||
**What does NOT change:** `HarmonicDef`, `EMAAlpha`, `Layer.UpdateTarget`, `Layer.AdvanceSample`, `OscillatorBank.RenderWindow`, `mixer.go`, `encode/mp3.go`. The waveform change is contained to `oscillator.go` and the `FreqConfig` struct.
|
||||
3. Update existing class frequencies to align with the group bands (frequency rebalancing). This is the highest-risk change in v1.2 because it alters the default sound of existing classes.
|
||||
|
||||
4. Update `NumLayers` constant from 14 to the new total (e.g., 30-35 depending on final protocol list):
|
||||
|
||||
```go
|
||||
const (
|
||||
NumLayers = 32 // updated count; GainPerLayer recomputed automatically
|
||||
GainPerLayer = 1.0 / float64(NumLayers)
|
||||
)
|
||||
```
|
||||
|
||||
**What does NOT change:** `FreqConfig` field types (only addition), `WaveformType`, `HarmonicDef`, `WaveformPresetHarmonics`, `SampleRate`, `WindowMs`, `SamplesPerWindow`, `WhisperFloor`. The `Group` field is metadata — it has no effect on synthesis math.
|
||||
|
||||
**Pan assignment:** With 30+ layers, the current hand-tuned pan positions become crowded. Recommendation: assign pan by group position (infrastructure = center, web = slight left, mail = slight right, etc.) rather than per-protocol. This creates a spatial "map" of the network that reinforces the group concept aurally.
|
||||
|
||||
### `config/config.go` — PrintConfig Group Ordering
|
||||
|
||||
**What changes:** `PrintConfig` currently emits classes in `AllClasses()` order. With `AllClasses()` updated to group-coherent order, `PrintConfig` output will naturally be grouped. No structural change to `PrintConfig` is required.
|
||||
|
||||
**Optional enhancement:** Emit a group comment header between group sections:
|
||||
|
||||
```toml
|
||||
# --- Infrastructure ---
|
||||
# ICMP -- 65.0 Hz (default)
|
||||
[sounds.ICMP]
|
||||
frequency = 65.0
|
||||
waveform = "sine"
|
||||
|
||||
# NTP -- 110.0 Hz (default)
|
||||
[sounds.NTP]
|
||||
...
|
||||
|
||||
# --- Web ---
|
||||
# HTTP -- 175.0 Hz (default)
|
||||
[sounds.HTTP]
|
||||
...
|
||||
```
|
||||
|
||||
This requires `PrintConfig` to detect group transitions in the class ordering and emit a comment. The `Group` field on `FreqConfig` provides this information. Implementation: iterate `AllClasses()`, track previous group, emit a comment when group changes.
|
||||
|
||||
**What does NOT change:** TOML schema shape (no `group` key in `[sounds.*]` blocks — groups are not user-configurable in v1.2), `SoundOverride` struct, `RawRule`, `LoadResult`, `Load`, `merge`, `addAutoFreqEntries`.
|
||||
|
||||
**Group field in TOML config:** Users should NOT be able to override the `group` of a class. Group is a built-in design decision, not a user parameter. The `SoundOverride` struct stays with only `Frequency` and `Waveform` pointer fields.
|
||||
|
||||
### `synth/bank.go` — No Changes Required
|
||||
|
||||
`NewBank` already ranges over the passed-in `cfgs` map keys. Adding 20 new entries to `ClassFreqConfigs` and passing them through the existing chain requires no changes to `bank.go`. The `gainPerLayer` computation (`1.0 / float64(len(cfgs))`) automatically scales down with more layers.
|
||||
|
||||
### `encode/mp3.go` — No Changes Required
|
||||
|
||||
`RunSynthesis` already accepts the full `freqCfgs` map and passes it to `NewBank`. No signature change needed.
|
||||
|
||||
### `cmd/netsynth/main.go` — No Changes Required
|
||||
|
||||
The pipeline wiring (`config.Load` → `NewClassifier` → `encode.RunSynthesis`) is already correct. Adding more built-in classes and a group field to `FreqConfig` is transparent to `main.go`.
|
||||
|
||||
---
|
||||
|
||||
### Feature 3: User-Defined Classification Rules
|
||||
## Data Flow: Unchanged for v1.2
|
||||
|
||||
**Where rules are wired today:** `main.go` lines 111, 175 — both `runLiveMode` and `runPcapMode` call `classify.NewClassifier(classify.DefaultRules)` directly. No config is passed.
|
||||
The v1.1 data flow is correct and does not need to change:
|
||||
|
||||
**Required change:** User rules from TOML prepend before `DefaultRules`. `Classifier` already supports arbitrary `[]Rule` — `NewClassifier(rules []Rule)` is the constructor. No change to `classifier.go` itself.
|
||||
|
||||
**`RuleConfig` TOML struct:**
|
||||
|
||||
```toml
|
||||
[[rule]]
|
||||
protocol = "tcp"
|
||||
dst_port = 8443
|
||||
class = "myservice"
|
||||
```
|
||||
main.go
|
||||
└─ config.Load(configPath)
|
||||
└─ LoadResult{FreqCfgs, UserRules, ConfigPath, AutoClasses}
|
||||
└─ classify.NewClassifier(append(userRules, DefaultRules...))
|
||||
└─ encode.RunSynthesis(snapshots, path, result.FreqCfgs)
|
||||
└─ synth.NewBank(tau, freqCfgs)
|
||||
└─ one Layer per freqCfgs entry
|
||||
```
|
||||
|
||||
```go
|
||||
// config/config.go
|
||||
type RuleConfig struct {
|
||||
Protocol string `toml:"protocol"`
|
||||
DstPort uint16 `toml:"dst_port"`
|
||||
Class string `toml:"class"` // must match a name in [[class]] or a builtin class name
|
||||
}
|
||||
```
|
||||
The only change to the data flowing through this pipeline is: `freqCfgs` now contains ~30-35 entries (up from 14), each `FreqConfig` has a new `Group` field (ignored by synthesis, used only by `PrintConfig`).
|
||||
|
||||
**Merging in main.go:**
|
||||
---
|
||||
|
||||
```go
|
||||
userRules := config.ToClassifyRules(cfg.Rules) // []classify.Rule
|
||||
allRules := append(userRules, classify.DefaultRules...)
|
||||
classifier := classify.NewClassifier(allRules)
|
||||
```
|
||||
## Integration Points Summary
|
||||
|
||||
**New `TrafficClass` values:** User-defined classes in TOML produce new `TrafficClass` string values (e.g., `"myservice"`). `AllClasses()` in `classify/types.go` is currently a hardcoded slice. For user-defined classes, `AllClasses()` cannot be the source of truth for bank layer construction. `NewBank` must instead iterate over whatever classes have a `FreqConfig` entry.
|
||||
|
||||
This is a critical integration point: `bank.go:NewBank` currently ranges over `classify.AllClasses()`. If user classes can appear, `NewBank` must accept the full config map and range over that instead.
|
||||
| Integration Point | Change Type | Risk |
|
||||
|-------------------|-------------|------|
|
||||
| `classify/types.go` — new constants + `AllClasses()` | Additive | LOW |
|
||||
| `classify/rules.go` — extended `DefaultRules` | Additive (new entries before catch-alls) | MEDIUM (order matters) |
|
||||
| `synth/config.go` — `Group` field on `FreqConfig` | Additive (new field) | LOW (no synthesis impact) |
|
||||
| `synth/config.go` — `ClassFreqConfigs` entries for new protocols | Additive | LOW |
|
||||
| `synth/config.go` — `ClassFreqConfigs` frequency rebalancing for existing protocols | Modifying defaults | HIGH (changes default audio output) |
|
||||
| `synth/config.go` — `NumLayers` constant update | Single constant | MEDIUM (affects `GainPerLayer`) |
|
||||
| `config/config.go` — group-header comment in `PrintConfig` | Optional enhancement | LOW (isolated to string output) |
|
||||
| `synth/bank.go` | No change | — |
|
||||
| `encode/mp3.go` | No change | — |
|
||||
| `cmd/netsynth/main.go` | No change | — |
|
||||
|
||||
---
|
||||
|
||||
@@ -210,167 +334,157 @@ This is a critical integration point: `bank.go:NewBank` currently ranges over `c
|
||||
|
||||
### New
|
||||
|
||||
| Component | Location | Purpose |
|
||||
|-----------|----------|---------|
|
||||
| `config` package | `config/config.go` | TOML struct, `Load()`, file discovery, merge with defaults |
|
||||
| `config/defaults.go` | optional split | `DefaultConfig()` wrapping existing `ClassFreqConfigs` values |
|
||||
None — no new packages or files are required.
|
||||
|
||||
### Modified
|
||||
|
||||
| Component | Change | Impact |
|
||||
|-----------|--------|--------|
|
||||
| `synth/oscillator.go` | Add `Waveform` type + `waveform` field; dispatch in `Advance` | Self-contained; no caller signature breaks except `NewOscillator` |
|
||||
| `synth/config.go` | Add `Waveform Waveform` field to `FreqConfig`; default to `WaveformSine` | Requires `NewLayer` to pass waveform to `NewOscillator` |
|
||||
| `synth/layer.go` | Pass `cfg.Waveform` to `NewOscillator` | One-line change |
|
||||
| `synth/bank.go` | Accept `map[classify.TrafficClass]FreqConfig` param instead of reading global; range over param keys not `AllClasses()` | Decouples bank from global; enables user classes |
|
||||
| `encode/mp3.go` | Accept `*config.Config` or merged `FreqConfig` map; pass to `NewBank` | Thin forwarding change |
|
||||
| `cmd/netsynth/main.go` | Add `--config` flag; load config; prepend user rules; pass config to `RunSynthesis` | Touches both `runLiveMode` and `runPcapMode` |
|
||||
| `classify/rules.go` | No change — `DefaultRules` stays as the fallback | Unchanged |
|
||||
| `classify/classifier.go` | No change — already accepts `[]Rule` | Unchanged |
|
||||
| `classify/types.go` | `AllClasses()` may need a note that it returns only builtins; bank no longer relies on it | Low risk; document only |
|
||||
| Component | What Changes | What Stays Same |
|
||||
|-----------|--------------|-----------------|
|
||||
| `classify/types.go` | New `TrafficClass` constants; `AllClasses()` extended + reordered by group | `ClassifiedPacket`, `WindowSnapshot`, string type |
|
||||
| `classify/rules.go` | New `Rule` entries for all new protocols | `Rule` struct, match algorithm, catch-all placement |
|
||||
| `synth/config.go` | `Group` field on `FreqConfig`; new entries in `ClassFreqConfigs`; existing frequency rebalancing; `NumLayers` updated | All synthesis constants, waveform types, `HarmonicDef` |
|
||||
| `config/config.go` | Optional: group-header comments in `PrintConfig` | All loading, parsing, merge, validation logic |
|
||||
|
||||
---
|
||||
|
||||
## Data Flow Changes
|
||||
## Build Order
|
||||
|
||||
### v1.0 Flow (config hardcoded)
|
||||
The following order ensures each step is independently testable.
|
||||
|
||||
```
|
||||
main.go
|
||||
└─ classify.NewClassifier(classify.DefaultRules)
|
||||
└─ encode.RunSynthesis(snapshots, path)
|
||||
└─ synth.NewBank(1.0)
|
||||
└─ ClassFreqConfigs[class] ← global, hardcoded
|
||||
```
|
||||
### Step 1: Define new TrafficClass constants and extend AllClasses()
|
||||
|
||||
### v1.1 Flow (config injected)
|
||||
**Files:** `classify/types.go` only.
|
||||
|
||||
```
|
||||
main.go
|
||||
└─ config.Load(configPath) ← NEW: resolve path, parse TOML, merge defaults
|
||||
└─ cfg *config.Config
|
||||
└─ classify.NewClassifier(
|
||||
append(config.ToClassifyRules(cfg.Rules), classify.DefaultRules...)
|
||||
) ← user rules prepend built-ins
|
||||
└─ encode.RunSynthesis(snapshots, path, cfg.FreqConfigs())
|
||||
└─ synth.NewBank(1.0, freqConfigs) ← map passed in, not read from global
|
||||
└─ freqConfigs[class] ← merged: user overrides + defaults
|
||||
```
|
||||
Add all new protocol constants. Update `AllClasses()` to include them in group-coherent order. Tests: verify `AllClasses()` returns the expected slice with all new members.
|
||||
|
||||
---
|
||||
No impact on synthesis, config, or main yet — new classes simply have no rules or FreqConfig entries yet, which is harmless.
|
||||
|
||||
## Suggested Build Order
|
||||
### Step 2: Extend DefaultRules for new protocols
|
||||
|
||||
The following order minimizes integration risk. Each step is independently testable before the next begins.
|
||||
**Files:** `classify/rules.go` only.
|
||||
|
||||
### Step 1: Waveform types in `synth/oscillator.go`
|
||||
Add new `Rule` entries before the catch-alls. Tests: `NewClassifier(DefaultRules).Classify(pkt)` for each new protocol's port returns the expected new class. Confirm catch-alls still fire for unrecognized traffic.
|
||||
|
||||
No external dependencies. Pure math. Testable with golden-sample unit tests (square wave sample at phase 0.25 should be 1.0, etc.). Does not affect `Layer`, `Bank`, or `encode` yet.
|
||||
Dependency: Step 1 must complete first (new constants must exist).
|
||||
|
||||
**Files changed:** `synth/oscillator.go` only.
|
||||
### Step 3: Add Group field to FreqConfig
|
||||
|
||||
### Step 2: Wire `Waveform` through `FreqConfig` and `Layer`
|
||||
**Files:** `synth/config.go` only.
|
||||
|
||||
Add `Waveform` to `FreqConfig`. Update `NewLayer` to pass it to `NewOscillator`. `ClassFreqConfigs` entries default to `WaveformSine` (zero value — valid if `WaveformSine = 0`).
|
||||
Add `Group string` to `FreqConfig`. Set `Group` on all existing 14 `ClassFreqConfigs` entries. Existing tests continue to pass (new field has zero value by default; synthesis ignores it). No callers need updating.
|
||||
|
||||
Existing tests continue to pass without modification since all existing configs use the zero-value waveform.
|
||||
This step is independent of Steps 1 and 2.
|
||||
|
||||
**Files changed:** `synth/config.go`, `synth/layer.go`.
|
||||
### Step 4: Add ClassFreqConfigs entries for new protocols (without frequency rebalancing)
|
||||
|
||||
### Step 3: Decouple `NewBank` from the global
|
||||
**Files:** `synth/config.go` only.
|
||||
|
||||
Change `NewBank(tau float64)` to `NewBank(tau float64, cfgs map[classify.TrafficClass]FreqConfig)`. Update `encode/mp3.go:RunSynthesis` to pass `synth.ClassFreqConfigs` as default.
|
||||
Add `FreqConfig` entries for each new protocol using placeholder frequencies that avoid the 65-1100 Hz range (use 2400-4000 Hz temporarily). This makes the end-to-end pipeline work for new classes without disrupting existing audio.
|
||||
|
||||
At this point the system is functionally identical to v1.0 but `NewBank` no longer reads a global.
|
||||
Tests: verify `ClassFreqConfigs` contains entries for all constants from `classify.AllClasses()`.
|
||||
|
||||
**Files changed:** `synth/bank.go`, `encode/mp3.go`.
|
||||
Dependency: Step 3 must complete first (Group field must exist). Step 1 must complete first (constants must exist).
|
||||
|
||||
### Step 4: `config` package — TOML structs and file discovery
|
||||
### Step 5: Frequency rebalancing — reassign all ClassFreqConfigs to group-coherent values
|
||||
|
||||
Implement `config.Load()`, file discovery, and the `DefaultConfig()` function that wraps `synth.ClassFreqConfigs`. No TOML parsing yet — start with the struct definitions and the merge logic.
|
||||
**Files:** `synth/config.go` only.
|
||||
|
||||
Add `github.com/BurntSushi/toml` dependency (`go get`).
|
||||
This is the riskiest step. Assign final group-coherent frequencies to all protocols (existing and new) following the band design above. Update `NumLayers` to the final count.
|
||||
|
||||
**Files added:** `config/config.go`.
|
||||
Test strategy: write a test that verifies all entries in `ClassFreqConfigs` have `BaseHz` in a specified range per group; verify no two entries have identical `BaseHz`; verify `NumLayers` matches `len(ClassFreqConfigs)`.
|
||||
|
||||
### Step 5: `--config` flag and user rule merging in `main.go`
|
||||
Manual validation: run `netsynth --read testfile.pcap` with varied traffic and listen. This step requires subjective audio evaluation that tests cannot replace.
|
||||
|
||||
Wire `config.Load()` into `run()`. Pass user rules to both `runLiveMode` and `runPcapMode`. Pass merged `FreqConfig` map to `RunSynthesis`.
|
||||
Dependency: Steps 3 and 4 must complete first.
|
||||
|
||||
At this point a minimal TOML config (empty file, or `[[rule]]` only) can be validated end-to-end.
|
||||
### Step 6: PrintConfig group-header comments (optional polish)
|
||||
|
||||
**Files changed:** `cmd/netsynth/main.go`.
|
||||
**Files:** `config/config.go` only.
|
||||
|
||||
### Step 6: Custom frequency and waveform overrides in config
|
||||
Add group transition detection in `PrintConfig`. When the `Group` field changes between consecutive `AllClasses()` entries, emit a `# --- GroupName ---` comment.
|
||||
|
||||
Implement the `[[class]]` TOML section parsing. Add `FreqConfigs()` method to `Config` that returns the merged map (user overrides applied over defaults). Write table-driven tests: "TOML sets HTTPS to 200 Hz sawtooth, bank layer for HTTPS uses 200 Hz sawtooth."
|
||||
Tests: verify `PrintConfig` output contains group header comments; verify ungrouped classes have no group header; verify user-defined classes (no group field) are not preceded by a group header.
|
||||
|
||||
**Files changed:** `config/config.go`.
|
||||
|
||||
### Step 7: User-defined classes end-to-end
|
||||
|
||||
Support `[[class]]` entries with names not in `classify.AllClasses()`. These become new `TrafficClass` values. User `[[rule]]` entries pointing to these classes are prepended to `DefaultRules`. The bank creates layers for all classes in the merged `FreqConfig` map.
|
||||
|
||||
This step requires the most cross-package coordination but by this point each piece is already in place.
|
||||
|
||||
**Files changed:** `config/config.go`, `cmd/netsynth/main.go` (verification that unknown class names don't panic).
|
||||
|
||||
---
|
||||
|
||||
## Component Boundaries After v1.1
|
||||
|
||||
| Component | Responsibility | Communicates With |
|
||||
|-----------|---------------|-------------------|
|
||||
| `config` | TOML parsing, file discovery, merge logic, `DefaultConfig()` | `synth` (FreqConfig type), `classify` (Rule type) |
|
||||
| `synth/oscillator` | Phase-accumulator for sine/square/sawtooth/triangle | Used by `Layer` |
|
||||
| `synth/bank` | Accepts freq config map, constructs one `Layer` per entry | `encode` passes config map in |
|
||||
| `encode` | Receives config map from `main`, passes to `NewBank` | Thin pass-through |
|
||||
| `cmd/netsynth/main` | Loads config, merges rules, wires all stages | All packages |
|
||||
| `classify` | Rules engine (unchanged); `DefaultRules` stays as package-level var | `main` constructs with merged rules |
|
||||
Dependency: Steps 3 and 5 must complete first (Group field must be populated in ClassFreqConfigs).
|
||||
|
||||
---
|
||||
|
||||
## Critical Integration Constraints
|
||||
|
||||
### `AllClasses()` Is Not the Source of Truth for Bank Construction
|
||||
### NumLayers Must Match len(ClassFreqConfigs) for Correct Gain Scaling
|
||||
|
||||
`bank.go` currently iterates `classify.AllClasses()` to construct layers. After v1.1, the bank must iterate the keys of the `FreqConfig` map passed to it. User-defined classes will not appear in `AllClasses()`. If this is not changed, user-defined class packets will be aggregated in `WindowSnapshot.Counts` but have no corresponding layer — they will produce silence and no error.
|
||||
`synth/config.go` defines `NumLayers = 14` as a constant. `GainPerLayer = 1.0 / float64(NumLayers)`. However, `bank.go:NewBank` computes `gainPerLayer` dynamically from `len(cfgs)`:
|
||||
|
||||
**Fix:** `NewBank` iterates `maps.Keys(cfgs)` (or equivalent range over the map), not `classify.AllClasses()`.
|
||||
```go
|
||||
gainPerLayer: 1.0 / float64(len(cfgs)),
|
||||
```
|
||||
|
||||
### Class Name Validation Must Happen at Config Load Time
|
||||
This means `NumLayers` and `GainPerLayer` in `config.go` are currently informational constants, not what the bank actually uses at runtime. The bank's dynamic computation is correct. Update `NumLayers` for documentation accuracy, but the audio math is not at risk even if this is temporarily inconsistent.
|
||||
|
||||
If a `[[rule]]` references a class name that has no corresponding `[[class]]` entry and is not a builtin, the system will silently mis-classify packets into a layer that doesn't exist. Validate at `config.Load()` time: every class name in `[[rule]]` must resolve to either a builtin `TrafficClass` or a `[[class]]` entry in the same config.
|
||||
### DefaultRules Catch-Alls Must Remain Last
|
||||
|
||||
### `encode.RunSynthesis` Signature Change Is a Breaking API Change
|
||||
`classify/rules.go` ends with `{Protocol: "tcp", DstPort: 0, ...}` and `{Protocol: "udp", DstPort: 0, ...}`. All new protocol rules must be inserted before these. A rule added after the catch-alls will never fire. This constraint must be enforced by convention (comment in the file) since Go has no static ordering guarantees for slice initialization beyond the literal order.
|
||||
|
||||
`encode.RunSynthesis(snapshots []classify.WindowSnapshot, outputPath string)` will need to accept the config. If any external code (tests, future callers) uses this signature, they will break. Keep the change to a single place and update all call sites in the same commit.
|
||||
### AllClasses() Ordering Drives PrintConfig Ordering
|
||||
|
||||
`config.PrintConfig` iterates `classify.AllClasses()` to emit the `[sounds.*]` section. If `AllClasses()` is updated to group-coherent order (Step 1), `PrintConfig` output naturally reflects groups. If Step 6's group-header comments are added, they depend on `AllClasses()` presenting entries in group-sorted order. The two changes must be coordinated.
|
||||
|
||||
### Frequency Rebalancing Is a Default-Breaking Change
|
||||
|
||||
Users who rely on the default audio profile (no config file) will hear different tones for previously-familiar classes. This is an intentional design improvement but should be documented in the release notes. Users who have overridden frequencies via TOML config are unaffected (their overrides take precedence).
|
||||
|
||||
### Group Field Is Not a TOML Configurable
|
||||
|
||||
The `SoundOverride` struct in `config/config.go` must not gain a `group` field. Groups are built-in design decisions. Allowing users to reassign protocols to different groups via TOML would create inconsistencies in `PrintConfig` output ordering and the `AllClasses()` group semantics. If this is requested in a future milestone, it warrants a separate design decision.
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns to Avoid
|
||||
|
||||
### Anti-Pattern: Reading Global `ClassFreqConfigs` from Multiple Places
|
||||
### Anti-Pattern: Computing Detuning at Runtime from Group Metadata
|
||||
|
||||
If `NewBank`, `encode.RunSynthesis`, and config loading all reference the package-level `synth.ClassFreqConfigs`, the merge point becomes ambiguous. The fix (Step 3 above) centralizes config reading to one place: `config.DefaultConfig()` reads from `ClassFreqConfigs` once when building defaults; everything downstream receives the already-merged map.
|
||||
It is tempting to define group base frequencies and detune offsets as data structures, then derive individual `BaseHz` values at synthesis startup. This adds complexity with no benefit — the final `BaseHz` values are static per-class constants that belong in `ClassFreqConfigs` directly. Bake the detuning arithmetic into the initial constant assignment, document it with a comment (`// Mail group: +10 Hz from group base 420 Hz`), and move on.
|
||||
|
||||
### Anti-Pattern: Storing `Waveform` as a String Everywhere
|
||||
### Anti-Pattern: Group as a `classify.TrafficClass` Field
|
||||
|
||||
Keeping `waveform` as a `string` from TOML all the way into `Oscillator` means every advance call parses or switches on a string. Parse the string to a `Waveform` int type at config-load time. The `Oscillator` field should be a typed `Waveform`, not `string`.
|
||||
Groups are audio concepts, not classification concepts. `classify.Rule` should not know about groups. `TrafficClass` is already a string type used as a map key throughout the codebase — adding a structured type would be a larger refactor than the feature warrants. Keep groups in `synth.FreqConfig`.
|
||||
|
||||
### Anti-Pattern: User Rules Appended After DefaultRules
|
||||
### Anti-Pattern: Adding Group-Based Routing to bank.go
|
||||
|
||||
User rules must **prepend** `DefaultRules`, not append. `DefaultRules` ends with catch-all rules (`DstPort: 0`) that match any TCP or UDP packet. Appending user rules after these catch-alls means they will never be reached.
|
||||
Do not make `OscillatorBank` aware of groups for mixing purposes (e.g., summing all Mail group layers before applying gain). The current architecture mixes all layers uniformly. Group-coherent mixing is a more complex feature than v1.2 requires and changes the synthesis model. Defer if requested.
|
||||
|
||||
### Anti-Pattern: Overcrowding 65-1100 Hz with Too Many Protocols
|
||||
|
||||
At 30+ built-in classes, some protocols will be heard less often than others. Resist the urge to map rarely-seen protocols (Telnet, FTP) to prominent mid-range frequencies. Relegate low-signal protocols to the upper end of each group band so they contribute texture without dominating when not present.
|
||||
|
||||
### Anti-Pattern: Hand-Tuning Frequencies Without Listening
|
||||
|
||||
Frequency assignments must be validated by ear, not just by looking at Hz values on a spreadsheet. Schedule a listening session with a realistic pcap file after Step 5 before Step 6. The architecture makes this easy — any frequency change is a one-line edit in `ClassFreqConfigs`.
|
||||
|
||||
---
|
||||
|
||||
## Scalability Considerations
|
||||
|
||||
| Concern | With 14 Classes (v1.1) | With 32 Classes (v1.2) |
|
||||
|---------|----------------------|----------------------|
|
||||
| Gain per layer | ~7.1% of max | ~3.1% of max |
|
||||
| Frequency collisions | None | Possible if not planned |
|
||||
| AllClasses() iteration | 14 items, trivial | 32 items, still trivial |
|
||||
| PrintConfig output lines | ~60 lines | ~130 lines |
|
||||
| EMA smoothing convergence | No change | No change (per-layer, independent) |
|
||||
| NewBank construction time | Negligible | Negligible |
|
||||
|
||||
The main practical change is perceptual loudness: with 32 layers each at 3.1% gain, the ambient mix becomes quieter when many protocols are active simultaneously. This is acceptable and matches the ambient/drone aesthetic. The `WhisperFloor` mechanism ensures inactive layers contribute minimally, so sessions with only 5-6 active protocols still sound full.
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
- Direct code inspection: `synth/config.go`, `synth/oscillator.go`, `synth/bank.go`, `synth/layer.go`, `classify/classifier.go`, `classify/rules.go`, `classify/types.go`, `encode/mp3.go`, `cmd/netsynth/main.go` — HIGH confidence
|
||||
- BurntSushi/toml usage in production Go projects (Hugo, dep): MEDIUM confidence (well-known in Go ecosystem)
|
||||
- Phase-accumulator waveform synthesis formulas (square, sawtooth, triangle): HIGH confidence (standard DSP, textbook formulas)
|
||||
- Direct code inspection: `synth/config.go`, `synth/bank.go`, `synth/layer.go`, `synth/oscillator.go`, `classify/types.go`, `classify/rules.go`, `classify/classifier.go`, `config/config.go`, `encode/mp3.go`, `cmd/netsynth/main.go` — HIGH confidence
|
||||
- Musical interval theory (detuning, harmonic relationships): HIGH confidence — standard acoustic physics
|
||||
- v1.2 protocol list: determined from feature research (see FEATURES.md for rationale on which protocols to include)
|
||||
|
||||
---
|
||||
|
||||
*Architecture research for: NetSynth v1.1 — custom sound mappings integration*
|
||||
*Researched: 2026-03-26*
|
||||
*Architecture research for: NetSynth v1.2 — extended protocol coverage with grouped families*
|
||||
*Researched: 2026-03-27*
|
||||
|
||||
Reference in New Issue
Block a user